[EDP] Allow editing job binaries

https://blueprints.launchpad.net/sahara/+spec/edp-edit-job-binaries

Currently there is no way to edit a job binary. If a path needs to be changed, for example, the job binary must be deleted and a new one created. The most common use case is a situation where a user creates a job binary, runs a job, and receives an error from the job because the path does not exist. Although it is not strictly necessary, editable job binary objects would be a convenience when a user needs to correct a path or credentials.

Problem description

There are no API methods for updating a job binary object in the REST API or at the conductor level.

The only way to correct a job binary is to delete an existing one and create a new one with corrected information. Although it is possible to use the same name, the object id will be different.

If editing is allowed, a user only needs to do a single operation to make corrections. Additionally, the id is preserved so that objects which reference it will reference the corrected path.

In the general case, editing a job binary should not be a problem for a job object that references it. Once a job execution enters the “RUNNING” state, any job binary objects it references indirectly through the job object have been uploaded to the cluster for execution. Consequently, editing a job binary object will cause no errors.

There is only a small window where editing should not be allowed. This is when a job binary object is referenced by a job execution in the “PENDING” state. At this point, binaries have not yet been uploaded to the cluster and a change during this window would cause the job to run with paths other than the ones that existed at submission time.

Note, the paths of binaries used by a job execution should be recorded in the job execution. This will remove a restriction on editing of paths in a job binary that is referenced by an existing job execution. This will be done in a separate blueprint listed in the references section (similar recording of data source paths used during an execution is supported in another blueprint).

Proposed change

Add an update operation to the REST API for job binary objects. Do not allow updates for job binaries that are referenced by job executions in the “PENDING” state (this can be checked during validation).

Allow the following fields in the job binary to be edited:

  • name

  • description

  • url if the value is not an “internal-db://” path

For binaries stored in the Sahara database, the URL is generated by Sahara and should not be editable.

Add the corresponding update operation to the python-saharaclient.

Alternatives

Do nothing

Data model impact

None

REST API impact

Backward compatiblity will be maintained since this is a new endpoint.

PUT /v1.1/{tenant_id}/job-binaries/{job_binary_id}

Normal Response Code: 202 (ACCEPTED)

Errors: 400 (BAD REQUEST), 404 (NOT FOUND)

Update the indicated job-binary object

Example

request

PUT http://sahara/v1.1/{tenant_id}/job-binaries/{job_binary_id}

{
    "description": "some description",
    "name": "my.jar",
    "url": "swift://container/correct_path"
}

response

HTTP/1.1 202 ACCEPTED
Content-Type: application/json

{
    "created_at": "2015-04-08 20:48:18",
    "description": "",
    "id": "640ca841-d4d9-48a1-a838-6aa86b12520f",
    "name": "my.jar",
    "tenant_id": "33724d3bf3114ae9b8ab1c170e22926f",
    "updated_at": "2015-04-09 10:48:18",
    "url": "swift://container/correct_path"
}

Other end user impact

This operation should be added to the python-saharaclient API as well

$ sahara job-binary-update [–name NAME] [–id ID] [–json]

Deployer impact

None

Developer impact

None

Sahara-image-elements impact

None

Sahara-dashboard / Horizon impact

To take advantage of this from the Horizon UI, we would need a selectable “Edit” action for each job binary on the job binaries page

Implementation

Assignee(s)

Primary assignee:

Trevor McKay

Other contributors:

Chad Roberts

Work Items

Add REST and support methods to Sahara Add operation to python-saharaclient Add operation to job binary screens in Horizon Add to WADL in api-ref

Dependencies

This is a blueprint to store the job binary paths in the job execution object. Implementing this first will allow editing of job binaries as long as they are not in the PENDING state. Otherwise, editing will have to be disallowed for job binaries referenced by an existing job execution.

https://blueprints.launchpad.net/sahara/+spec/edp-store-binary-paths-in-job-executions

Testing

Unit tests in Sahara and python-saharaclient

Documentation Impact

Potentially any user documentation that talks about editing of other objects like templates

References

None