As sahara’s API has evolved there have been several features introduced in the form of routes and methods that could be crafted in a more consistent and predictable manner. Additionally, there are several new considerations and methodologies that can only be addressed by updating the major version of the API. This document serves as a roadmap to implement an experimental v2 API which will form the basis of the eventual stable version.
This is an umbrella specification covering many changes, there will be followup specifications to cover some of the more intricate details.
The current version of sahara’s REST API, 1.1, contains several methodologies and patterns that have created inconsistencies within the API and with respect to the API Working Group’s evolving guidelines. Many of these are due to the iterative nature of the design work, and some have been created at a time before stable guidelines existed.
Examples of inconsistencies within the current API:
In addition to resolving the inconsistencies in the API, a new version will provide an opportunity to implement features which will improve the experience for consumers of the sahara API.
Examples of features to implement in the new API:
These are just a few examples of issues which can be addressed in a new major version API implementation.
To address the creation of a new major version API, an experimental /v2 endpoint should be created. This new endpoint will be clearly marked as experimental and no contract of stability will be enforced with regards to the content of its sub-endpoints. Changes to the /v2 endpoint will be tracked through features described in this specification, and through further specifications which will be created to better describe the details of larger changes.
When all the changes to the /v2 endpoint have been made such that it has a 1:1 feature compliance with the current API, and the Python sahara client has been updated to use these new endpoints, the experimental status of the API should be assessed with the goal of marking it as stable and ready for public consumption.
The individual changes will be broken down into individual tasks. This will allow the sahara team members to more easily research and implement the changes. These efforts will be coordinated through a page on the sahara wiki site.
The initial changes to create the /v2 endpoint should also include moving the Identity project identifier to a header named OpenStack-Project-ID. In all other respects, the endpoints currently in place for the /v1.1 API will be carried forward into the new endpoint namespace. This will create a solid base point from which to make further changes as the new API evolves and moves towards completion of all features described in the experimental specifications.
Removing the project identifier from the URI will help to create more consistent, reusable, routes for client applications and embedded HREFs. This move will also help decouple the notion of URI scoped resources being tied to a single project identifier.
The following list is an overview of all the changes that should be incorporated into the experimental API before it can be considered for migration to stable. These changes are not in order of precedence, and can be carried out in parallel. Some of these changes can be addressed with simple bugs, which should be marked with [APIv2] in their names. The more complex changes should be preceeded by specifications marked with the same [APIv2] moniker in their names. For both types of changes, the commits should contain Partial-Implements: bp v2-api-experimental-impl to aid in tracking the API conversion process.
Overview of changes:
This list is not meant to contain all the possible future changes, but a window of the minimum necessary changes to be made before the new API can be declared as stable.
The move to stable for this API should not occur before the Python sahara client has been updated to use the new functionality.
An alternative might be to make changes to the current version API, but this is inadvisable as it breaks the API version contract for end users.
Although the current version API can be changed, there is no way to safely make the proposed changes without breaking backward compatibility. As the proposed changes are quite large in nature it is not advisable to create a “1.2” version of the API.
Most of these changes will not require modifications to the data model. The two main exceptions are the payload name changes for hadoop_version and oozie_job_id. As the data model will continue to be used for the v1.1 API until it is deprecated, it is not advisable to rename these fields at this time. When the v2 API has been made stable, and the v1.1 API has been deprecated, these fields should be revisisted and changed in the data model.
During the experimental phase of the API, these translations will occur in the code that handles requests and responses. After the API has transitioned to production mode, migrations should be created to align the data models with the API representations and translations should be created for the older versions only as necessary. As the older version API will eventually be deprecated, these changes should be scheduled to coincide with that transition.
As this specification is addressing a high level change of the API, the following changes are enumerated in brief. Full details should be created for changes that will require more than just renaming an endpoint.
In the experimental phase, this change should have no noticeable affect on the end user. Once the API has been declared stabled, users will need to switch python-saharaclient versions as well as upgrade their horizon installations to make full use of renamed features.
During the experimental phase, this change will have no effect on deployers.
When the API reaches the stable phase, deployers will be responsible for upgrading their installations to ensure that sahara and python-saharaclient are upgraded as well as changing the service catalog to represent the base endpoint.
As this change is targeted for experimental work, developers should know that the details of the v2 API will be constantly changing. There is no guarantee of stability.
This change should not require changes to horizon as many of the primitives that are changing already display the proper names, for example “Job Templates”. When this change moves to the stable phase, horizon should be re-evaluated.
The main work item for this specification is the initial v2 commit.
This change should not require new dependencies.
Unit tests will be created to exercise the new endpoints. Additionally, the gabbi testing framework should be investigated as a functional testing platform for the REST API.
To improve security testing, tools such as Syntribos and RestFuzz should be investigated for use in directed testing efforts and as possible gate tests.
These investigations should result in further specifications if sufficient results are discovered to warrent their creation as they will deal with new testing modes for the sahara API server.
As the v2 API reaches stable status, and the python-saharaclient has been ported to use the new API, the current functional tests should provide the necessary framework to ensure successful end-to-end testing.
During the experimental phase, this work will not produce documentation. As the evaluation for stable approaches there will need to be a new version of the WADL files for the api-ref site, if necessary. There is the possibility that this site will change its format, in which case these new API documents will need to be generated.
Further, the v2 API should follow keystone’s model of publishing the API reference documents in restructured text format to the specs repository. This would make the API much easier to document and update as new specification changes could also propose their API changes to the same repo. Also, the WADL format is very verbose and the future of this format is under question within the OpenStack documentation community. The effort to make accurate documentation for sahara’s API should also include the possibility of creating Swagger output as the v2 API approaches stable status, this should be addressed in a more separate specification as that time approaches.
Liberty summit etherpad, https://etherpad.openstack.org/p/sahara-liberty-api-v2
Mitaka summit etherpad, https://etherpad.openstack.org/p/sahara-mitaka-apiv2