Since Kilo, nova has implemented API microversions and the other components (Ironic, etc) also implemented it now. However, currently Tempest does not have any support for microversions and doesn’t test it at all. This proposal is to add microversions testing support in Tempest.
On microversions mechanism, each microversion change is very small. For example, version 2.2’s change is:
Add 'type' to os-keypairs response Change status code for os-keypairs create method from 200 to 201 Change status code for os-keypairs delete method from 202 to 204
In long term, a lot of microversions will be implemented because all API changes should be done with microversions. Now Ironic also implements this microversions and the other projects have a plan to implement it. So we need to implement consistent basic test way for these projects.
Implement test classes for each microversion When adding a new microversion which changes an API, basically we need to implement a test class for the API. In addition, each test class contains its microversion range with class values like min_microversion and max_microversion. For example, we have added a new attribute ‘type’ to os-keypairs response on nova’s microversion 2.2, and the corresponding test class will be:
class KeyPairsV22Test(base.BaseKeypairTest): min_microversion = '2.2' [..]
as os-keypairs test class. In the above case, max_microversion is not contained. That means unlimited as max_microversion. If we change the API again with microversion ‘2.100’ as an example, the test class will be:
class KeyPairsV22Test(base.BaseKeypairTest): min_microversion = '2.2' max_microversion = '2.99' [..]
and we need to add a test like:
class KeyPairsV100Test(base.BaseKeypairTest): min_microversion = '2.100' [..]
Add configuration options for specifying test target microversions We need to specify test target microversions because the supported microversions are different between OpenStack clouds. For operating multiple microversion tests in a single Tempest operation, configration options should represent the range of test target microversions. New configration options also are min_microversion and max_microversion, and the test classes will be selected like the following:
TestClass A: min_microversion = None, max_microversion = 'latest' TestClass B: min_microversion = None, max_microversion = '2.2' TestClass C: min_microversion = '2.3', max_microversion = 'latest' TestClass D: min_microversion = '2.5', max_microversion = '2.10'
|Configration (min, max)||Test classes (Passed microversion)|
|None, None||A(Not passed), B(Not passed), C & D - Skipped|
|None, ‘2.3’||A(Not passed), B(Not passed), C(‘2.3’), D - Skipped|
|‘2.2’, ‘latest’||A(‘2.2’), B(‘2.2’), C(‘2.3’), D(‘2.5’)|
|‘2.2’, ‘2.3’||A(‘2.2’), B(‘2.2’), C(‘2.3’), D - Skipped|
|‘2.10’, ‘2.10’||A(‘2.10’), B - Skipped, C(‘2.10’), D(‘2.10’)|
|None, ‘latest’||A(Not passed), B(Not passed), C(‘2.3’), D(‘2.5’)|
|‘latest’, ‘latest’||A(‘latest’), B - Skipped, C(‘latest’), D - Skipped|
So basically the configration min_microversion value is passed on the microversion header. However if the selected class’ min_microversion is bigger, the class’ min_microversion is passed instead. If you’d like to always pass the maximum micoversion then, you need to set the max_microversion and the min_microversion to be the same value, like the 5th example above.
The default configuration values should be (None, None) like 1st example for running on the existing clouds which don’t support microversions. So we need to change the configuration values with openstack-dev/devstack and openstack-infra/project-config for operating microversion tests on the gate.
The microversion ‘latest’ is a magic keyword as final example. When passing ‘latest’ as the microversion to each component(Nova, etc.), the component takes the latest microversion action on the server side. Some microversions will be backwards incompatible and the ‘latest’ action can break the gate test if Tempest doesn’t support the microversion at the time. To avoid such situation, we should not specify ‘latest’ on regular gate jobs. It is nice to specify it as experimental job to know we need to update Tempest for supporting the latest microversion.
These configration options should be added for each project(Nova, Ironic, etc.) because the microversion numbers are different between projects.
Once all frameworks are migrated to Tempest-lib, other projects can use the same for their microversion testing. Document needs to be updated how to consume the microversion testing framework with some example.