The purpose of this spec is to define a standard for attaching meta data such as Universally Unique Identifiers (UUIDs) to tests that are to be used by tempest. This specification should be enforced both for tests that are to live in the tempest repository as well as tests that will eventually reside within their own repositories.
While there are many reasons to want meta data attached to tests. One of the complaints that led to this specification is the lack of a unique identifier for tests. If you cannot track a specific test over time, long term data carries less meaning and progress is more complicated to track.
Each test will have a decorator that will accept a required arguments containing a uuid and meta as kwargs. This open ended approach for meta data should insure flexibility for the future. Such as allow Defcore to directly tag tests with capability data moving forward.
This change will also require the development of a tool for inserting the decorators with uuid content for all the existing tests. This tool can also be used as a uniqueness validation and missing uuid checker in the gate.
It was also suggest that meta data can be written into parseable doc strings.
This change will require the following components be developed.
This tool will check for unique UUID decorators for all test_* methods in test_*py files. This is done in a single pass to build a list of undecorated tests, which can the be used as a gate check in the qa pipeline. From this list another tool will generate unique UUID metadata and insert it into all undecorated tests.
With the same tool you can add the –insert-missing arg. This will generate and insert the missing decorators.
meta_decorator_check.py --insert-missing /path/to/test(s)
The UUID and meta decorator format will be as follows:
'@test.meta(uuid='12345678-1234-5678-1234-567812345678', otherdata='another value')
For sphinx autodoc generation it will append the following to the doc string:
uuid: '12345678-1234-5678-1234-567812345678' otherdata :'another value'
To insure the data is passed to subunit output we add the testtools attr:
uuid='12345678-1234-5678-1234-567812345678' otherdata='another value'
New hacking will need to be added to insure that a decorator is set for all tests. First a new rule to be added to tempest/hacking/checks.py.
“”“Check that a test has unique UUID metadata
All tests should have a unique UUID identifier in the metadata of the form: '@test.meta(uuid=‘12345678-1234-5678-1234-567812345678’) to give a stable point of reference.T108”“”
The Tempest coding guide will be updated to reflect the new hacking rule.