Use Templates for Scenario Tests Configuration¶
https://blueprints.launchpad.net/sahara/+spec/scenario-test-config-template
Users that want to run the scenario tests available in the Sahara repository need to modify the provided YAML files, which are really template files even if not marked as such. The usability of this process could be improved by extending the test runner to read the templates and perform the substitution of the required variables instead.
Problem description¶
The scenario tests provided in the Sahara repository are template files even if they are simply marked as YAML files; this is not clear from the README.rst file. Users that want to run them need to manually replace the required variables (the variables are needed because they depend on the environment: host ip, credentials, network type, flavors used, etc). This step needs to be done both by developers and testers, and by wrapper scripts used to run the script on a CI. The repository of the main Sahara CI, sahara-ci-config, contains code which replaces the variables:
https://github.com/stackforge/sahara-ci-config/blob/master/slave-scripts/functions-common.sh#L148
Proposed change¶
The current template files (mostly under etc/sahara-ci right now) need to be properly identified as templates. The chosen format is Mako, because it is already a dependency for the scenario test runner (runner.py). The files will be marked using a special suffix (file.yaml.mako) and the variables used will be converted in Mako format. The usage of templating would be limited to variable replacement, which means no logic in the templates.
The test runner will continue to handle normal YAML files as usual, in addition to template files.
runner.py will also take a simply INI-style file with the values for the variables used in the template. It will be used by the runner to generate the real YAML files used for the input (in addition to the normal YAML files, if specified).
A missing value for some variable (key not available in the INI file) will raise an exception and lead to the runner termination. This differs from the current behavior of sahara-ci-config code, where a missing values just prints a warning, but given that this would likely bring to a failure, enforcing an early termination limit the resource consumption.
The current sahara-ci-config code allows to specify more details for the replacement variables, like specific end keys where to stop the match for a certain key, but they are likely not needed with a proper choice of names for the variables.
Finally, sahara/tests/scenario/README.rst should be changed to document the currently used variables and the instruction on how to feed the key/value configuration file. The code in sahara-ci-config should be changed as well to create such configuration file and to use the new names of the template files for the respective tests.
Alternatives¶
None
Data model impact¶
None
REST API impact¶
None
Other end user impact¶
None
Deployer impact¶
CI systems which runs tox -e scenario should be updated to use the new filenames.
Developer impact¶
Developers/QE running tests need to use the new template names.
Sahara-image-elements impact¶
None
Sahara-dashboard / Horizon impact¶
None
Implementation¶
Assignee(s)¶
- Primary assignee:
ltoscano
Work Items¶
The change will be implemented as follow:
allow runner.py to use mako template files as input;
copy the existing files to the new name and use non-ambiguous variable names for templates;
change sahara-ci scripts to use to set the new variables and use the renamed template files;
remove the old yaml files;
(optional) clean up unnedded code (insert_scenario_value, etc)
Repositories affected: - sahara: 1, 2, 4 - sahara-ci-config: 3, 5
Dependencies¶
None
Testing¶
Successful CI run will ensure that the new code did not regress the existing scenarios.
Documentation Impact¶
sahara/tests/scenario/README.rst needs to be updated.
References¶
None
