Currently all Barbican functional tests run under the same userid. This blueprint adds the ability for a Barbican functional test to run under a different userid from that default. This is needed to support the ability to execute tests in parallel and ensure that the underlying system state doesn’t change as a result of other concurrently executing tests.
Today all Barbican functional tests run under the same user. Since these tests run in parallel then any assumptions about the underlying system state are invalid - for example, a test for paging secrets could fail if another tests does secret operations in parallel.
The proposed change not only solves this problem, but also provides a generalized way to run tests in an isolated manner, under their own userid.
The first step involves determining which test(s) need to run as a non-default user. This is done at testcase creation time, by the testcase author.
Next the testcase author will create their test within class that extends a proposed new class (functionaltests.api.base.DynamicUserTestCase) which extends the existing functionaltests.api.base.TestCase to indicate that the test(s) should not be run under the default user, but rather under a new user.
The user name, password, and tenant name for this new user are derived from settings in barbican’s etc/dev_tempest.conf file. Under a new [key-manager] section are three new settings:
dynamic_user_userid_prefix=testuser_ dynamic_user_password=topsecret dynamic_user_tenant_prefix=testtenant_
As part of each invocation of the test, the oslotest fixture for the DynamicUserTestCase setUp() code will:
Next the test itself will be run as usual.
When the test completes, the oslotest fixture tearDown() code will logout and delete the new user.
The components of barbican that will need to be updated to support this new function are:
One alternative that was discussed was to create a new decorator called @run_as for which could be used for any test, not requiring a new subclass of Testcase.
We rejected that approach for several reasons:
No data model impact.
No REST API impact.
This change allows dynamic user creation and deletion on a per-testcase basis. It also allows a testcase to specify an existing user to run under. There are no changes to any API as all of these updates are focused under the functionaltests and do not affect the Barbican API.
No notification/audit impacts for this change.
The test logging will show the user id that each test runs under in the test logs. They are not part of the API, but could be useful for problem determination and debugging.
The only performance impact of this CR is on the functionaltests path. It has no effect on the product’s performance.
Tests that request a different user will incur additional pathlength to:
Need to ensure that the fields for dynamic user creation in etc/dev_tempest.conf are acceptable for the tester.
Developers adding functional tests can choose to have those tests run under a different userid, if they desire. Otherwise, all functional tests will run under the default user.
Unit tests are NOT impacted by this change.
Upon approval of this blueprint we will code and post for review the implementation of the new class as well as the supporting code in the utilities and fixtures.
This function depends on keystone for the user and tenant creation and deletion API.
This code is part of the functional testing of Barbican and will be exercised in the devstack gate as well as locally on a developer/tester machine by running “tox -e functional”
Currently there is no documentation on how to write Barbican tests (it is an open story in the Barbican work backlog). The prose in this blueprint can be used as a starting point to describe the usage of the DynamicUserTestCase.
An important item to cover in the documentation is guidance as to when a test needs to run as a DynamicUserTestCase. We should use an example such as:
- test A that creates a large number of secrets, then fetches the list of all secrets and validates the size of that list against the number created.
- test B that simply creates a new secret.
If test B runs in the middle of the execution of test A then the secret count expected by test A will be off by one, and test A will fail.
However, if test A runs as a different user from test B then there will be no interference between the two and both tests will pass as expected.