1 Eliminate trove-integration and redstack

In the very beginning, when Trove was integrated into OpenStack, one of the subjects of some contention was the redstack tool, and the fact that Trove had a somewhat non-standard structure in the form of a trove-integration repository.

It was, I understand, one of the stipulations of the Technical Committee at the time that this be normalized.

For a number of reasons, (which we will describe in the Problem Description section) this structure is proving to be quite a nusiance and it would be beneficial to the project to do this now.

The goal of this project is to eliminate the trove-integration project and the redstack tool through a normal process of deprecation. That is to say that existing releases, through Newton, will continue to be driven and may (I stress, may) use the tool, but new versions from Ocata will not.

Launchpad Blueprint: https://blueprints.launchpad.net/trove/+spec/eliminate-trove-integration-and-redstack

1.1 Problem Description

The trove-integration project was created as a place to store scripts to create, manipulate and test DBaaS vm’s, and to house the scripts used to manage the development environment. The redstack tool was the swiss army knife, it could provision your development environment, make guest images, load them, and later run tests as well.

On the whole, a very useful thing, there is no doubt about that.

It did introduce several serious problems.

  1. trove-integration is not versioned

It has only a master branch. Therefore all code that is version sensitive has to accomplish that through crafty branching constructs.

  1. trove-integration is a repository outside the trove repository

This brings with it dependencies on the trove repository, such as when a new datastore needs to be added, or a new version of a datastore is to be added. Trove ‘elements’ go into trove-integration but get used by trove.

  1. The CI is very painful

Since trove-integration isn’t branched, we have to test each trove-integration branch against not just master, but also stable branches.

  1. The versioning just got more complex

We now have to deal with multiple operating systems (ubuntu and fedora), multiple versions (liberty, mitaka, newton, ocata, …), different versions of operating systems (trusty, and now xenial). The code is now convoluted enough that attempting to add support for xenial (as I had to do for SQL Server) became a nightmare.

  1. There is no easy way to build images

This is the #1 problem faced by users of Trove; how do I build images. A solution was proposed (https://review.openstack.org/#/c/312806/) that would address this through the creation of a trove-image-builder repository. This solution has some merits and demerits, we choose here to propose a different solution (see second bullet in proposed end state) that will address that problem as well.

  1. redstack is ‘non-standard’

The current mode/phase approach provided by devstack is standardized across all projects, and redstack is one of the few (if not the only) non-standard approach. We already support the devstack plugin and redstack (in the provisioning a stack use-case) is nothing more than a thin veneer on devstack.

1.2 Proposed Change

The proposed change is in multiple parts, and several of the sections don’t match the typical headings one expects in a blueprint that is aimed at a new feature or function in Trove. Therefore the sections may be marked as Not Applicable. Additional sections have been added where required.

One thing that is important to highlight is that while this change is highly disruptive, and moves a whole lot of code around, we are largely talking about moving around a couple of hundred files (elements) from one place to another, and leveraging a devstack plugin that we already use extensively. There will be an impact on the tests that are in trove-integration but we already do most of our testing out of the trove repository.

Changes that have been proposed in the trove-integration repository; which are pending review and merge will be negatively impacted. But that will be the case no matter when we choose to make a change like this. Most of the elements are highly parameterized and the changes actually required to make this all work are minimal.

I feel that the best way to describe this proposed change is to begin with a description of the proposed ‘end-state’.

1.2.1 Proposed End State

  • The elements from trove-integration will get folded into the trove repository

  • The elements will now be versioned along with the server, they are in the same repository. Tagging the Ocata release will also cause the release of a set of elements for the release and similarly the tagging of the Pike release will cause the release of a set of elements for that release.

    A set of tools will be provided that will make this easier, more description of that below.

  • A set of sample configuration files will be provided that will cause devstack to install and configure Trove. A user can copy these into their own localrc, or use the commands to set the right environment variables before invoking devstack/stack.sh

  • The trove devstack plugin will have additional code in the stack/extra step that will build and load a set of guest images of the users choosing. This image or images will be built using the elements from the trove repository.

    This will cause the standard demo user to be used (not alt_demo).

    In running devstack, one must configure a configuration file for devstack and one specifies the datastore whose image is to be built and loaded in that configuration file. I was initially contemplating allowing for the specification of a comma-seperated list but a single datastore may be a good place to start. This datastore would also be set as the default.

    One devstack environment can operate with multiple datastores. Three commands (trove-image-builder.bash, load-trove-image.bash and activate-trove-image.bash) will be provided. The first will build the image for you. The second will load it into glance and register it with Trove, the third will set a specified datastore/version to the default.

  • The trove devstack plugin will have additional code in the stack/post-config phase that will configure the Trove test environment, add the right flavors, and do the other things that are required for Trove testing.

  • Scenario tests that are already in the trove repository will be run without the use of the redstack tool.

  • Releases Newton and before may continue to use trove-integration and redstack, releases Ocata and forward shall not use trove-integration and redstack. For example, new elements for Ocata and the future will not go into trove-integration. This means that during this transition (which may last for two releases) some changes may have to go to both trove-integration, and trove.

  • The Ocata CI will have a new set of jobs that will leverage this new state, no longer using redstack for its provisioning, and the cross version testing that is going on in trove-integration will no longer need to be supported.

1.2.2 Configuration

  • Some sample configuration files will be provided and these will help a user configure their localrc before running stack.sh. This will cause devstack to completely provision Trove, and additionally build and load guest images as instructed by the user.

1.2.3 How we get there

Getting to the desired end state will be a multi-step exercise.

Step 1

Take everything in the trove-integration repository at some specific point in time and drop it into the trove repository. For convenience, I’m going to drop it into a folder called integration.

It would be nice to do this and preserve history but that is going to be quite a pain to accomplish. An email has been sent to the infra mailing list requesting help with this but in the intermin, the commit https://review.openstack.org/#/c/384195/ does what we need (sans histories).

Step 2

Make changes to the stuff now in the trove repository and make it such that the redstack command (which will be renamed to trovestack) can be executed just as it used to be in the trove-integration branch. This also includes some small amendments to the documentation.

Commit https://review.openstack.org/#/c/384746/ does this.

Step 3

Refactor the CI jobs to use trovestack in the trove tree instead of redstack from the trove-integration tree. This means all the CI jobs (or maybe most) will become legacy jobs and new jobs will be created for Ocata and higher.

Step 4

There is a huge amount of cruft in trovestack that handled the fact that redstack in trove-integration was not versioned and released with each OpenStack version. We should do this cleanup because sooner rather than later as we will soon have to handle the Ubuntu 16.04 transition and having that dimension in addition to the OpenStack version dimension will be very difficult to manage.

Step 5

Take the image build aspect of trovestack (formerly redstack) and make it an independent tool. https://review.openstack.org/#/c/374952/ began this process and took the elements out of the place where they were copied in from trove-integration and starts to make it a standalone tool. We will update trovestack to use this tool and avoid the duplication of the image building logic.

Take the image loading logic (including the setting of the default datastore) and make that a standalone tool and modify trovestack to use that tool.

Step 6

Refactor the CI to automatically build images for different datastores and verify that the image build process is proper. This will be a new job.

Step 7

Refactor the integration tests into the tests directory.

1.2.4 Database

Not applicable

1.2.5 Public API

Not applicable

1.2.7 Python API

Not applicable

1.2.9 Internal API

Not applicable

1.2.10 Guest Agent

The guest agent is already part of the trove repository and will remain there. It will now be in the same versioned repository as the server code which has dependencies on it.

1.2.11 Alternatives

There are several options that have been considered, some at great length.

  1. Do nothing

Well, we’ve done that for three years and it hasn’t been fun. I think we can safely say that there are strong reasons to disqualify this option.

  1. Just get the image build out of redstack, leave the rest

This was part (at least) of the approach behind setting up the trove-image-builder repository. It would be an improvement over the present state but still would have cross repository dependencies. Considering that there is no clear justification for releasing trove-image-builder at a different cadence than trove, I see no justification in having an independent repository.

We know that there is a tight coupling, and there likely will be one for a long time longer, between the server side of Trove and the guest agent/images. Having two repositories just makes handling that a pain.

  1. Use the trove-image-builder approach that was proposed earlier.

We could use the approach that was proposed earlier (https://review.openstack.org/#/c/312806/) but this solution has the same kind(s) of challenges that one faces with trove-integration. Two repositories, releasing them in sync, etc.,

Having elements in the trove repository is a better alternative.

1.3 Dashboard Impact (UX)

No dashboard impact is expected.

The dashboard will be enabled (if the user enables the plugin). Code already exists for that in the trove plugin.

1.4 Implementation

1.4.1 Assignee(s)

Primary assignee:

amrith

Dashboard assignee:

<none>

1.4.2 Milestones

Target Milestone for completion:

Ocata-1

1.4.3 Work Items

  • Move elements from trove-integration to trove

  • Write wrapper (trove-image-builder) script for simplicity of use

  • Create sample configuration file settings

  • Modify CI jobs

1.5 Upgrade Implications

This change has no upgrade implications as we are talking largely about refactoring a development infrastructure.

1.7 Testing

We will test the creation of each of the images, and putting them through their paces in the CI with our existing scenario tests.

1.8 Documentation Impact

Internal (developer) documentation will need to be updated.