The neutron database schema will be made consistent. It will be independent of the configured core plugin and service plugins.
The database schema is different based on which core and service plugins are configured. If a new or different plugin is configured after a deployment is started, the earlier migrations that correspond with the new configuration will be missing.
This means that the migration process is not idempotent, resulting in schema versions which depend on the current configuration. In other words, when one environment’s schema is at a specific version it may not be the same as another environment at the same version. This even happens in the same environment during a downgrade if the configuration changes.
Make all migrations unconditional. Make a single point in the migration timeline where all tables are created and the database schema is the same no matter which plugins are used.
This solution will introduce a “healing” migration that will call the DDLs needed to complete the database schema so that it contains all tables for all database models. This migration will occur in the timeline between the icehouse_release version and the juno_release version.
The refactoring will comprise the following:
The healing migration will work like any other alembic migration, i.e. it will allow upgrade and downgrade. However, it will be different in the following aspects:
In the upgrade direction the healing migration will introspect the schema and only add tables that are missing. Therefore it cannot be run in offline mode. (See also Online Requirement.)
In the downgrade direction the healing migration will make no schema changes. This means it will not remove any tables.
More reasons why we cannot support offline mode for the healing migration:
Instead of a healing migration we could break the migration timeline after Icehouse and create a new one beginning at Juno that includes all tables. A manual script could be provided to convert the schema from the old timeline to the new. This alternative approach has the following disadvantages:
Some database models may need to be updated in order to have non-conflicting models based on core and service plugins.
End (non-admin) users should see no impact.
The healing migration is an online operation that must be run by the DB Administrator. Thus the deployer must co-ordinate with the DBA when upgrading or downgrading through Juno.
The healing migration will be run when migrating from Icehouse or earlier to Juno. It behaves like a normal migration, but it does not support upgrade in offline mode.
Downgrade of the healing migration does nothing. Thus all tables are present in the schema if downgrading from db_healing to a previous version.
Note: Greenfield deployment of the Juno release or later will start at the new migration timeline and therefore no healing will be involved.
Most of the work lies in developing a robust healing migration. Alembic will be used where possible to maximize automation, but some healing may need to be manually coded.
Examples of conflicts which may need manual coding to resolve:
Unit (and functional?) testing of migrations shall be added. We plan to utilize the unit test framework from the graduated oslo.db package.