Glance Expand/Contract migrations with Alembic¶
This spec outlines the motivation and implementation details for porting Glance database migrations from SQLAlchemy-based scripts to Alembic.
Existing database migrations are implemented in Glance using SQLAlchemy-
migrate scripts. This approach makes designing migrations for rolling upgrades
( & ) extremely uncomfortable. The
migrations would have to be created in strict sequential order with all of the
latter following all of the former. This places unnecessary restrictions on
the order of migrations and becomes very error prone as multiple schema changes
are introduced in a single cycle. Essentially, using SQLAlchemy-migrate
contract migrations to be run atomically and does
not easily facilitate separating them into phases as required during rolling
We are proposing to port the migration flows to be handled by Alembic .
There are numerous benefits to this change. Alembic allows labeling the
various migration scripts with designated branch names (e.g.
contract), which makes it easier to group the two types of changes and
run them separately.
Another benefit is that Alembic migrations are chained in the manner of linked list. This is accomplished by explicitly specifying revision dependencies in each migration script. This will allow developers to get away from dependence on numeric ordering of migration scripts and obviate the need for placeholder migrations for retroactive backports, of the like: 023_placeholder. To facilitate this feature, a new naming convention is proposed for migration scripts:
Additionally, Alembic supports automatic generation of migration scripts using SQLAlchemy’s ORM. In the future, developers are encouraged to utilize this feature to minimize their interaction with the underlying DBMS and rely on the Python code to define the new schema.
The best aleternative available is to continue using SQLAlchemy-migrate scripts. The downsides of this appropach are outlined in the Problem description section above.
Data model impact¶
REST API impact¶
Other end user impact¶
Performance of existing offline migration flows is not expected to be impacted in a signifficant way.
Other deployer impact¶
The API of the glance-manage command-line utility remains backwards compatible.
However, the operators will notice that the
db_version command returns a
named head revision rather than a number. This change will be documented.
Additionally, the operators will need to be made aware of the new
db_data_migrate APIs in the
glance-manage command-line utility. Refer to the Documentation Impact
Going forward, any time a database schema change is implemented, developers will need to create Alembic migration scripts conforming to the established naming convention and making sure to link each new revision to the appropriate down revision.
alembic_migrations folder is added under
This folder will contain all the necessary code to implement
Alembic based migrations, including:
Alembic configuration and environment files. For details see: Migration Environment
helper scripts for creating initial Glance DB schema
versionsdirectory, containing migration scripts added in different release cycles
- Primary assignee:
Implement Alembic migration infrastructure and scripts for existing database migrations (up to Newton).
expand/contractmigrations for new Ocata features .
Update the glance-manage utility to utilize Alembic for
db_syncand related commands. Add
db_contractcommands to prepare for rolling upgrades.
A new Python module will be added to Glance’s requirements list:
alembic - version 0.8.7 or hihger
Existing and new database migration tests (unit and functional) will be ported and/or added to exercise the updated glance-manage utility and test the integrity of Alembic-based migration scripts.
The new database version names resulting from the move to Alembic will need to
be documented to make developers aware of the change to
Also, as part of the drive to enable the
-upgrade tag , this change will introduce and implement new API methods
for the glance-manage utility:
db_data_migrate. These will need to be thoroughly documented and their
usage explained in the glance-manage reference as well as the upgrade guide.
It is advised to wait until Glance is able to assert the zero-downtime upgrade
tag to add documentation for the new commands.