This spec adds the ability to sanely and safely purge soft-deleted rows from the manila database for all relevant tables. Presently, we keep all deleted rows. And this is unmaintainable as we move towards more upgradable releases. Today, most operators depend on manual DB queries to delete this data, but this opens up to human errors.
The goal is to have this be an extension to the manila-manage db command. Similar specs are being submitted to all the various projects that touch a database.
Very long lived OpenStack installations will carry around database rows for years and years. This brings the following problems:
To date, there is no “mechanism” to programmatically purge the deleted data.
Operators should have the ability to purge deleted rows, possibly on a schedule (cron job) or as needed (Before an upgrade, prior to maintenance). The intended use would be to specify a number of days prior to today for deletion, e.g. “manila-manage db purge 10” would purge deleted rows that have the “deleted_at” column greater than 10 days ago
The proposal is to add a “purge” command to manila-manage db command collection. This will take a non-negative number (0 will delete the rows “up to now”) of days argument and use that for a data match. Like:
- DELETE FROM shares
- WHERE deleted_at <= NOW() - INTERVAL <specified_days> DAY
Note: row with attribute(s) used as a foreign key will not be deleted even if it satisfies the deleted_at time requirement.
To accomplish this, we rearrange the table list in case of foreign key constraint, To make the logic simple and direct, we would hard code the whole table list with variable ‘PURGE_TABLE_LIST’, the existing tables in order are below (30 in total, ‘manila_nodes’ is excluded as it does not exist in db):
This list should be synchronized when the database schema is changed.
Today, this can be accomplished manually with SQL commands, or via script.
A new manila-manage command will be added:
manila-manage db purge <age_in_days>
Low, This only touches already soft deleted rows.
This has the potential to improve performance for very large databases. Very long-lived installations can suffer from inefficient operations on large tables. This would have negative DB performance impact while the purge is running.
Developer should update the table list whenever the tables’ relationship changed.
Unit testcases which focus on algorithm:
Unit testcases which focus on the hard code list (PURGE_TABLE_LIST)’s consistency:
Documentation of this feature will be added to the admin guide and developer reference.
This is already discussed and accepted in other OpenStack components, such as Glance  and Cinder .