MySQL Manager Refactor

There are a number of forks of MySQL that differ to varying degrees. This blueprint proposes the creation of a class structure for MySQL-derived datastores to avoid duplication of code for features and capabilities shared in common.

It is expected that the lessons learned from this effort will be applicable to future efforts to provide differentiation for systems such as MongoDB and PostgreSQL.

https://blueprints.launchpad.net/trove/+spec/mysql-manager-refactor

Problem Description

In recent years, there have been a number forks of MySQL with varying levels of divergence from MySQL Community Edition (CE). Commonly used variants of MySQL include Percona and MariaDB. Specialized versions specific to a variant also exist, such as Galera for MariaDB or MySQL Cluster (NDB). Despite the differences, there is a great deal shared among these variants. As such, wholly separate datastore implementations of each would result in duplicated code, leading to maintainability difficulties and potential confusion for operators expecting common functionality to be treated the same across systems.

Openstack Trove would benefit from the refactoring of the MySQL manager with more consistent support for MySQL-like systems, simplified code maintenance as the variants evolve and the ability to provide differentiating features. New variants could also be introduced with relative ease.

Proposed Change

The existing MySQL datastore code is found in trove/guestagent/datastore/mysql. This datastore already effectively serves as a sort of superclass, with both the Percona and MySQL datastores pointing to the same management code. MariaDB is supported if the underlying MariaDB instance is treated as if it were MySQL.

The crux of this blueprint is to move the majority of the existing MySQL manager code into a new set of abstract classes, with stub subclasses for MySQL, Percona and MariaDB datastores inheriting from them.

Maturity-Aware Reorganization

The base mysql code will reside in the current mysql datastore directory. This would result in

  • The creation of new base implementations for the manager and service modules in the trove/guestagent/datastore/mysql directory

  • The creation of explicit datastores for Percona and MariaDB with implementation classes that inherit from base MySQL.

The resulting file and directory structure would change from:

trove/guestagent/datastore/mysql/manager.py
trove/guestagent/datastore/mysql/service.py

to:

trove/guestagent/datastore/mysql/manager_base.py
trove/guestagent/datastore/mysql/service_base.py
trove/guestagent/datastore/mysql/manager.py
trove/guestagent/datastore/mysql/service.py
trove/guestagent/datastore/experimental/mariadb/manager.py
trove/guestagent/datastore/experimental/mariadb/service.py
trove/guestagent/datastore/experimental/percona/manager.py
trove/guestagent/datastore/experimental/percona/service.py

This approach has the benefit of less potential confusion about the maturity level of the base code. However, it is not as not as clean an organization: MySQL CE is treated as both a base system and an implementing datastore.

A pass through the MySQL manager code will be done to identify methods that should be made abstract. These methods would then be brought “down” into the subclasses.

This blueprint does not attempt to create optimized MariaDB or Percona datastores, but rather to lay the groundwork for their creation.

Service class dependency injection

The service.py module contains a number of classes, such as MySqlAppStatus, that are used in the manager.py module. These are currently tightly coupled: the mysql manager has an explicit import for each.

The make it possible to extend these classes arbitrarily for different datastores, and eliminate the tight-coupling, the old references will be refactored into class properties, which are to be injected by the concrete manager class.

Strategy consolidation

Currently not included in the scope of this refactor, but an important future consideration, are the associated strategies that can also have differences across variants. For example, it may make sense to bring some or all of the logic from the replication strategy into the datastore subtree to provide differentiation.

Configuration

Guest agent configuration options that point to fully qualified classnames, e.g:

datastore_registry_ext =
  mysql:trove.guestagent.datastore.mysql.manager.Manager,
  percona:trove.guestagent.datastore.mysql.manager.Manager

will need to point to the new class names, e.g:

datastore_registry_ext =
  mysql:trove.guestagent.datastore.mysql.manager.Manager,
  percona:trove.guestagent.datastore.percona.manager.Manager,
  mariadb:trove.guestagent.datastore.experimental.mariadb.manager.Manager

Database

Nothing expected, but confirm.

Python API

None.

CLI (python-troveclient)

None.

Public API

None.

Public API Security

None.

Internal API

None.

Guest Agent

Behaviour should remain the same, but location of the code would change.

Alternatives

Two alternatives are discussed in the main Proposed Change section.

Implementation

Assignee(s)

Primary assignee:

Launchpad/IRC: atomic77

Email: atomic@tesora.com

Milestones

Target Milestone for completion:

liberty-1

Work Items

  • Reorganize code

  • Create stub implementations of Percona and MariaDB datastores that inherit from base MySQL classes.

  • Review MySQL datastore implementation for initial candidates for abstract methods. Bring down and reimplement in each of the three datastore implementations.

  • Write additional integration tests

Upgrade Implications

As with any change to the layout of the source tree, care must be taken by the operator to ensure that the updating of the code on the guest agent coincides with the updating of configuration files. This would only be an issue for operators that eventually want to leverage the new optimized managers for Percona, MariaDB, etc. as the location of the MySQL CE manager would remain backwards-compatible.

Dependencies

None.

Testing

Additional tests should be added to ensure that subclassing is working correctly e.g. ensure that some Percona-specific code is not running against MySQL datastores, etc.

Documentation Impact

The documentation should be updated to inform operators of the new locations of datastore implementations that could be added to the guestagent configuration file.

References

A related blueprint is experimental-datastores [1] as this impacts the organization of datastore implementations into directories based on maturity level.

[1] https://blueprints.launchpad.net/trove/+spec/experimental-datastores