Module Management¶
Historically, Trove has supported open source databases. As more datastores were added it was inevitable that this would eventually change to include proprietary databases as well. Starting with the Liberty release this is the case (support for Vertica and DB2 now being available) and with this comes the issue of managing the licenses of said databases. In addition, operators may find it useful to include other software on their images that may require ‘activation’ by end-users (for example New Relic’s analytical suite). A method of activating this software is also needed.
The concept of applying a ‘license’ or ‘activation’ or ‘configuration’ of this third-party software is what is referred to herein as ‘module’ management.
Launchpad Blueprint: https://blueprints.launchpad.net/trove/+spec/module-management
Problem Description¶
Users of a particular cloud may be willing to purchase a license to use a datastore from the cloud vendor on a pay-as-you-go based model, and this is the model assumed at the moment (as both Vertica and DB2 redstack images include a fully-functional and licensed database). It is also desirable, however, that users be allowed to ‘bring their own license.’ In this scenario the user provides a license file that the database requires, and as such a mechanism needs to be in place to ‘activate’ and/or ‘renew’ the license through the use of the user provided file.
This same problem exists for any other proprietary software that an operator may wish to include in their Trove images. These software packages also typically require activation through the use of a license key or file (such as New Relic 1) or configuration of some kind.
Proposed Change¶
Trove’s responsiblity towards module management will be restricted to the scope of encrypting and storing the required data file (for example, a license file) and providing a way to apply this data to a new or existing Trove instance or cluster. A mechanism will be put in place to allow end users the ability to manage adding, deleting, listing, viewing and updating these module data files.
Methods to apply, remove, query and retrieve the actual ‘module’ data file on the Trove instance will also be provided.
A repeatable option (–module) will be added to the create and cluster-create commands to allow adhoc module selection. In addition, modules can be set to auto-apply, which will have the effect of the Guest Agent installing that module on any instance created with the relevant datastore combination.
Configuration¶
The following configuration changes are anticipated.
A way of specifying valid module ‘types’ will be needed for proper validation on module create:
cfg.StrOpt('module_types', default=None,
help='A list of module types supported.'),
A key will be needed in order to be able to encrypt the module data file before storing it in the database:
cfg.StrOpt('module_aes_cbc_key', default='module_aes_cbc_key',
help='OpenSSL aes_cbc key for module encryption.'),
Database¶
A new table (modules) will be added to the Trove schema:
Column
Type
Allow Nulls
Description
id
varchar(36)
No
ID of module (autogenerated)
type
varchar(255)
No
Type of module. This will correlate directly to the required plugin (i.e. a plugin must exist of this ‘type’)
tenant_id
varchar(36)
No
ID of tenant to apply module to. ‘all’ means module applies to all tenants
datastore
varchar(36)
No
Name of datastore to apply module to. ‘all’ means module applies to all datastores
datastore_version
varchar(36)
No
Name of datastore version to apply module to. ‘all’ means module applies to all datastores
name
varchar(255)
No
Name of module
description
varchar(512)
Yes
Description of module
auto_apply
tinyint(1)
No
Should this module be automatically applied during instance/cluster create. Will default to ‘no’ if not provided
visible
tinyint(1)
No
Should this module be visible to non-admin users. Will default to ‘yes’ if not provided
live_update
tinyint(1)
No
Can this module be updated while applied-to instances still exist. If set to ‘no’ all instances must have the corresponding module removed before it can be updated. Defaults to ‘no’
contents
blob
No
Encrypted module contents
md5
varchar(32)
No
MD5 hash of module contents
created
DateTime
No
Created date
updated
DateTime
No
Updated date
deleted
tinyint(1)
Yes
Deleted flag
deleted_at
DateTime
Yes
Deleted date
A unique index will be created from the (datastore, datastore_version, name) fields, to allow easy determination of the correct module to apply to a specific instance.
An MD5 hash of the module contents will be stored in the module record as well. This hash will be reported back when querying the module from a running instance, as the original module record could have been modified with new contents after it was initially applied.
On installing the module contents on a given instance, a file will be created in a know location using <datastore>-<datastore_version>-<name>.lic as a pattern.
Creating modules that apply to ‘all’ tenants or ‘all’ datastores and ones that are auto-applied will require admin credentials.
Setting a module to ‘not’ visible is also an admin-only option. This will allow administrators to ‘hide’ modules from users if they so desire. Modules that are marked visible=False will not be returned in commands such as list or show unless requested by an admin user. Non-admin users won’t be able to apply a non-visible module, however they will still be auto-applied if so designated.
A new table (instance_modules) will be added to the Trove schema to track which modules have been applied to each instance:
Column
Type
Allow Nulls
Description
id
varchar(36)
No
ID of association (autogenerated)
instance_id
varchar(36)
No
ID of instance
module_id
varchar(36)
No
ID of module
md5
varchar(32)
No
MD5 hash of module contents
created
DateTime
No
Created date
updated
DateTime
No
Updated date
deleted
tinyint(1)
Yes
Deleted flag
deleted_at
DateTime
Yes
Deleted date
Public API¶
New ReST API calls will be added to the Trove infrastructure. These fall into two categories - ones to manage the maintenance of the actual modules, and ones to handle the instance interactions.
In addition, the create and cluster-create calls will be enhanced.
Module Maintenance¶
To retrieve a list of all modules that can be applied, the following request would be made:
Request:
GET v1/modules
Response:
{
'modules' : [
{
'id': <id>,
'type': 'vertica_license',
'tenant': <id>,
'datastore': 'vertica',
'datastore_version': 'all',
'name': '100GB',
'description': 'Vertica license for 100GB',
'auto_apply': False,
'visible': True, # returned for admin only
'live_update': False,
'md5': <md5>,
'created': <date>,
'updated': <date>,
},
{
'id': <id>,
'type': 'new_relic_activation',
'tenant': <id>,
'datastore': 'all',
'datastore_version': 'all',
'name': 'new_relic',
'description': 'New Relic activation',
'auto_apply': True,
'visible': True, # returned for admin only
'live_update': True,
'md5': <md5>,
'created': <date>,
'updated': <date>,
},
]
}
Response Codes:
200 Success
Note that an admin user will receive the modules for all tenants, whereas regular users will see modules for their tenant only.
To retrieve a list of valid modules that can be applied to a specific datastore, the following request would be made:
Request:
GET v1/datastores/{datastore_id}/modules
Response:
{
'modules' : [
{
'id': <id>,
'type': 'new_relic_activation',
'tenant': <id>,
'datastore': 'all',
'datastore_version': 'all',
'name': 'new_relic',
'description': 'New Relic activation',
'auto_apply': True,
'visible': True, # returned for admin only
'live_update': True,
'md5': <md5>,
'updated': <date>,
},
]
}
Response Codes:
200 Success
To show the details of a particular module, the following request would be made:
Request:
GET v1/modules/<id>
Response:
{
'id': <id>,
'type': 'new_relic_activation',
'tenant': <id>,
'datastore': 'all',
'datastore_version': 'all',
'name': 'new_relic',
'description': 'New Relic activation',
'auto_apply': True,
'visible': True, # returned for admin only
'live_update': True,
'md5': <md5>,
'created': <date>,
'updated': <date>,
}
Response Codes:
200 Success
404 Not Found
To create a module, the following request would be made:
Request:
POST /v1.0/modules
{
'type': 'vertica_license',
'tenant': <id>,
'datastore': 'vertica',
'datastore_version': 'all',
'name': '100GB',
'description': 'Vertica license for 100GB',
'auto_apply': False,
'visible': False, # admin-only option
'live_update': True,
'contents': <module_contents>,
}
Response:
{
"module": {
'id': <id>,
'type': 'vertica_license',
'tenant': <id>,
'datastore': 'vertica',
'datastore_version': 'all',
'name': '100GB',
'description': 'Vertica license for 100GB',
'auto_apply': False,
'visible': False, # returned for admin only
'live_update': True,
'md5': <md5>,
'created': <date>,
'updated': <date>,
}
}
Response Codes:
200 Success
400 Bad Request
To update a module, the following request would be made:
Request:
PATCH /v1.0/modules/{module_id}
{
'type': 'new_type',
'tenant': <id>,
'datastore': 'new_datastore',
'datastore_version': 'new_datastore_version',
'name': 'new_name',
'description': 'new_description',
'auto_apply': True,
'visible': False, # admin-only option
'live_update': True,
'contents': <module_contents>,
}
Response:
{
"module": {
'id': <id>,
'type': 'new_type',
'tenant': <id>,
'datastore': 'new_datastore',
'datastore_version': 'new_datastore_version',
'name': 'new_name',
'description': 'new_description',
'auto_apply': True,
'visible': False, # returned for admin only
'live_update': True,
'md5': <new_md5>,
'created': <date>,
'updated': <date>,
}
}
Response Codes:
200 Success
400 Bad Request
404 Not Found
To delete a module, the following request would be made:
Request:
DELETE /v1.0/modules/{module_id}
{
}
Response:
This operation has no response body
Response Codes:
200 Success
404 Not Found
To query which instances have a particular module applied, the following request would be made:
Request:
GET v1/modules/{module_id}/instances
{
}
Response:
{
'instance': <id>,
'modules' : [
{
'name': '100GB',
'id': <id>,
'md5': <md5>,
'installed': <date>,
},
{
'name': 'new_relic',
'id': <id>,
'md5': <md5>,
'installed': <date>,
},
]
}
Response Codes:
200 Success
404 Not Found
Instance Interaction¶
To apply modules to an instance, the following request would be made:
Request:
POST v1/{tenant_id}/instances/{instance_id}/modules
{
'modules' : [
{
"id": <id>,
},
]
}
Response:
{
'type': 'vertica_license',
'datastore': 'vertica',
'datastore_version': 'all',
'name': '100GB',
'md5': <md5>,
}
Response Codes:
202 Success
400 Bad Request
404 Not Found
To query an instance about installed modules, the following request would be made:
Request:
GET v1/{tenant_id}/instances/{instance_id}/modules
{
}
Response:
{
'modules' : [
{
'type': 'vertica_license',
'datastore': 'vertica',
'datastore_version': 'all',
'name': '100GB',
'filename': 'vertica-all-100GB.lic',
'md5': <md5>,
'installed': <date>,
'status': 'OK',
'error_message': None,
},
{
'type': 'new_relic_activation',
'datastore': 'all',
'datastore_version': 'all',
'name': 'new_relic',
'filename': 'all-all-new_relic.lic',
'md5': <md5>,
'installed': <date>,
'status': 'FAILED',
'error_message': 'New Relic binaries not found',
},
]
}
Response Codes:
200 Success
404 Not Found
To retrieve a module from an instance, the following request would be made:
Request:
GET v1/{tenant_id}/instances/{instance_id}/modules/{module_id}
{
}
Response:
{
'filename': 'vertica-all-100GB.lic',
'contents': <module_contents>,
'md5': <md5>,
}
Response Codes:
200 Success
404 Not Found
To delete a module from an instance, the following request would be made:
Request:
DELETE v1/{tenant_id}/instances/{instance_id}/modules/{module_id}
{
}
Response:
This operation has no response body
Response Codes:
202 Success
404 Not Found
Creation Enhancements¶
The instance create API will be enhanced to include a module field, containing a list of modules to apply. These will be sent down during the normal ‘prepare’ call and the appropriate plugin called once this instance has been provisioned correctly.
{
'modules' : [
{
"id": <id>,
},
]
}
In a similar manner, the cluster create API will also be enhanced to include module information in the instances field, as is currently done with flavors, AZs, etc.
Public API Security¶
Since the file will be transmitted clear text across the management network, there is a chance that the module can be intercepted if the network is compromised.
It should be ensured that each plugin created does not ‘execute’ the contents of the supplied module data file, as this would present the opportunity for a security breach. This seems unlikely though (and will not be the case for the proposed implementations) as most module data files will be passed to another process for validation, and it is up to that process to ensure proper security is maintained. Code reviews will be vital to make sure no plugin accidentally executes this data.
Python API¶
New methods will be added to the Python API to facilitate the licensing. A few existing methods will need to be extended as well.
Module Maintenance¶
def module_list(self, datastore=None):
"""Get a list of all modules that can be applied. Return only
those that apply to the datastore if it is passed in.
"""
def module_list_instances(self, module):
"""Get a list of all instances that have a given module applied."""
def module_show(self, module):
"""Show the details of the module."""
def module_create(self, module_type, name, description, contents,
datastore, datastore_version='all', auto_apply=False,
all_tenants=False, visible=True, live_update=False):
"""Create a new module."""
def module_update(self, module, module_type=None, name=None,
description=None, contents=None, datastore=None,
datastore_version=None, auto_apply=None,
all_tenants=None, visible=None, live_update=None):
"""Update an existing module."""
def module_delete(self, module):
"""Delete a module."""
Instance Interaction¶
def module_apply(self, instance, modules):
"""Apply modules to an instance."""
def module_query(self, instance):
"""Query an instance about installed modules."""
def module_retrieve(self, instance, module=None, filename=None):
"""Retrieve the module data file from an instance and save it in
filename. If module is not supplied, retrieve all the modules.
If filename is not supplied, use the generated filename found
on the instance.
"""
def module_remove(self, instance, module):
"""Remove a module from an instance."""
Creation Enhancements¶
For instance.create, the modules field will be added to the call:
def create(self, name, flavor_id, volume=None, databases=None, users=None,
restorePoint=None, availability_zone=None, datastore=None,
datastore_version=None, nics=None, configuration=None,
replica_of=None, slave_of=None, replica_count=None,
modules=None):
"""Create (boot) a new instance."""
For cluster.create, the modules field will be added to the [‘cluster’][‘instances’] data structure that is already being passed in.
CLI (python-troveclient)¶
The following Trove CLI commands (upon completion) will be fully functional
module-list Displays all modules for the tenant.
module-show Shows details for a particular module resource.
module-create Creates a new module resource.
- module-update Updates module details for a particular module
resource.
module-delete Delete a module resource.
module-apply Apply the given modules to a Trove instance.
- module-query Query the given Trove instance for any installed
modules.
module-retrieve Retrieves the current modules from a Trove instance.
module-remove Remove a module from a Trove instance.
- create –module [–module]
Creates a new instance and applies the given modules.
- cluster-create –instance=module=<id>[,module=<id>]
Creates a new cluster and applies the given modules to each instance.
Internal API¶
Changes also need to be made to the internal API to include any module IDs as a part of the message body that is sent to the task manager.
The API server will need to make calls to the Guest Agent for the instance interaction type commands.
Guest Agent¶
In the Guest Agent, the modules will be managed with a plugin style architecture based on the stevedore.driver.DriverManager paradym. Each plugin will need to implement ‘apply’, ‘query’ and ‘remove’ actions. The ‘query’ action will need to report the status of the module ‘apply’ action. This would report (at a minimum) ‘OK’ or ‘FAILED’ plus any other state that seems reasonable for users of the relevant software. If possible, the ‘error_message’ field should be filled with useful information if an error occurs.
A simple plugin ‘base class’ that defines the contract will be provided. It will also provide functionality such as placing the file contents into a specified location and retrieving the file will be added. This can be used as the basis for all other plugins.
The Guest Agent code will use the module ‘type’ to determine if a plugin exists for the given module. If no plugin can be found, then an error will be written to the log and processing stopped.
To provide a concrete, real-world plugin implementation, a Vertica license module plugin will be created to allow licenses to be applied to a Vertica datastore. A New Relic plugin will also be created to illustrate activation of other third party software on a guest image.
Alternatives¶
None
Dashboard Impact (UX)¶
A multi-dropdown will need to be added to the instance create dialog that contains all modules for the selected datastore. These modules, along with any auto-apply ones, will need to be sent along on the create call. The same will be needed for the cluster create dialog.
A module detail panel will need to be created. This panel will have fields representing the attributes of a module (see module-create command).
A ‘modules’ list panel will need to be created. This will have buttons for ‘delete’ and ‘update’ and will have a link to the detail page for each listed module. This will be a high-level panel, similar to ‘Instances.’
The instance list panel will need to have a new action added: ‘apply module.’ This will cause a pop-up where the available modules are displayed. The selected module will then be passed in to the module-apply command.
The instance detail panel will need to run ‘module-query’ and display the results in a new section ‘modules.’ Alternately, a link could be placed here that would open a module list panel with the results of the ‘module-query’ call. Here, buttons for ‘module-remove’ and ‘module-retrieve’ would be needed.
Implementation¶
Assignee(s)¶
- Primary assignee:
[peterstac]
Milestones¶
Mitaka
Work Items¶
The work will be undertaken with the following tasks:
Client (Python and CLI) changes
Server (API) changes
Guest Agent module plugin infrastructure
Vertica/New Relic plugin implementation
Upgrade Implications¶
Since this change is net-new, no upgrade issues are expected.
Dependencies¶
None.
Testing¶
Generic int-tests will be written, however these will not be run under MySQL testing as it requires no module-based handling.
Documentation Impact¶
This is a net-new feature, and as such will require documentation.
Appendix¶
None
