Flavor Extra Spec Validation (Extended)¶

https://blueprints.launchpad.net/nova/+spec/flavor-extra-spec-image-property-validation-extended

Introduce a YAML-based schema describing flavor extra specs, image metadata properties, and the relationship between each.

Problem description¶

Flavor extra specs are one of the Wild West aspects of nova. There are a number of issues we’d like to address:

Lack of documentation for many flavor extra specs and image metadata properties [1]. Yes, we have Glance image metadata definitions but they’re generally rather out-of-date, we don’t/can’t consume them in Nova, and they’re not aimed towards user-facing documentation.
Outdated, incomplete or incorrect Glance metadata definitions.
No warnings if there is a typo in your extra spec, resulting in different behavior to that expected.
No defined way to do things like deprecate a flavor extra spec, resulting in continued reinvention of the wheel.

Use Cases¶

As a deployer, I’d like to know what flavor extra specs and image metadata properties are available and why I’d want to use them.
As a deployer, I’d like nova to tell me when I’ve used a flavor extra spec that doesn’t exist or has a typo in it.
As a developer, I’d like an easy way to deprecate flavor extra specs, which is something that will only become more common if we do things like move tracking of dedicated CPUs into placement.
As a documentation writer, I’d like to be able to cross-reference the various flavor extra specs and image metadata properties available.

Proposed change¶

A flavor extra spec is a key-value pair. For example:

hw:cpu_policy=dedicated

Different solutions are needed to validate the value part of an extra spec compared to the key part. This spec aims to tackle validation of both key and value, starting with the latter and then moving onto the former.

The following are considered out-of-scope for this change:

Enforcement of extra spec dependencies. For example, if extra spec A requires extra spec B be configured first. We will document the dependency but it won’t be enforced.
Enforcement of virt driver dependencies. Unfortunately, while flavor extra specs should be generic, this isn’t always the case. As above, we will document this dependency but it won’t be enforced.
Hard enforcement of key validation. Eventually we will want to track all possible extra spec names and raise a warning or error for errant values, but this is likely to take some time to perfect. In the interim, we will merely log these potentially errant values.

This change builds upon Flavor extra spec image metadata validation <http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/flavor-extra-spec-image-property-validation.html>, which covers some of these issues for us.

Value validation¶

Value validation is the easier of the two issues to tackle. It will resolve issues like the following in a generic manner:

hw:cpu_policy=deddddicated

For a generic extra spec, a definition of a validator will need to contain the following:

Name or key of the extra spec, e.g. cpu_policy for the above example
Namespace of the extra spec, e.g. hw for the above example
Description of the extra spec
Support status of the extra spec
Valid values; whether it’s an integer, a free-form string, a string matching a given regex, an enum, or something else entirely
Virt driver dependencies; this is only for documentation purposes and will not be enforced
Extra spec dependencies; this is only for documentation purposes and will not be enforced

For many extra specs namespaces, we propose maintaining the definitions in-tree. To do this, we propose adding a new module, nova.api.validation.extra_specs, which will contain definitions for flavor validators. These will be defined using two new base objects, BaseValidator and BaseExtraSpec. BaseValidator will be subclassed to represent a namespace while BaseExtraSpec will be subclassed to represent an individual extra spec. BaseExtraSpec subclasses will be registered again a namespace.

For example:

class HWValidator(BaseValidator):
    """A validator for the ``hw`` namespace."""
    name = 'hw'
    description = (
        'Extra specs that modify behavior of the virtual hardware '
        'associated with instances.'
    )

class CPUPolicy(BaseExtraSpec):
    """A validator for the ``hw:cpu_policy`` extra spec."""
    name = 'cpu_policy'
    description = (
        'The policy to apply when determining what host CPUs the guest '
        'CPUs can run on. If ``shared`` (default), guest CPUs can be '
        'overallocated but cannot float across host cores. If '
        '``dedicated``, guest CPUs cannot be overallocated but are '
        'individually pinned to their own host core.'
    )
    deprecated = True
    value = {
        'type': str,
        'description': 'The CPU policy.',
        'enum': [
            'dedicated',
            'shared'
        ],
    }

class NUMACPUs(BaseExtraSpec):
    """A validator for the ``hw:numa_cpu.{id}`` extra spec."""
    name = 'numa_cpu.{id}'
    description = (
        'A mapping of **guest** CPUs to the **guest** NUMA node '
        'identified by ``{id}``. This can be used to provide asymmetric '
        'CPU-NUMA allocation and is necessary where the number of guest '
        'NUMA nodes is is not a factor of the number of guest CPUs.'
    )
    params = [
        {
            'name': 'id',
            'type': int,
            'description': 'The ID of the **guest** NUMA node.',
        },
    ]
    value = {
        'type': str,
        'description': (
            'The guest CPUs, in the form of a CPU map, to allocate to the '
            'guest NUMA node identified by ``{id}``.'
        ),
        'pattern': r'\d+((-\d+)?(,\^?\d+(-\d+)?)?)*',
    }

register(HWValidator, CPUPolicy)
register(HWValdiator, NUMACPUs)

While many of the definitions will be maintained in-tree, some namespaces will require special handling as they’re owned by external services, e.g. the traits namespace (owned by os-traits) or the accel namespace (proposed for use by cyborg). For these, we propose using stevedore to allow external projects to register custom validators. For example, nova would provide the following:

nova.extra_spec_validators =
    hw = nova.api.validation.extra_specs:HWValidator
    os = nova.api.validation.extra_specs:OSValidator
    traits = nova.api.validation.extra_specs:TraitsValidator
    resources = nova.api.validation.extra_specs:ResourcesValidator
    custom = nova.validators.extra_specs:NoopValidator
    * = nova.validators.extra_specs:YAMLValidator

Cyborg could extend this by providing something like the following:

nova.extra_spec_validators =
    accel = cyborg.extra_specs_validator:AccelValidator

Finally, there are extra specs that are operator defined and therefore will not be known by a consuming service. For these, we propose introducing a schema definition file. This a YAML-formatted file, which describes the flavor extra specs available. The YAML format is chosen as it allows us to define a specification in a declarative manner while avoiding the need to write Python code. The format of this file will nonetheless mirror the format of the Python objects. For example:

---
version: 1.0
metadata:
- name: numa_nodes
  namspace: hw
  description: >
    The number of NUMA nodes the instance should have.
  value:
    type: integer
    description: >
      The number of NUMA nodes the instance should have.

- name: numa_cpus.{id}
  namspace: hw
  description: >
    A mapping of **guest** CPUs to the **guest** NUMA node identified by
    ``{id}``. This can be used to provide asymmetric CPU-NUMA allocation
    and is necessary where the number of guest NUMA nodes is is not a
    factor of the number of guest CPUs.
  parameters:
  - name: id
    type: integer
    description: >
      The ID of the **guest** NUMA node.
  value:
    type: string
    format: '\d+((-\d+)?(,\^?\d+(-\d+)?)?)*'
    description: >
      The guest CPUs, in the form of a CPU map, to allocate to the guest
      NUMA node identified by ``{id}``.

Regardless of the source of the extra spec validator, they will be used by the API behind the openstack flavor set command. A microversion will be introduced for this command to avoid breaking existing tools that are inadvertently setting the wrong values.

Key validation¶

We also want to be able to catch invalid extra specs themselves. It will resolve issues like the following in a generic manner:

hw:cpu_pollllicy=dedicated

This involves maintaining a registry of valid extra specs. Not all extra specs can be known ahead of time and for dynamic extra specs, such as those proposed in Support filtering by forbidden aggregate membership <http://specs.openstack.org/openstack/nova-specs/specs/stein/approved/negative-aggregate-membership.html>. For these, we can rely on a custom namespace validator or YAML specification provided by the operator. However, completing this registry both in-tree and out-of-tree is expected to be a complex endeavour and for this reason we won’t enforce validation of keys as part of this spec.

Other changes¶

We also propose adding tooling to (a) render reStructuredText documentation from the definitions and (b) convert the definitions into Glance metadata definition files. Both of these tools will live within the nova tree, allowing us to remain the single source of truth for these things.

Alternatives¶

We could ignore some of the above issues and try to solve others in a piecemeal fashion. This will likely be far more tedious and time consuming as modifications will be needed in far more places.

Data model impact¶

None.

REST API impact¶

We will add a REST API microversion to the POST flavors/{flavor_d}/os-extra_specs API to catch invalid flavor extra specs.

Security impact¶

None.

Notifications impact¶

None.

Other end user impact¶

End users will have better documentation for the available flavor extra specs and image metadata properties.

Performance Impact¶

None.

Other deployer impact¶

Operators will now need to add new flavor extra specs to the YAML schema file or they will see errors when using the new API microversion.

Developer impact¶

Developers should now add new flavor extra specs to the nova.compute.extra_specs module to take advantage of the validation available.

Upgrade impact¶

None.

Implementation¶

Assignee(s)¶

Primary assignee:: stephenfinucane
Other contributors:: None

Work Items¶

Produce extra spec definitions for all in-tree flavor extra specs.
Add code to validate this against the image metadata properties and flavor extra specs on instance create, resize and rebuild operations.
Add a Sphinx extension to render this spec into documentation and another tool to convert the spec into Glance metadata definitions.
Add parser for YAML-formatted definitions and document how operators can and should use this.

Dependencies¶

None.

Testing¶

Unit tests.

Documentation Impact¶

There will be better docs, through the power of Sphinx.

References¶

History¶

Revisions¶
Release Name	Description
Train	Introduced

Flavor Extra Spec Validation (Extended)

Flavor Extra Spec Validation (Extended)¶

Problem description¶

Use Cases¶

Proposed change¶

Value validation¶

Key validation¶

Other changes¶

Alternatives¶

Data model impact¶

REST API impact¶

Security impact¶

Notifications impact¶

Other end user impact¶

Performance Impact¶

Other deployer impact¶

Developer impact¶

Upgrade impact¶

Implementation¶

Assignee(s)¶

Work Items¶

Dependencies¶

Testing¶

Documentation Impact¶

References¶

History¶

Nova Specs

Page Contents