Flavor framework - Templates and meta-data

https://blueprints.launchpad.net/neutron/+spec/neutron-flavor-framework-templates

The Flavor Framework spec introduces a static framework for creating different feature levels for a service. This proposal adds the ability to have the objects being created for the service, influence the flavor’s behavior.

Problem Description

The original flavor framework allows operators to create custom feature levels for a given service. But, some features require per-instance data to be effectively deployed.

An example would be a load balancer that offers page caching. With flavors, you can specify one flavor for no page caching, and another for page caching everything. But, full page caching often breaks legacy applications, and certain URLs often need to be exempted, on a per load-balancer basis, such as “do page caching, but not for /legacy/bank/thingie”.

Another example would be an operator choosing to enable DDoS protection for a certain level of load balancer, but security features like those often need to have a per-instance whitelist for the end-admin to be able to quickly deal with false positives.

Proposed Change

This proposal suggests two changes to the existing flavor framework:

  • Add a model/table for “flavor object meta-data”, which is meta-data that is stored for any given flavor-enabled neutron object, back-referenced with that objects’ UUID, and a mixin (/me hides) that adds a “flavor_meta” attribute to said object.
  • In the “metadata” field passed to drivers, support jinja templating syntax, with macro substitution available for the neutron model attributes or the above per-instance meta data. Macro substitution will happen before the resulting “metadata” is passed to the backend plugin/driver, so drivers do not need to be aware of this templating.

This is intended to be a temporary mechanism for operators to use to enable certain features that are otherwise missing from openstack, without directly modifying neutron.

Examples:

With the static flavor framework, you might have a flavor named “AwesomeSauce”, which includes load balancing and DDoS protection, and the driver metadata might look like this:

{
  vendor_z23_ddos_protection: true
}

and that is exactly what would be passed to the z23 lbaas driver. With the templating syntax, that example becomes:

{
  vendor_z23_ddos_protection: true
  z23_funky_whitelist: [ {{ meta.whitelist }} ]
}

With a corresponsing flavor_meta field on the LoadBalancer object being:

{
  whitelist: "1.2.3.4, 8.8.8.8"
}

and the resulting metadata passed to the driver would be:

{
  vendor_z23_ddos_protection: true
  z23_funky_whitelist: [ '1.2.3.4', '8.8.8.8' ]
}

Data Model Impact

One new logical model will be added to the database.

FlavorMeta
  id: uuid
  object_id: uuid
  key: string
  value: string

The ServiceProfile model will be modified to add an “allowed_meta_keys” attribute, that when taken in conjunction with the template that can be defined in the “metadata” attribute, defines the allowed additional of the data that the end user can submit as flavor_meta data.

The following existing models will be modified with a ‘meta’ method which will lookup associated FlavorMeta data:

LoadBalancer

As more features are made flavor aware, their root models should add the meta method (mixin?).

REST API Impact

Supporting flavor-enabled models will add an attribute for flavor_meta. Setting this attribute will create an associated FlavorMeta model.

Attribute Name Type Access Default Value Validation/ Conversion Description
flavor_meta list RW, all ‘’ list key/value per-object meta data

Resource: /service_profile

Attribute Name Type Access Default Value Validation/ Conversion Description
allowed_meta_keys list RW, admin ‘’ list keys for flavor_meta

Security Impact

None

Notifications Impact

None

Other End User Impact

This is an additional hook for operators to enable. If present, users can interact with the flavor_meta attribute via the API.

Performance Impact

None

IPv6 Impact

None

Other Deployer Impact

None

Developer Impact

Services/models that want to support flavors and this templating mechanism will need to add the appropriate model entry and db migration.

Community Impact

This change allows operators greater flexibility in enabling advanced services within the Neutron framework.

Many of the features that can be enabled via this mechanism can and should be added as features in their corresponding service projects. This proposal specifies a mechanism for operators and vendors to enable features which, for whatever reason, are not being addressed by the OpenStack dev community yet. This could be because the feature is too niche to be broadly applicable, the release timeline is too far in the future, or a good open-source alternative simply does not exist yet. This proposal adds a mechanism to handle the interim, until the mainline feature is supported.

Best practices:

  • If you need to migrate flavors, the prior decision is that if you have to create a new service entity with a new flavor. They can not be modified at this time. This applies to all flavors, not just this proposal.
  • If an equivalent community feature exists, it is encouraged that operators use that feature/give feedback/contribute. The recommended best practice for drivers is for features exposed in the main-line to override features exposed through flavor meta-data
  • It should be self-evident that enabling a feature offered by a single vendor does two things for the Operator: 1. It allows the operator to utilize that feature and expose it to their users. 2. If users use said feature, the operator may find it more difficult to switch vendors if they choose to do so at a later date. Furthermore, for features offered by multiple vendors, care must be taken in abstracting those features, if the operator desires to support multiple vendors.

Alternatives

  • The first alternative is to do nothing. This results in what many vendors are doing today, which is to brew up proprietary neutron solutions in order to expose more advanced features. This results in inconsistent solutions for operators, more difficulty tracking trunk, and vendor lock-in.
  • Another alternative is the same as this proposal minus the templating on the flavor metadata. Since the flavor metadata is tied to a particular driver, and thus vendor specific, removing the templating would force vendors to expose vendor specific goo to their end users. In addition, since multiple service profiles (drivers/vendors) can be used to implement a single flavor, not having templating would mean that that multiple backend support would break unless those backends supported the exact same back-end metadata, which is unlikely and impractical.
  • Finally, the most straightforward alternative is to implement each feature natively into the services API. In the aforementioned page caching example, add page caching as a feature in the API, with a URL exception list, and wait for all drivers to implement to that backend. This adds maintenance and development load to the entire community, and means that Neutron will have a built-in lag for adding new features that appear in the marketplace.

Implementation

Work Items

  • Add new data model and migration.
  • Add mixin for neutron models, with ‘meta’ method.
  • Add mixin to LoadBalancer, and its REST attributes.
  • Modify flavors to apply jinja templating to service profile metadata attr.

Dependencies

  • Main flavors framework
  • LBaaS v2

Testing

Tempest Tests

Flavor tests need to be enhanced to include per-object meta-data and some basic templated flavor metadata, and verify that substituted data is passed to the backend.

Functional Tests

Tests to verify the flavor_meta field in models, and that the jinja substitution is happening properly in the flavors code before being passed to backends.

API Tests

Modify flavor API tests to include flavor_meta field for objects.

Documentation Impact

User Documentation

This change is invisible to end users.

Developer Documentation

Deployers will need documentation for the new API fields and the templating syntax.

References