API Validation¶

Problem description¶

Currently Cinder doesn’t have a consistent request validation layer. Some resources validate input at the resource controller and some fail out in the backend. Ideally, Cinder would have some validation in place to catch disallowed parameters and return a validation error to the user.

The end user will benefit from having consistent and helpful feedback, regardless of which resource they are interacting with.

Use Cases¶

As a user or developer, I want to observe consistent API validation and values passed to the Cinder API server.

Proposed change¶

One possible way to validate the Cinder API is to use jsonschema similar to Nova, Keystone and Glance (https://pypi.org/project/jsonschema). A jsonschema validator object can be used to check each resource against an appropriate schema for that resource. If the validation passes, the request can follow the existing flow of control through the resource manager to the backend. If the request body parameters fails the validation specified by the resource schema, a validation error wrapped in HTTPBadRequest will be returned from the server.

Example: “Invalid input for field ‘name’. The value is ‘some invalid name value’.

Each API definition should be added with the following ways:

• Create definition files under ./cinder/api/schemas/.

• Each definition should be described with JSON Schema.

• Each parameter of definitions(type, minLength, etc.) can be defined from current validation code, DB schema, unit tests, Tempest code, or so on.

Some notes on doing this implementation:

• Common parameter types can be leveraged across all Cinder resources. An example of this would be as follows:

  from cinder.api.validation import parameter_types
  # volume create schema
  <snip>
       create = {
          'type': 'object',
          'properties': {
              'volume': {
                  'type': 'object',
                  'properties': {
                      'size': parameter_types.positive_integer,
                      'availability_zone': parameter_types.availability_zone,
                      'source_volid': {
                          'format': 'uuid',
                      },
                      'description': parameter_types.description,
                      'multiattach': parameter_types.boolean,
                      'snapshot_id': {
                          'format': 'uuid',
                      },
                      'name': parameter_types.name,
                      'imageRef': {
                          'format': 'uuid',
                      },
                      'volume_type': {
                          'format': 'uuid',
                      },
                      'metadata': {
                          'type': 'object',
                       },
                      'consistencygroup_id': {
                          'format': 'uuid',
                      },
                  },
              'required': ['size'],
              'additionalProperties': False,
          }
          'required': ['server'],
          'additionalProperties': False,
      }
  }
  
  create_v312 = copy.deepcopy(create)
  create_v312['properties']['volume'][
      'properties']['group_id'] = {'format': 'uuid',},
  
  parameter_types.py:
  
  name = {
      'type': 'string', 'minLength': 0, 'maxLength': 255,
  }
  
  positive_integer = {
      'type': ['integer', 'string'],
      'pattern': '^[0-9]*$', 'minimum': 1, 'minLength': 1
  }
  
  description = name
  availability_zone = name
  
  # This registers a FormatChecker on the jsonschema module.
  # It might appear that nothing is using the decorated method but it gets
  # used in JSON schema validations to check uuid formatted strings.
  from oslo_utils import uuidutils
  
  @jsonschema.FormatChecker.cls_checks('uuid')
  def _validate_uuid_format(instance):
      return uuidutils.is_uuid_like(instance)
  
  boolean = {
      'type': ['boolean', 'string'],
      'enum': [True, 'True', 'TRUE', 'true', '1', 'ON', 'On', 'on',
               'YES', 'Yes', 'yes',
               False, 'False', 'FALSE', 'false', '0', 'OFF', 'Off', 'off',
               'NO', 'No', 'no'],
  }
  

• The validation can take place at the controller layer using below decorator:

  from cinder.api.schemas import volume
  
  @wsgi.response(http_client.ACCEPTED)
  @validation.schema(volume.create, "3.0")
  @validation.schema(volume.create_v312, "3.12")
  def create(self, req, body):
      """creates a volume.
  
       version 3.12 added groupd_id to the volume request body.
       If user has requested volume create with version v2 then the
       'validation.schema' decorator will ignore its schema validation and
       will pass the request as it is in the create method."""
  

• Initial work will include capturing the Block Storage API Spec for existing resources in a schema. This should be a one time operation for each major version of the API. This will be applied to the Block Storage V3 API.

• When adding a new extension to Cinder, the new extension must be proposed with its appropriate schema.

Implementation¶

Dependencies¶

The extension’s which are there under cinder/api/contrib/ are getting called by v2 as well as v3. So if we add schema validation for v3 then we will have to remove the existing validation of parameters which is there inside of controller methods which will again break the v2 apis.

Solution:

1. Do the schema validation for v3 apis using the @validation.schema decorator similar to Nova and also keep the validation code which is there inside of method to keep v2 working.

2. Once the decision is made to remove the support to v2 we should remove the validation code from inside of method.

Testing¶

Tempest tests can be added as each resource is validated against its schema. These tests should walk through invalid request types.

We can follow some of the validation work already done in the Nova V3 API:

• Validation Testing

• Negative Validation Testing

Negative validation tests should use tempest.test.NegativeAutoTest

Documentation Impact¶

1. The cinder API documentation will need to be updated to reflect the REST API changes.

2. The cinder developer documentation will need to be updated to explain how the schema validation will work and how to add json schema for new API’s.

References¶

Useful Links:

• [Understanding JSON Schema] (http://spacetelescope.github.io/understanding-json-schema/reference/object.html)

• [Nova Validation Examples] (http://git.openstack.org/cgit/openstack/nova/tree/nova/api/validation)

• [JSON Schema on PyPI] (https://pypi.org/project/jsonschema)

• [JSON Schema core definitions and terminology] (http://tools.ietf.org/html/draft-zyp-json-schema-04)

• [JSON Schema Documentation] (http://json-schema.org/documentation.html)