Validation tool for Murano Application Packages

https://blueprints.launchpad.net/murano/+spec/murano-app-validation

Murano doesn’t have validation mechanisms for MuranoPL language and application packages.

This spec proposes a validation toolkit that improves workflow by making error detection possible at the early stages of application development lifecycle.

Problem description

Early error detection simplifies application development, troubleshooting and support.

Currently Murano does not provide mechanisms to perform validation of the Murano application package before running deployment process. Thus Murano cannot prevent uploading of invalid package and developer cannot verify that application package contains valid structure and code before deployment process.

Many problems with application packages (such as invalid package structure, errors in code, typos, etc.) can be detected and solved by using static validation tool.

Proposed change

Introduce a new toolkit for validation MuranoPL language and Murano application packages.

  • Validation toolkit SHOULD be implemented as a standalone Python package (library) with CLI interface.

  • Validation library will be used to validate Murano packages during loading process.

  • CLI tool can be used by application developer to perform early checks before package is assembled.

  • Toolkit should be extensible with custom validators. It will provide API for extension purposes.

  • Toolkit should be configurable to run list of specified inspections or to ignore specified inspections.

Toolkit should implement the following validation suites: * Manifest validation * Classes validation * UI definition validation Other validators will be implemented later as part of this tool or as external packages

Manifest validation

  • manifest.yaml file should exist and should be a valid YAML document.

  • manifest.yaml structure should respond to manifest structure described in Manifest structure.

  • All files referenced in Classes section should exist in Classes subdirectory and should be valid YAML documents.

  • Warning should be reported in case when additional files exist in Classes subdirectory but not referenced in the manifest file.

  • UI definition file should exist for Application package (MuranoPL =< 1.4)

Class validation

  • Valid classes and methods should check if properties are defined correctly and there are no extra keys that are not supported.

  • Check if class is in correct namespaces according to manifest and class name matches.

  • Contract fields should be valid YAQL expressions.

  • Methods blocks should be checked for MuranoPL code structure correctness. Method blocks may consist only of Expressions, Assignments and Block constructs. See full terms description for details.

  • (Optional) Detect code that cannot be parsed as YAQL expression but is considered to be a valid expression.

UI validation

Errors

Error code consists of prefix and error number. Prefix is an upper case alphabetical sequence. It can be empty for general errors. Errors specific for some validator should use validator short name as prefix. Error number is an upper case alphanumerical sequence (one single letter E or W following by three digits). Examples: E007, MPL:E042 for errors and W777, UI:W123 for warnings. Validation tool should have ability to produce documentation of supported errors and warnings.

Package structure reference

http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#package-structure

Package consists of:

  • Manifest - File in package root directory named manifest.yaml.

  • MuranoPL classes - YAML files located in Classes directory or its subdirectories (optional).

  • UI definition - YAML file in UI directory (optional).

  • Resources - files in Resources directory (optional).

  • Logo (optional)

  • images.lst file (optional)

Manifest structure

Format

Application package format. Can be specified as <FORMAT>/<VERSION>, where format is a format name string (can be MuranoPL or Heat.HOT). If format is omitted, MuranoPL is used by default.

  • Required: no

  • Default: MuranoPL/1.0

  • Format: <FORMAT>/<VERSION>

Type

  • Required: yes

  • Type: Enum(Application, Library)

Name

Package human readable name

  • Required: no

  • Type: String

FullName

Package fully qualified name. Should be in lowerCamelCase notation separated by dots.

  • Required: yes

  • Type: String

Version

Version of package

  • Required: no

  • Default: 0.0.0

  • Type: String, should have valid SemVer format

Description

  • Required: no

  • Type: String

Author

  • Required: no

  • Type: String

Tags

  • Required: no

  • Type: List(String)

Classes

  • Required: no

  • Type: Dict

    • Keys: String, Fully qualified class name

    • Values: String, Filename relative to Classes directory

Require

Package external requirements.

  • Required: no

  • Type: Dict

    • Keys: Dependent package FullName

    • Values: null or string with exact version (‘1’, ‘2.3’, ‘4.5.6’) or

      valid spec_strings according to python-semanticversion docs

UI

File with package UI definition

  • Required: yes (for package type Application)

  • Type: String, Filename relative to root directory

  • Default: logo.png

Logo

Package logo in png format.

  • Required: no

  • Type: String, Filename relative to UI directory

  • Default: ui.yaml

Meta

  • Required: no

  • Type: Dict or List(Dict) with metadata information

Class structure

Name

  • Required: yes, if more than one class defined in the file

  • Type: String

  • Format:

    • Matches Python class name format, except leading double underscore (__ClassName).

    • Matches CamelCase naming style.

Properties

  • Required: no

  • Type: Dict

For more details see Class properties.

Methods

  • Required: no

  • Type: Dict

For more details see Class methods.

Extends

Single or multiple class names.

  • Required: no

  • Type: String | YAML List

Namespaces

  • Required: no

  • Type: Dict

  • Keys: String; Namespace alias

  • Values: String; Namespace full qualified name

  • Format:

    • Namespace alias is a valid YAQL identifier or =.

    • Namespace FQN is a string divided by dots (.).

    • Recommendation is to use lowerCamelCase naming style.

Import

  • Required: no

  • Type: String, Class name, List(Class names)

Usage

  • Required: no

  • Type: Enum(Class, Meta)

  • Default: Class

Meta

  • Required: no

  • Type: Dict or List(Dict) with metadata information

MetaClass structure

In addition to regular Class structure, MetaClass (Class with Usage: Meta) can have additional properties:

Cardinality

  • Required: no

  • Type: Enum(One, Many)

Applies

  • Required: no

  • Type: Enum(Package, Type, Method, Property, Argument, All) or List(Enum)

Inherited

  • Required: no

  • Type: boolean

Class properties

Contract

  • Required: yes

  • Type: YAQL Expression

Usage

  • Required: no

  • Type: Enum

Usage

In

Default

Out

InOut

Const

Runtime

Config

Since version 1.1

Static

Since version 1.2

Default

  • Required: no

  • Type: Any value

Meta

  • Required: no

  • Type: Dict or List(Dict) with metadata information

Class methods

Scope

New in version 1.4.

  • Required: no

  • Type: Enum

  • Values: Public, Session

Arguments

  • Required: no

  • Type: List(Dict)

For more details see Method’s Argument.

Body

  • Required: no

  • Type: YAQL Expression | List(YAQL Expression)

Usage

  • Required: no

  • Type: Enum

Usage

Runtime

Default

Static

Since 1.3

Extension

Since 1.3

Action

Deprecated since 1.4

Meta

  • Required: no

  • Type: Dict or List(Dict) with metadata information

Method’s Argument

  • Type: Dict(String, Dict)

  • Key: Argument name

Values:

  • Contract: YAQL Expression

  • Default: YAQL Expression

  • Usage: Enum added since 1.4

Usage

Standard

Default

VarArgs

KwArgs

Meta

  • Required: no

  • Type: List with metadata information

Extensions

Validation tool should be extensible and provide unified mechanism for builtin checks and extensions.

Extensions should be implemented as a python package and registered using entry points.

Extensions are loaded using stevedore library.

CLI Usage

Usage: mpl-check [options] <PACKAGE | DIRECTORY>

Arguments:
    DIRECTORY      Application working directory (e.g. contains
                        manifest.yaml, Classes/, etc.)
    PACKAGE        Compressed application (ZIP-archive)

Options:
--ignore=errors    Skip errors and warnings.
--select=errors    Check only for selected errors and warnings.
--scope=validators List of validators to execute
--strict           Consider warnings as an errors.

Alternatives

Implement validation as part of Murano and validate the package when it is uploaded to Murano.

Pros:

  • Easier to implement.

Cons:

  • Suitable for Murano itself, but not for developers.

  • Integration with CI which requires execution of functional tests and is slower than static validation.

  • Cannot be integrated with text editors or IDEs.

Data model impact

None

REST API impact

Murano API for upload packages should use this tool to do package validation before store it. If package is not valid, HTTP error with code 400 should be returned to user with list of check errors in description.

Versioning impact

None

Other end user impact

  • Murano application package developer will be able to validate package during development process.

Deployer impact

None

Developer impact

Application developer will be able to validate application code during development process without importing package to Murano and running deployment process or test suite.

Murano-dashboard / Horizon impact

Error message is displayed in Horizon when trying to upload malformed package.

Implementation

Application class

Application class provides an entry point to the checker. It is responsible for configuration, extensions loading and validation execution.

Application options are registered using oslo.config

Methods:

  • __init__(self, options)

  • load_plugins(self) - Finds validators from installed plugins

  • prepare_check_suit(self, loaded_package, validator_list) - Collect prepared checks from all validators in validator_list against specified package. Returns list with prepared checks.

  • list_errors(self, filters)- Lists errors/warnings exposed by all available validators

Validator classes

Validator class represents set for specific checks and logic to register them.

Classes

BaseValidator
  +-- PackageValidator
  +-- ManifestValidator
  +-- UiValidator
  +-- ClassValidator

Attributes

  • short_name short name for validator. Used as prefix for validator specific errors

Methods

  • __init__(loaded_package) - Init validator for specific format

  • run(self) - Runs validator against specified suit of files or whole package if file_suit=None (for Package validator)

Checkers

Checker is a function that performs checking logic. It takes data from args and yields error if it is not valid. Checkers should not contain logic for data preparation. Checker can be reused multiple times with different options in a single suit of checks. Checker can be wrapped with custom filters (i.e. manifest version), so it is called only for matching conditions.

Exceptions

Class CheckerError contains information including error code, message, file name and position in that file. Additionally it can include code snippet, if it’s available. For YAML data it’s available using yaml.error.Mark class. Otherwise code snippet can be generated during error formatting.

Attributes

  • code - Error code (Usually single letter and three digits)

  • message - Human readable error reason

  • filename - File name relative to package root

  • line - Line number (starts from 0, optional)

  • column - Column number (starts from 0, optional)

  • source - One line code snippet with highlighted column (optional)

Package loaders

Package loaders provide interface to Murano packages in multiple formats. Currently packages can be loaded from directory, single file or zip archive package.

PackageLoader
 +-- DirectoryLoader
 +-- FileLoader
 +-- ZipLoader

Methods

  • try_load(cls, path, *options) - [classmethod] Tries to load package, if it can be loaded returns instance of loaded package, otherwise None.

Loaded package

Loaded package class introduces a general interface for access to packages of different format.

Methods

  • get_file(self, name) - File wrapper if file exists, None otherwise

  • format(self) - Ru

  • exists(self, name) - Returns true if file exists in the package.

  • list(self, subdir=None) - Returns list of all files in the package (or in it’s subdirectory).

  • search_for(self, regex) - Return iterator of files fit to regex

File wrapper allows to access to raw file context with method raw() and to parsed dict with method parsed()

Formatters

Formatter classes help to represent error report in various ways. Errors can be printed to stdout/stderr using text representation, which suits developer or written to file in JSON/YAML format.

Assignee(s)

Primary assignee:

Alexander Saprykin <cutwater>

Other contributors:

Krzysztof Szukielojc <kszukielojc> Sergey Slipushenko <sslypushenko>

Work Items

  1. Develop core framework with support of extensions.

  2. Implement manifest and package structure checks.

  3. Implement Murano PL classes checks.

  4. Implement UI definition file validation.

  5. Implement YAQL checks.

  6. Integrate package validation with murano API.

  7. Develop CLI tool based on core framework

  8. Create documentation.

Dependencies

Testing

Package should include unit and functional tests.

Documentation Impact

Documentation for the library should include:

  • Usage information that describes how to use the application.

  • API reference.

  • Plugin developer documentation.

References

None