Validation tool for Murano Application Packages¶
https://blueprints.launchpad.net/murano/+spec/murano-app-validation
Murano doesn’t have validation mechanisms for MuranoPL language and application packages.
This spec proposes a validation toolkit that improves workflow by making error detection possible at the early stages of application development lifecycle.
Problem description¶
Early error detection simplifies application development, troubleshooting and support.
Currently Murano does not provide mechanisms to perform validation of the Murano application package before running deployment process. Thus Murano cannot prevent uploading of invalid package and developer cannot verify that application package contains valid structure and code before deployment process.
Many problems with application packages (such as invalid package structure, errors in code, typos, etc.) can be detected and solved by using static validation tool.
Proposed change¶
Introduce a new toolkit for validation MuranoPL language and Murano application packages.
Validation toolkit SHOULD be implemented as a standalone Python package (library) with CLI interface.
Validation library will be used to validate Murano packages during loading process.
CLI tool can be used by application developer to perform early checks before package is assembled.
Toolkit should be extensible with custom validators. It will provide API for extension purposes.
Toolkit should be configurable to run list of specified inspections or to ignore specified inspections.
Toolkit should implement the following validation suites: * Manifest validation * Classes validation * UI definition validation Other validators will be implemented later as part of this tool or as external packages
Manifest validation¶
manifest.yaml
file should exist and should be a valid YAML document.manifest.yaml
structure should respond to manifest structure described in Manifest structure.All files referenced in
Classes
section should exist inClasses
subdirectory and should be valid YAML documents.Warning should be reported in case when additional files exist in
Classes
subdirectory but not referenced in the manifest file.UI definition file should exist for Application package (MuranoPL =< 1.4)
Class validation¶
Valid classes and methods should check if properties are defined correctly and there are no extra keys that are not supported.
Check if class is in correct namespaces according to manifest and class name matches.
Contract fields should be valid YAQL expressions.
Methods blocks should be checked for MuranoPL code structure correctness. Method blocks may consist only of Expressions, Assignments and Block constructs. See full terms description for details.
(Optional) Detect code that cannot be parsed as YAQL expression but is considered to be a valid expression.
UI validation¶
UI definition should contain
Application
andForms
sectionsYAQL expressions in UI definition should be valid
Form
section should contain only valid forms http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#forms
Errors¶
Error code consists of prefix and error number. Prefix is an upper case
alphabetical sequence. It can be empty for general errors. Errors specific for
some validator should use validator short name as prefix. Error number is an
upper case alphanumerical sequence (one single letter E
or W
following by three digits). Examples: E007
, MPL:E042
for errors and
W777
, UI:W123
for warnings. Validation tool should have ability to
produce documentation of supported errors and warnings.
Package structure reference¶
http://docs.openstack.org/developer/murano/draft/appdev-guide/murano_packages.html#package-structure
Package consists of:
Manifest - File in package root directory named
manifest.yaml
.MuranoPL classes - YAML files located in
Classes
directory or its subdirectories (optional).UI definition - YAML file in
UI
directory (optional).Resources - files in
Resources
directory (optional).Logo (optional)
images.lst file (optional)
Manifest structure¶
Format
Application package format. Can be specified as <FORMAT>/<VERSION>
,
where format is a format name string (can be MuranoPL
or Heat.HOT
).
If format is omitted, MuranoPL
is used by default.
Required: no
Default:
MuranoPL/1.0
Format:
<FORMAT>/<VERSION>
Type
Required: yes
Type: Enum(Application, Library)
Name
Package human readable name
Required: no
Type: String
FullName
Package fully qualified name. Should be in lowerCamelCase notation separated by dots.
Required: yes
Type: String
Version
Version of package
Required: no
Default:
0.0.0
Type: String, should have valid SemVer format
Description
Required: no
Type: String
Author
Required: no
Type: String
Tags
Required: no
Type: List(String)
Classes
Required: no
Type: Dict
Keys: String, Fully qualified class name
Values: String, Filename relative to
Classes
directory
Require
Package external requirements.
Required: no
Type: Dict
Keys: Dependent package
FullName
- Values:
null
or string with exact version (‘1’, ‘2.3’, ‘4.5.6’) or valid spec_strings according to python-semanticversion docs
- Values:
UI
File with package UI definition
Required: yes (for package type
Application
)Type: String, Filename relative to root directory
Default:
logo.png
Logo
Package logo in png format.
Required: no
Type: String, Filename relative to
UI
directoryDefault:
ui.yaml
Meta
Required: no
Type: Dict or List(Dict) with metadata information
Class structure¶
Name
Required: yes, if more than one class defined in the file
Type: String
Format:
Matches Python class name format, except leading double underscore (
__ClassName
).Matches CamelCase naming style.
Properties
Required: no
Type: Dict
For more details see Class properties.
Methods
Required: no
Type: Dict
For more details see Class methods.
Extends
Single or multiple class names.
Required: no
Type: String | YAML List
Namespaces
Required: no
Type: Dict
Keys: String; Namespace alias
Values: String; Namespace full qualified name
Format:
Namespace alias is a valid YAQL identifier or
=
.Namespace FQN is a string divided by dots (
.
).Recommendation is to use lowerCamelCase naming style.
Import
Required: no
Type: String, Class name, List(Class names)
Usage
Required: no
Type: Enum(Class, Meta)
Default:
Class
Meta
Required: no
Type: Dict or List(Dict) with metadata information
MetaClass structure¶
In addition to regular Class structure, MetaClass (Class with Usage: Meta) can have additional properties:
Cardinality
Required: no
Type: Enum(One, Many)
Applies
Required: no
Type: Enum(Package, Type, Method, Property, Argument, All) or List(Enum)
Inherited
Required: no
Type: boolean
Class properties¶
Contract
Required: yes
Type: YAQL Expression
Usage
Required: no
Type: Enum
Usage |
|
---|---|
In |
Default |
Out |
|
InOut |
|
Const |
|
Runtime |
|
Config |
Since version 1.1 |
Static |
Since version 1.2 |
Default
Required: no
Type: Any value
Meta
Required: no
Type: Dict or List(Dict) with metadata information
Class methods¶
Scope
New in version 1.4.
Required: no
Type: Enum
Values: Public, Session
Arguments
Required: no
Type: List(Dict)
For more details see Method’s Argument.
Body
Required: no
Type: YAQL Expression | List(YAQL Expression)
Usage
Required: no
Type: Enum
Usage |
|
---|---|
Runtime |
Default |
Static |
Since 1.3 |
Extension |
Since 1.3 |
Action |
Deprecated since 1.4 |
Meta
Required: no
Type: Dict or List(Dict) with metadata information
Method’s Argument¶
Type: Dict(String, Dict)
Key: Argument name
Values:
Contract: YAQL Expression
Default: YAQL Expression
Usage: Enum added since 1.4
Usage |
|
---|---|
Standard |
Default |
VarArgs |
|
KwArgs |
Meta
Required: no
Type: List with metadata information
Extensions¶
Validation tool should be extensible and provide unified mechanism for builtin checks and extensions.
Extensions should be implemented as a python package and registered using entry points.
Extensions are loaded using stevedore library.
CLI Usage¶
Usage: mpl-check [options] <PACKAGE | DIRECTORY>
Arguments:
DIRECTORY Application working directory (e.g. contains
manifest.yaml, Classes/, etc.)
PACKAGE Compressed application (ZIP-archive)
Options:
--ignore=errors Skip errors and warnings.
--select=errors Check only for selected errors and warnings.
--scope=validators List of validators to execute
--strict Consider warnings as an errors.
Alternatives¶
Implement validation as part of Murano and validate the package when it is uploaded to Murano.
Pros:
Easier to implement.
Cons:
Suitable for Murano itself, but not for developers.
Integration with CI which requires execution of functional tests and is slower than static validation.
Cannot be integrated with text editors or IDEs.
Data model impact¶
None
REST API impact¶
Murano API for upload packages should use this tool to do package validation before store it. If package is not valid, HTTP error with code 400 should be returned to user with list of check errors in description.
Versioning impact¶
None
Other end user impact¶
Murano application package developer will be able to validate package during development process.
Deployer impact¶
None
Developer impact¶
Application developer will be able to validate application code during development process without importing package to Murano and running deployment process or test suite.
Murano-dashboard / Horizon impact¶
Error message is displayed in Horizon when trying to upload malformed package.
Implementation¶
Application class¶
Application class provides an entry point to the checker. It is responsible for configuration, extensions loading and validation execution.
Application options are registered using oslo.config
Methods:
__init__(self, options)
load_plugins(self)
- Finds validators from installed pluginsprepare_check_suit(self, loaded_package, validator_list)
- Collect prepared checks from all validators in validator_list against specified package. Returns list with prepared checks.list_errors(self, filters)
- Lists errors/warnings exposed by all available validators
Validator classes¶
Validator class represents set for specific checks and logic to register them.
Classes
BaseValidator
+-- PackageValidator
+-- ManifestValidator
+-- UiValidator
+-- ClassValidator
Attributes
short_name
short name for validator. Used as prefix for validator specific errors
Methods
__init__(loaded_package)
- Init validator for specific formatrun(self)
- Runs validator against specified suit of files or whole package if file_suit=None (for Package validator)
Checkers¶
Checker is a function that performs checking logic. It takes data from args and yields error if it is not valid. Checkers should not contain logic for data preparation. Checker can be reused multiple times with different options in a single suit of checks. Checker can be wrapped with custom filters (i.e. manifest version), so it is called only for matching conditions.
Exceptions¶
Class CheckerError
contains information including error code, message,
file name and position in that file. Additionally it can include
code snippet, if it’s available. For YAML data it’s available using
yaml.error.Mark
class. Otherwise code snippet can be generated during
error formatting.
Attributes
code - Error code (Usually single letter and three digits)
message - Human readable error reason
filename - File name relative to package root
line - Line number (starts from 0, optional)
column - Column number (starts from 0, optional)
source - One line code snippet with highlighted column (optional)
Package loaders¶
Package loaders provide interface to Murano packages in multiple formats. Currently packages can be loaded from directory, single file or zip archive package.
PackageLoader
+-- DirectoryLoader
+-- FileLoader
+-- ZipLoader
Methods
try_load(cls, path, *options)
- [classmethod] Tries to load package, if it can be loaded returns instance of loaded package, otherwise None.
Loaded package¶
Loaded package class introduces a general interface for access to packages of different format.
Methods
get_file(self, name)
- File wrapper if file exists, None otherwiseformat(self)
- Ruexists(self, name)
- Returns true if file exists in the package.list(self, subdir=None)
- Returns list of all files in the package (or in it’s subdirectory).search_for(self, regex)
- Return iterator of files fit to regex
File wrapper allows to access to raw file context with method raw()
and
to parsed dict with method parsed()
Formatters¶
Formatter classes help to represent error report in various ways.
Errors can be printed to stdout
/stderr
using text representation,
which suits developer or written to file in JSON
/YAML
format.
Assignee(s)¶
- Primary assignee:
Alexander Saprykin <cutwater>
- Other contributors:
Krzysztof Szukielojc <kszukielojc> Sergey Slipushenko <sslypushenko>
Work Items¶
Develop core framework with support of extensions.
Implement manifest and package structure checks.
Implement Murano PL classes checks.
Implement UI definition file validation.
Implement YAQL checks.
Integrate package validation with murano API.
Develop CLI tool based on core framework
Create documentation.
Dependencies¶
Testing¶
Package should include unit and functional tests.
Documentation Impact¶
Documentation for the library should include:
Usage information that describes how to use the application.
API reference.
Plugin developer documentation.
References¶
None