masakari-specs

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Introduction paragraph – why are we doing anything? A single paragraph of prose that operators can understand. The title and this first paragraph should be used as the subject line and body of the commit message respectively.

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

At this point, if you would like to just get feedback on if the problem and proposed change fit in Masakari, you can stop here and post this for review to get preliminary feedback. If so please say: Posting to get preliminary feedback on the scope of this spec.

Alternatives

What other ways could we do this thing? Why aren’t we using those? This doesn’t have to be a full literature review, but it should demonstrate that thought has been put into why the proposed solution is an appropriate one.

Data model impact

Changes which require modifications to the data model often have a wider impact on the system. The community often has strong opinions on how the data model should be evolved, from both a functional and performance perspective. It is therefore important to capture and gain agreement as early as possible on any proposed changes to the data model.

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Note that the schema should be defined as restrictively as possible. Parameters which are required should be marked as such and only under exceptional circumstances should additional parameters which are not defined in the schema be permitted (eg additionaProperties should be False).

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

For more detailed guidance, please see the OpenStack Security Guidelines as a reference (https://wiki.openstack.org/wiki/Security/Guidelines). These guidelines are a work in progress and are designed to help you identify security best practices. For further information, feel free to reach out to the OpenStack Security Group at openstack-security@lists.openstack.org.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Work items or tasks – break the feature up into the things that need to be done to implement it. Those parts might end up being done by different people, but we’re mostly trying to understand the timeline for implementation.

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Please discuss the important scenarios needed to test here, as well as specific edge cases we should be ensuring work correctly. For each scenario please specify if this requires specialized hardware, a full openstack environment, or can be simulated inside the Masakari tree.

Please discuss how the change will be tested. We especially want to know what tempest tests will be added. It is assumed that unit test coverage will be added so that doesn’t need to be mentioned explicitly, but discussion of why you think unit tests are sufficient and we don’t need to add more tempest tests would need to be included.

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

Which audiences are affected most by this change, and which documentation titles on docs.openstack.org should be updated because of this change? Don’t repeat details discussed above, but reference them here in the context of documentation for multiple audiences. For example, the Operations Guide targets cloud operators, and the End User Guide would need to be updated if the change offers a new feature available through the CLI or dashboard. If a config option changes or is deprecated, note here that the documentation needs to be updated to reflect this specification’s change.

References

Please add any useful references here. You are not required to have any reference. Moreover, this specification should still make sense when your references are unavailable. Examples of what you could include are:

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2024.2 Dalmatian	Introduced

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2024.2 Dalmatian	Introduced

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2025.1 Epoxy	Introduced

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2025.1 Epoxy	Introduced

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2025.2 Flamingo	Introduced

Example Spec - The title of your blueprint

Wed, 19 Mar 2025 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2025.2 Flamingo	Introduced

host monitors by kubernetes

Tue, 30 Jan 2024 00:00:00

https://blueprints.launchpad.net/masakari/+spec/host-monitors-by-kubernetes

Problem description

When using Openstack on Kubernetes, the current host monitoring process lacks efficiency and simplicity, requiring additional software such as Consul or Pacemaker.

Proposed change

This spec is mainly to add a new host monitoring driver to masakari-monitors using the Kubernetes-client. By leveraging the Kubernetes API, kubernetes-native openstack operator can efficiently retrieve and monitor the status of the host (node) without the complexities associated with external configuration tools like Consul or Pacemaker.

Administrator can set the host monitoring driver to Kubernetes in the host configuration.

Alternatives

None

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

This periodically calls Kubernetes-api based on the value of monitoring_interval.

Other deployer impact

“kubernetes” is added to masakari-monitors configuration as the new host.monitoring_driver type for kubernetes-native openstack operator

Developer impact

None

Implementation

Assignee(s)

Primary assignee: * do-gyun kim <dogyun7949@gmail.com>

Work Items

Create a new monitoring driver using the kubernetes-client.
Update docs about host monitors.
Add unit tests.

Dependencies

This feature require python kubernetes-client library.

Testing

Add required unit tests which will run in gate.

Documentation Impact

Update masakari-hostmonitor reference documentation.

References

None

History

Revisions
Release Name	Description
2025.2 Flamingo	Introduced

Example Spec - The title of your blueprint

Mon, 29 Jan 2024 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2024.1 Caracal	Introduced

Example Spec - The title of your blueprint

Mon, 29 Jan 2024 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2024.1 Caracal	Introduced

Example Spec - The title of your blueprint

Thu, 06 Jul 2023 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2023.2 Babcat	Introduced

Example Spec - The title of your blueprint

Thu, 06 Jul 2023 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2023.2 Babcat	Introduced

vm evacuations for host recovery

Tue, 27 Dec 2022 00:00:00

https://blueprints.launchpad.net/masakari/+spec/vm-evacuations-for-host-recovery

Problem description

If one compute node failed, Masakari will evacuate the instances from the failed host.

If a large number of hosts fail at the same time, the resources of computing nodes are dramatically reduced. There would not be enough resources for all instances to recovery. So it is reasonable that the very important instances to be firstly evacuated, and evacuations can be aborted once the cloud environment encounters an irreversible condition.

When the failed hosts come back, the restored resources may be lying idle. In order to make full use of the restored resources, It needs to move instances to the restored hosts. Sometimes there may be a distribution on purpose. The vm moves automatically such as DRS or manually could mess up the distribution. So it is a good idea to save the evaucations when the host is failed, and move instances back when the host is restored according to the previous evaucations.

Proposed change

This spec is mainly to record vm moves information in the database, mainly including instance_uuid, notification_uuid, source_host, dest_host, type, status, start_time and end_time.

User can get vm moves information of a ‘COMPUTE_HOST’ type notification by vmove API.

Alternatives

None

Data model impact

The table vmoves will be added into the Masakari database.

created_at: Datetime.
updated_at: Datetime.
deleted_at: Datetime.
deleted: Boolean.
uuid: UUID. UUID of the vmove.
notification_uuid: UUID. UUID of notification the vmove belong to.
instance_uuid: UUID. UUID of instance.
instance_uuid: String. Name of instance.
source_host: String. Source host name of the vmove.
dest_host: String. Destination host name of the vmove.
start_time: Datetime. Start time of the vmove.
end_time: Datetime. End time of the vmove.
type: String. Represents possible types for the vmove, such as migration, live_migration or evacuation.
status: String. Represents possible statuses for the vmove, such as pending, ongoing, ignored, failed or succeeded.
message: String. Display some meaningful information if the vmove is failed or ignored.

REST API impact

Following vmove API will be introduced in a new API micro-version.

GET /notifications/<notification_id>/vmoves

response example:

{
    "vmoves": [
        {
            "uuid": "239f95ca-fd46-44d2-8ff8-35e8a9c94f69",
            "instance_uuid": "33826ebd-af0f-445d-833f-e06340f7ae1c",
            "instance_name": "vm-1",
            "notification_uuid": "c0fa1a39-c150-4b86-ae97-8fae31700c67",
            "source_host": "node01",
            "dest_host": "node02",
            "start_time": "2022-11-22 14:50:22",
            "end_time": "2022-11-22 14:50:35",
            "type": "evacuation",
            "status": "succeeded",
            "message": null
        },
        {
            "uuid": "65a5da84-5819-4aea-8278-a28d2b489028",
            "instance_uuid": "e1a5a45b-f251-47cf-9c5f-fa1e66e1286a",
            "instance_name": "vm-2",
            "notification_uuid": "c0fa1a39-c150-4b86-ae97-8fae31700c67",
            "source_host": "node01",
            "dest_host": "node02",
            "start_time": "2022-11-22 14:50:23",
            "end_time": "2022-11-22 14:50:38",
            "type": "evacuation",
            "status": "succeeded",
            "message": null
        }
    ]
}

GET /notifications/<notification_id>/vmoves/<vmove_id>

response example:

{
     "vmove":
         {
             "uuid": "239f95ca-fd46-44d2-8ff8-35e8a9c94f69",
             "instance_uuid": "33826ebd-af0f-445d-833f-e06340f7ae1c",
             "instance_name": "vm-1",
             "notification_uuid": "c0fa1a39-c150-4b86-ae97-8fae31700c67",
             "source_host": "node01",
             "dest_host": "node02",
             "start_time": "2022-11-22 14:50:22",
             "end_time": "2022-11-22 14:50:38",
             "type": "evacuation",
             "status": "succeeded",
             "message": null
         }
 }

Security impact

None

Notifications impact

None

Other end user impact

The masakari-dashboard and openstacksdk will be updated to support vm moves for host type notification in a new micro-version.

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

suzhengwei <suzhengwei@inspur.com>

Work Items

Create the object definition, database schema, updating engine to handle this.
Create a new API microversion to get information for all vmoves and get detailed information about a particular vmove.
Update docs about vm moves for host recovery
Update masakari-dashboard and openstacksdk to manage vm moves.
Add unit and functional tests.

Dependencies

None

Testing

Unit and functional test is neccessary.

Add required unit and functional tests which will run in gate.

Documentation Impact

Update Masakari API reference documentation.

References

None

History

Revisions
Release Name	Description
Xena	Introduced
Yoga	Re-proposed

Example Spec - The title of your blueprint

Thu, 06 Oct 2022 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2023.1 Antelope	Introduced

Example Spec - The title of your blueprint

Thu, 06 Oct 2022 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
2023.1 Antelope	Introduced

Example Spec - The title of your blueprint

Tue, 29 Mar 2022 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Zed	Introduced

Example Spec - The title of your blueprint

Tue, 29 Mar 2022 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Zed	Introduced

host monitor by consul

Tue, 19 Oct 2021 00:00:00

https://blueprints.launchpad.net/masakari-monitors/+spec/host-monitor-by-consul

Problem description

Usually, there are management network, tenant network and storage network in one cloud platform. The compute nodes may have management, tenant and storage interface to connect to these three networks.

Currently, Masakari hostmonitor uses pacemaker and pacemaker-remote to monitor hosts’ connection. Actually, it is monitoring the hosts heartbeat through management interface. Once a host’s management connectivity is detected down, it will send notification to masakari to trigger host failure recovery workflow.

This solution has some flaws especially when management connectivity is down and the other two connectivity tenant and storage are up. Users can still access their VMs without any interruptions, so there is no need to send host failure notification in this case.

Proposed change

This spec introduces a new host monitor. Specifically, host connectivity monitoring via management, tenant and storage interfaces by consul agent.

The low-level architecture for host monitoring is shown as below:

Each host runs three consul agents, which respectively bind management, tenant and storage interfaces. They make up three independent consul cluster.

Consul agent runs in server mode on controller nodes, while in client mode on compute nodes.

For example, consul cluster via management connectivity. All agents bind management interface, and are responsible for running checks and keeping services in sync.

Consul is built on top of Serf which provides a full gossip protocol that is used for multiple purposes. Serf provides membership, failure detection, and event broadcast. Consul uses gossip protocol to manage membership. If an agent is found disconnected, it will broadcast messages to the cluster quickly.

Host-monitor periodically retrieves all consul members heath data from local consul agents. It picks out every nodes’ management, tenant and storage health state separately, and combines them together. Then it will send notification depending on the HA strategy - host states and the corresponding actions. There will be a config file for user to decide the HA strategy and the default HA strategy is as follows:

management	tenant	storage	actions
up	up	down	recovery
up	down	down	recovery
down	up	down	recovery
down	down	down	recovery

‘up’ represents connectivity up.
‘down’ represents connectivity down.
‘recovery’ represents host recovery.

User can define the HA strategy according to the physical environment. For example, if there is only one consul cluster of management, the HA strategy would be the same as the existing solution based on pacemaker.

management	actions
down	recovery

Alternatives

None

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

suzhengwei <sugar-2008@163.com>

Work Items

Masakari host-monitor driver based on consul
masakari documentation updates

Dependencies

Requires that consul agents are installed and running to monitor the hosts management, tenant and storage connectivity.

Testing

Unit tests and functional tests will be needed.

Documentation Impact

The admin configuration documentation need to be updated.

References

https://www.consul.io/

History

Revisions
Release Name	Description
Victoria	Introduced
Xena	Re-proposed
Yoga	Re-proposed

host monitor by consul

Tue, 19 Oct 2021 00:00:00

https://blueprints.launchpad.net/masakari-monitors/+spec/host-monitor-by-consul

Problem description

Proposed change

This spec introduces a new host monitor. Specifically, host connectivity monitoring via management, tenant and storage interfaces by consul agent.

The low-level architecture for host monitoring is shown as below:

Each host runs three consul agents, which respectively bind management, tenant and storage interfaces. They make up three independent consul cluster.

Consul agent runs in server mode on controller nodes, while in client mode on compute nodes.

For example, consul cluster via management connectivity. All agents bind management interface, and are responsible for running checks and keeping services in sync.

management	tenant	storage	actions
up	up	down	recovery
up	down	down	recovery
down	up	down	recovery
down	down	down	recovery

‘up’ represents connectivity up.
‘down’ represents connectivity down.
‘recovery’ represents host recovery.

management	actions
down	recovery

Alternatives

None

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

suzhengwei <sugar-2008@163.com>

Work Items

Masakari host-monitor driver based on consul
masakari documentation updates

Dependencies

Requires that consul agents are installed and running to monitor the hosts management, tenant and storage connectivity.

Testing

Unit tests and functional tests will be needed.

Documentation Impact

The admin configuration documentation need to be updated.

References

https://www.consul.io/

History

Revisions
Release Name	Description
Victoria	Introduced
Xena	Re-proposed
Yoga	Re-proposed

Example Spec - The title of your blueprint

Tue, 19 Oct 2021 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Yoga	Introduced

Example Spec - The title of your blueprint

Tue, 19 Oct 2021 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Yoga	Introduced

Add option to enable/disable a segment

Sat, 27 Mar 2021 00:00:00

https://blueprints.launchpad.net/masakari/+spec/enable-to-segment

Problem description

Sometimes, operators want to disable instance-ha temporarily. For example, when we plan to update the hardware or pull the latest code on the compute nodes where the masakari-monitors are running, we are required to stop masakari-api and masakari-engine services so that it won’t execute any ha recovery workflow. If you forgot to stop these services, then it would end up in a mess and you will need to spend lot of your time in recovering instances which are messed up.

So, we need a simple solution which will allow masakari to disable instance-ha for a temporarily period without needing operator to stop these services.

Proposed change

This spec is mainly to add an option to enable/disable a segment

Add enabled option to a segment. When creating a new segment, it will be enabled by default. Option enabled has a boolean value: ‘True’ means the segment is enabled, notifications of this segment will be processed; ‘False’ means the segment is disabled, notifications of this segment will be ignored.
User can modify enabled option by PUT /segment/<segment_uuid> API.

Alternatives

None

Data model impact

Yes, add a new db column enabled of type Boolean with default value True to failover_segments table.

REST API impact

Following changes will be introduced in a new API micro-version.

POST /segments

request example:

{
    "segment": {
        "service_type": "COMPUTE",
        "recovery_method": "AUTO",
        "name": "segment",
        "enabled": True
    }
}

response example:

{
    "segment": {
        "uuid": "5fd9f925-0379-40db-a7f8-786a0b655b2a",
        "deleted": false,
        "created_at": "2017-04-21T08:59:53.991030",
        "description": null,
        "recovery_method": "AUTO",
        "updated_at": null,
        "service_type": "COMPUTE",
        "deleted_at": null,
        "id": 4,
        "name": "segment",
        "enabled": True
    }
}

PUT /segments/{segment_id}

request example:

{
    "segment": {
        "name": "new_segment",
        "enabled": False
    }
}

response example:

{
    "segment": {
        "uuid": "5fd9f925-0379-40db-a7f8-786a0b655b2a",
        "deleted": false,
        "created_at": "2017-04-21T08:59:54.000000",
        "description": null,
        "recovery_method": "AUTO",
        "updated_at": "2017-04-21T09:47:03.748028",
        "service_type": "COMPUTE",
        "deleted_at": null,
        "id": 4,
        "name": "new_segment",
        "enabled": False
    }
}

GET /segments

response example:

{
    "segments": [
        {
            "uuid": "9e800031-6946-4b43-bf09-8b3d1cab792b",
            "deleted": false,
            "created_at": "2017-04-20T10:17:17.000000",
            "description": "Segment1",
            "recovery_method": "auto",
            "updated_at": null,
            "service_type": "Compute",
            "deleted_at": null,
            "id": 1,
            "name": "segment2",
            "enabled": True
        }
    ]
}

GET /segments/<segment_uuid>

response example:

{
    "segment": {
        "uuid": "5fd9f925-0379-40db-a7f8-786a0b655b2a",
        "deleted": false,
        "created_at": "2017-04-21T08:59:53.991030",
        "description": null,
        "recovery_method": "AUTO",
        "updated_at": null,
        "service_type": "COMPUTE",
        "deleted_at": null,
        "id": 4,
        "name": "new_segment",
        "enabled": False
    }
}

Security impact

None

Notifications impact

Field enabled will be added into segment in masakari notifications. For example the wire format of the create.segment.start notification looks like the following:

{
    "event_type": "segment.create.start",
    "message_id": "e44cb15b-dcba-409e-b0e1-9ee103b9a168",
    "payload": {
        "masakari_object.data": {
            "description": null,
            "fault": null,
            "name": "test",
            "recovery_method": "auto",
            "service_type": "compute",
            "enabled": True
        },
        "masakari_object.name": "SegmentApiPayload",
        "masakari_object.namespace": "masakari",
        "masakari_object.version": "1.1"
    },
    "publisher_id": "masakari-api:fake-mini",
    "timestamp": "2018-11-22 09:25:12.393979"
}

Other end user impact

The python-masakariclient, masakari-dashboard and openstacksdk will be updated to support enabled parameter of the segment in a new micro-version.

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

suzhengwei <sugar-2008@163.com>

Work Items

Create a new API microversion to handle enabled parameter in segments.
Update docs for enabled to segment
Update python-masakariclient, masakari-dashboard and openstacksdk to manage enabled parameter of the segment..
Add functional tests

Dependencies

None

Testing

Unit and functional test is neccessary.

Add required unit and functional tests which will run in gate.

Documentation Impact

Update Masakari API reference documentation.

References

None

History

Revisions
Release Name	Description
Ussuri	Introduced
Victoria	Re-proposed

Repeated check to determine host status

Sat, 27 Mar 2021 00:00:00

https://blueprints.launchpad.net/masakari-monitors/+spec/retry-check-when-host-failure

Problem description

If the platform has a bad network stability, judging from pacemaker and corosync, the host status would swing between up and down. If the host monitor react quickly in this condition, it would result in host recovery, which is not expected.

Proposed change

It is imprecise to determine host status by once check. repeated checks is more reliable.

While the host-monitor keeps a sequence of one host’ latest status. The host status is ‘down’ only if the sequence of its latest status is consistently ‘down’.

The length of the sequence is determined by the monitoring_samples configuration, which has a default int value 1. It means the host status is ‘down’ only if the once check status of the host is ‘down’.

Meanwhile, the default value of the configuration monitoring_interval is suggested to set to 60 seconds, just the same as before.

Alternatives

None

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

suzhengwei <sugar-2008@163.com>

Work Items

None

Dependencies

None

Testing

Unit tests are needed.

Documentation Impact

Update user documentations.

References

None

History

Revisions
Release Name	Description
Wallaby	Introduced

Example Spec - The title of your blueprint

Fri, 05 Mar 2021 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Xena	Introduced

Example Spec - The title of your blueprint

Fri, 05 Mar 2021 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Xena	Introduced

Example Spec - The title of your blueprint

Wed, 18 Nov 2020 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see the Masakari docs, especially the Contributor’s section.
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Wallaby	Introduced

Example Spec - The title of your blueprint

Mon, 27 Apr 2020 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Victoria	Introduced

Example Spec - The title of your blueprint

Mon, 27 Apr 2020 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Victoria	Introduced

Example Spec - The title of your blueprint

Mon, 02 Sep 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Ussuri	Introduced

Example Spec - The title of your blueprint

Mon, 02 Sep 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Ussuri	Introduced

Masakari Spec Lite

Mon, 15 Jul 2019 00:00:00

Please keep this template section in place and add your own copy of it between the markers. Please fill only one Spec Lite per commit.

<Title of your Spec Lite>

link:: <Link to the blueprint.>
problem:: <What is the driver to make the change.>
solution:: <High level description what needs to get done. For example: “We need to add client function X.Y.Z to interact with new server functionality Z”.>
impacts:: <All possible *Impact flags (same as in commit messages) or ‘None’.>

Optionals (please remove this line and fill or remove the rest until End):

how:: <More technical details than the high level overview of solution if needed.>
alternatives:: <Any alternative approaches that might be worth of bringing to discussion.>
timeline:: <Estimation of the time needed to complete the work.>
reviewers:: <If reviewers has been agreed for the functionality, list them here.>
assignee:: <If known, list who is going to work on the feature implementation here>

End of Template

Add db purge support

link:

https://blueprints.launchpad.net/masakari/+spec/db-purge-support

problem:

Masakari do not have any way to delete notification records from notification database table. If there are large number of notifications in the db, it’s going to slow down db search. Similarly we can purge records from other database tables.

solution:

Add db purge support to masakari-manage command which will purge the deleted/unused records from the database. As masakari do not have delete notification api, notification records will be purged on the basis of status and updated_at column. Notifications which are in finished, ignored and failed status will be purged from the notification table. For other tables records which are marked as deleted will be purged from the tables. Two optional command line configuration options ‘age_in_days’ defaults to 30 and ‘max_rows’ defaults to -1 will be introduced to provide operator flexibility in purging the records. -1 means purge command will remove all records from tables which matches ‘age_in_days’ criteria. If operator specifies ‘max_rows’ greater than 0 then those many records will be purged from entire database (not per table) in one operation.

For notification table as we do not have deleted_at value; ‘age_in_days’ will be calculated on the basis of updated_at value. For example, if ‘age_in_days’ is mentioned as 10 while purging then notifications which are having status finished, ignored and failed and updated before 10 days will be eligible for purging. For other tables, records which were deleted before 10 days will be eligible for purging.

Example: $ masakari-manage db purge This will purge all records from each table which are deleted or updated before 30 days.

$ masakari-manage db purge –age_in_days 60 This will purge all records from each table which are deleted or updated before 60 days.

$ masakari-manage db purge –max-rows 50 This will purge total 50 records from entire database which are deleted or updated before 30 days.

$ masakari-manage db purge –age_in_days 60 –max-rows 50 This will purge 50 records from entire database which are deleted or updated before 60 days.

alternatives:

Add new delete_all api which will have status and age_in_days as input parameters to delete the notifications. In this case user can specify input notification status as either ignored, failed or finished and based on age_in_days value those records will be deleted from the notification table. For example if user specifies status as ‘finished’ and age_in_days as 10 then notifications which are having ‘finished’ status and updated before 10 days will be deleted from the notification table.

Advantages: 1. Operator has flexibility to decide which notifications needs to be deleted from the notification table.

Disadvantages: 1. Only notification records will be deleted in this case, records from other tables will remain as it is.

impacts:

None

timeline:

Expected to be merged within the Pike time frame.

reviewers:

sam47priya@gmail.com, sagaray@nttdata.co.jp, tushar.vitthal.patil@gmail.com

assignee:

Pooja Jadhav

Add db purge support

Send Event Notifications

Mon, 15 Jul 2019 00:00:00

https://blueprints.launchpad.net/masakari/+spec/notifications-in-masakari

Problem description

Currently, Masakari doesn’t send any event notifications on hosts, failover segments and notifications RestFul API operation requests made by an user.

It would be useful to receive create, update and delete event notifications on any of this information changing, and the payload should contain the same information for create and update as accessible from the API.

Use Cases

Operators will be able to know following things by event notifications:

Begin/End of API process
Errors in event of failure to process APIs request
Payload which will contain the contents of the API request

Operators can use event notifications for analyzing, monitoring and metering purposes.

Proposed change

Versioned notifications will be emitted for the following events:

Segments

Create segment
segment.create.start

segment.create.end

segment.create.error

Update segment
segment.update.start

segment.update.end

segment.update.error

Delete segment
segment.delete.start

segment.delete.end

segment.delete.error
Hosts

Create host
host.create.start

host.create.end

host.create.error

Update host
host.update.start

host.update.end

host.update.error

Delete host
host.delete.start

host.delete.end

host.delete.error
Notifications

Create notification
notification.create.start

notification.create.end

notification.create.error

Process notification
notification.process.start

notification.process.end

notification.process.error

Note

Process notification event is emitted only when masakari-engine starts processing received notifications by executing recovery workflow.

Event notification generally contain following information:

{
    "priority": <string, selected from a predefined list[2] by the sender>,
    "event_type": <string, defined by the sender>,
    "timestamp": <string, the isotime of when the notification emitted>,
    "publisher_id": <string, defined by the sender>,
    "message_id": <uuid, generated by oslo>,
    "payload": <json serialized dict, defined by the sender>
}

Versioned notifications:

Similar to the other OpenStack services Masakari will emit event notification to the message bus with the Notifier class provided by oslo.messaging-doc.

In versioned notification, the payload isn’t a free form dictionary but a serialized oslo versioned object. In other words, the payload should be the format which is serializable by oslo-versionedobjects library.

This is a sample of segment.create.start versioned notification:

{
    "event_type": "segment.create.start",
    "timestamp": "2018-11-22 09:25:12.393979",
    "payload": {
        "masakari_object.name": "SegmentApiPayload",
        "masakari_object.data": {
            "service_type": "compute",
            "fault": null,
            "recovery_method": "auto",
            "description": null,
            "name": "test"
        },
        "masakari_object.version": "1.0",
        "masakari_object.namespace": "masakari"
    },

    "publisher_id": "masakari-api:fake-mini",
    "message_id": "e44cb15b-dcba-409e-b0e1-9ee103b9a168"
}

Alternatives

None

Data model impact

Add osloversioned.objects for hosts, failover segment and notification. No changes will be made to the database schema.

REST API impact

None

Security impact

None

Notifications impact

Masakari doesn’t support event notification feature. This spec will add this new feature.

Other end user impact

None

Performance Impact

There will be a slight performance impact due to the overhead of sending event notifications during processing of each RestFul API request. Operator can also disable event notifications completely using configuration options.

Other deployer impact

Following config section and option will be added in masakari.conf:

[oslo_messaging_notifications]
driver=messaging

driver
Type: multi-valued
Default:''
The Drivers(s) to handle sending notifications. Possible values are messaging, messagingv2, routing, log, test, noop

Deployers should prepare the messaging system (e.g.RabbitMQ) to receive event notifications if they want to use event notification feature.

Developer impact

After this feature lands in the code, henceforth, developers will need to add new event notifications if they decide to add new RestFul APIs.

Implementation

Assignee(s)

Primary assignee:

Rikimaru Honjo <honjo.rikimaru@po.ntt-tx.co.jp>
Kengo Takahara <takahara-kn@njk.co.jp>
Shilpa Devharakar <shilpa.devharakar@nttdata.com>

Work Items

Add base classes for event notification mechanism
Add osloversioned.objects to be used to send event notifications
Add methods to send notifications for each newly added osloversioned.object
Send event notification for create/update/delete operations
Add unit tests for code coverage
Add documentation on how to use this feature

Dependencies

None

Testing

No need to write tempest tests as unit tests are sufficient to check whether the event notifications are sent or not for each create, update and delete operations.

Documentation Impact

Add documentation to explain how to use event notification feature so that operator can write code to receive these events for their own purpose.

References

History

Revisions
Release Name	Description
Queens	Introduced
Rocky	Approved
Stein	Re-proposed

Example Spec - The title of your blueprint

Mon, 15 Jul 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Train	Introduced

Example Spec - The title of your blueprint

Mon, 15 Jul 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Train	Introduced

Example Spec - The title of your blueprint

Mon, 28 Jan 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Stein	Introduced

Example Spec - The title of your blueprint

Mon, 28 Jan 2019 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Stein	Introduced

Add progress details for recovery workflows

Fri, 04 Jan 2019 00:00:00

https://blueprints.launchpad.net/masakari/+spec/progress-details-recovery-workflows

This blueprint proposes to have a feature that notifies events for recovery workflows.

Problem description

Currently, Masakari doesn’t send any events during recovery operation request received by Masakari monitor.

It would be useful to receive events at each stage of task of recovery workflow along with completion status and progress details so that operator will come to know about what’s happening during execution.

Use Cases

Operators will be able to know following things by detailed progress details captured during each event of recovery:

Beginning/End of each task of recovery flow
Errors of failure of process recovery
Progress details which will contain the details of each task

Proposed change

Masakari Recovery Workflow is a certain set of tasks executed to recover from failure. Masakari supports three types of recovery failures:

instance-failure
process-failure
host-failure

For each of these failures, Masakari executes a workflow to recover from failure. Currently Masakari uses taskflow library to execute the workflow which consists of recovery actions which are predefined and are executed linearly. Proposing here to record these recovery actions with the help of Taskflow persistence feature. Masakari will persist the flow so that it can be resumed, restarted or rolled-back on engine failure.

Taskflow supports persistence of workflow which helps to persist each task details in the database. For more details please refer persistence-doc

Taskflow has below three tables where workflow/task details are getting stored:

logbooks
flowdetails
atomdetails

In particular, for each flow there is a corresponding flowdetails record, and for each task there is a corresponding atomdetails record. These form the basic level of information about how a flow will be persisted.

With the help of importing persistence package taskflow_persistence and by accessing Masakari storage via masakari engine, able to import Taskflow tables into Masakari. In taskflow library there is workflow, and each workflow has task which has state and status. With the help of notifier_method will update progress details for detailed execution flow for each task of recovery.

Saved recovery task details (failures, successes, intermediary results) going to render on Horizon on tabular format which helps operators to understand progress/status of recovery. Each flow execution details stored with scale 0 to 1, so that operator will able to get progress completion along with detailed information of each task.

Explaining below the how actions/events that going to be recorded for ‘instance-failure recovery workflow’ along with progress details:

Stop Instance Task: Below listed are possible events along with progress details that will be recorded:

Starting of Stop instance task:

"progress_details" = {
  "progress": 0.50,
  "progress_data": "Started execution of StopInstanceTask <INSTANCE_UUID>"
}

Skipping recovery event if an instance is not HA_Enabled and “process_all_instances” config option is also disabled:

"progress_details" = {
  "progress": 1,
  "progress_data": "Skipping recovery for instance <INSTANCE_UUID> as it is not Ha_Enabled"
}

Ignored recovery event if an instance VM state is either in ‘paused’, ‘rescued’:

"progress_details" = {
  "progress": 1,
  "progress_data": "Ignoring recovery for instance <INSTANCE_UUID> as it is in paused/rescued state"
}

Stop instance event:

"progress_details" = {
  "progress": 1,
  "progress_data": "Finished execution of StopInstanceTask <INSTANCE_UUID>"
}

Failure event in case failed to stop instance:

"progress_details" = {
  "progress": 1,
  "progress_data": "Failed to stop instance <INSTANCE_UUID>"
}

Start Instance Task: Below listed are possible events along with progress details that will be recorded:

Start instance event:

"progress_details" = {
  "progress": 0.5,
  "progress_data": "Started execution of StartInstanceTask <INSTANCE_UUID>"
}

Finish of Start instance event:

"progress_details" = {
  "progress": 1,
  "progress_data": "Finished execution of StartInstanceTask <INSTANCE_UUID>"
}

Failure event in case failed to start instance or if invalid state of it:

"progress_details" = {
  "progress": 1,
  "progress_data": "Failed to start instance <INSTANCE_UUID>"
}

Confirm Instance Active Task: Below listed are possible events along with progress details that will be recorded:

Start of Confirm instance event:

"progress_details" = {
  "progress": 0.5,
  "progress_data": "Confirming instance <INSTANCE_UUID> is Active"
}

Finish of Confirm instance started event:

"progress_details" = {
  "progress": 1,
  "progress_data": "Confirmed instance <INSTANCE_UUID> is Active"
}

Failure event in case failed to confirm instance:

"progress_details" = {
  "progress": 1,
  "progress_data": "Failed to confirm instance <INSTANCE_UUID>"
}

Note

Events are emitted only when masakari engine starts processing received notifications by executing recovery workflow.

Mentioning below the database entries that going to be recorded for ‘instance-failure recovery workflow’:

LogBook: 'instance_recovery'
- uuid = 68e86fda-25ba-4b1d-a9fc-d999bc1c796e
- created_at = 2019-01-08 08:15:21
- updated_at = 2019-01-08 08:15:21
- meta: {"notification_uuid": "9ca38361-eef9-4fca-a1fe-49ef0c7e23e8"}
FlowDetail: 'instance_recovery_engine'
- uuid = 6a780ae7-9c63-42d9-8510-aa020d7ee566
- state = SUCCESS
TaskDetail: 'StopInstanceTask'
- uuid = c165b8c2-5123-4489-99c1-97eafff72d24
- state = SUCCESS
- version = 1.0
- failure = False
- meta: {}
- results: <CONTEXT_DETAILS>
TaskDetail: 'StopInstanceTask'
- uuid = c165b8c2-5123-4489-99c1-97eafff72d24
- state = SUCCESS
- version = 1.0
- failure = False
- meta:
    + progress = 100.00%
    + progress_details = {
        "progress": 1,
        "progress_details": {
            "at_progress": 1,
            "details": {
                "progress_details": [
                    "progress_details" = {<progress_details_of_event_1>, <progress_details_of_event_2>, ..., <progress_details_of_event_n>}
                ]
            }
        }
     }
- results: NULL
TaskDetail: 'StartInstanceTask'
- uuid = a4155556-fb5a-44f8-b8aa-ab8ecfe8f1ce
- state = SUCCESS
- version = 1.0
- failure = False
- meta:
    + progress = 100.00%
    + progress_details = {
        "progress": 1,
        "progress_details": {
            "at_progress": 1,
            "details": {
                "progress_details": [
                    "progress_details" = {<progress_details_of_event_1>, <progress_details_of_event_2>, ..., <progress_details_of_event_n>}
                ]
            }
        }
     }
- results: NULL
TaskDetail: 'ConfirmInstanceActiveTask'
- uuid = 0ea82633-599b-422d-8fd2-df2057efb29d
- state = SUCCESS
- version = 1.0
- failure = False
- meta:
    + progress = 100.00%
    + progress_details = {
        "progress": 1,
        "progress_details": {
            "at_progress": 1,
            "details": {
                "progress_details": [
                    "progress_details" = {<progress_details_of_event_1>, <progress_details_of_event_2>, ..., <progress_details_of_event_n>}
                ]
            }
        }
     }
- results: NULL

Mentioning below how the recorded data will be used to render task details in tabular format for ‘instance-failure recovery workflow’ on Horizon:

* Stop Instance Task
============================================  ==========================  ==========================  ====================================================
Request ID                                    Action                      Start Time                  Message
============================================  ==========================  ==========================  ====================================================
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      StopInstanceTask            Jan 10 2019, 10:40 a.m      Started execution of StopInstanceTask <INSTANCE_UUID>
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      StopInstanceTask            Jan 10 2019, 10:41 a.m      Finished execution of StopInstanceTask <INSTANCE_UUID>
============================================  ==========================  ==========================  ====================================================

* Start Instance Task
============================================  ==========================  ==========================  ====================================================
Request ID                                    Action                      Start Time                  Message
============================================  ==========================  ==========================  ====================================================
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      StartInstanceTask           Jan 10 2019, 10:41 a.m      Starting instance <INSTANCE_UUID>
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      StartInstanceTask           Jan 10 2019, 10:42 a.m      Started instance <INSTANCE_UUID>
============================================  ==========================  ==========================  ====================================================

* Confirm Instance Active Task
============================================  ==========================  ==========================  ====================================================
Request ID                                    Action                      Start Time                  Message
============================================  ==========================  ==========================  ====================================================
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      ConfirmInstanceActiveTask   Jan 10 2019, 10:43 a.m      Confirming instance is Active <INSTANCE_UUID>
req-679033b7-1755-4929-bf85-eb3bfaef7e0b      ConfirmInstanceActiveTask   Jan 10 2019, 10:43 a.m      Confirmed instance is Active <INSTANCE_UUID>
============================================  ==========================  ==========================  ====================================================

Alternatives

Send Versioned notifications similar to the other OpenStack services for recovery workflows.

Data model impact

Below tables will get added into Masakari Database

alembic_version
logbooks
flowdetails
atomdetails

Note

alembic_version here stores version information of taskflow database version, not of Masakari database. Masakaari database as of now is not under alembic control.

For example in case of ‘instance-failure recovery workflow’, data will be stored in below columns

logbooks: Parent table, one entry for each notification received.
flowdetails: Child table for logbooks, one entry for each notification received.
atomdetails: Child table for flowdetails, one entry for each task of recovery.

Note

Foreign key association is not there for taskflow persistence tables. If we delete logbook entry, respective child entries also got deleted.

REST API impact

A new microversion will be created to add event details to GET /notifications/<notification_uuid> API.

Security impact

None

Notifications impact

Masakari recovery failure doesn’t support event notification feature. This spec will add this feature.

Other end user impact

None

Performance Impact

There will be a slight performance impact due to the overhead for storing events during processing of each recovery failure into database.

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

Jayashri Bidwe <Jayashri.Bidwe@nttdata.com>
Vrushali Kamde <Vrushali.Kamde@nttdata.com>

Work Items

Fetch backend as Masakari backend for each taskflow
Execute taskflow with all details at each task that required
Populate meta with progress status
Update the notification API for GET /notifications/<notification_uuid> in a new microversion to pass the stored event related information of recovery failure
Update unit tests for code coverage
Add documentation on how to use this feature at Horizon

Dependencies

None

Testing

No need to write tempest tests as unit tests are sufficient to check whether the events are sent or not for recovery operations.

Documentation Impact

None

References

History

Revisions
Release Name	Description
Stein	Introduced

Recovery method customization

Mon, 07 May 2018 00:00:00

https://blueprints.launchpad.net/masakari/+spec/recovery-method-customization

This spec talks about making recovery workflow configurable. Operator can configure the workflow in a config file which can be used to build and execute the recovery workflow.

What is recovery workflow?

Recovery workflow is nothing but certain set of actions executed to recover from failure. Masakari supports three types of recovery failures:

instance-failure
process-failure
host-failure

For each of these failures, Masakari executes a workflow to recover from failure on receiving the notification.

Problem description

Masakari uses taskflow library to execute the workflows which consists of recovery actions which are predefined and are executed linearly. If operator wants to add/remove any existing recovery actions to any of these workflow, then there is no other way to do so without making changes in the code. For example in case of ‘host-failure recovery workflow’, predefined workflow is:

disable_compute_node
prepare_ha_enabled_instances
evacuate and confirm_evacuate

If operator wants to remove task ‘disable_compute_node’ from the workflow or add a new task such as send an alert mail to operator then it’s not possible with the current implementation.

Use Cases

Operator may want to add/remove tasks from the existing workflow based on their requirements. For example, in case of ‘host-failure recovery’ workflow predefined flow is; disable_compute_node, prepare_ha_enabled_instances, evacuate and confirm_evacuate. Some of the possible recovery workflow combinations can be:

send Alert/Mail to operator/users of vms -> disable_compute_node -> prepare_ha_enabled_instances -> evacuate -> update the pricing/ metering DB -> confirm_evacuate -> send Alert/Mail to operator/user (recovery done)
send Alert/Mail to operator/users of vms -> disable_compute_node -> prepare_ha_enabled_instances -> evacuate -> confirm_evacuate -> send Alert/Mail to operator/users of vms (recovery done)
send Alert/Mail to operator/users of vms

Proposed change

Make a provision to add/remove tasks from the existing workflow based on the requirements. We plan to decompose the existing hard-coded recovery workflow into separate tasks and then tied them together to form a workflow which can be configured in a new conf ‘masakari-custom-recovery-methods.conf’ file as explained below: Add a section ‘[taskflow_driver_recovery_flows]’ in newly added masakari-custom-recovery-methods.conf file. Under this add below config options for configuration of customized recovery actions for each type of workflow. Each config option will be dictionary containing key/value pairs for tasks to be executed. For example: pre:[v1,v2],main:[v1,v2],post:[v1,v2,v3] Here key will be pre/main/post and value will be the list of tasks to execute for recovery failure. If file does not exist, then default tasks will be executed that will be configured during registration of configuration options.

‘instance_failure_recovery_tasks’ is a dictionary containing key as
pre/main/post and value will be the comma-separated list of tasks to be executed for process failure.
‘process_failure_recovery_tasks’ is a dictionary containing key as
pre/main/post and value will be the comma-separated list of tasks to be executed for process failure.
‘host_auto_failure_recovery_tasks’ is a dictionary containing key as
pre/main/post and value will be the comma-separated list of tasks to be executed for host failure for auto recovery.
‘host_rh_failure_recovery_tasks’ is a dictionary containing key as
pre/main/post and value will be the comma-separated list of tasks to be executed for host failure for rh recovery.

For example,

[taskflow_driver_recovery_flows]
instance_failure_recovery_tasks = pre:['custom_pre_task','stop_instance_task'],
                                  main:['start_instance_task','custom_main_task'],
                                  post:['confirm_instance_active_task','custom_post_task']
process_failure_recovery_tasks = pre:['disable_compute_node_task'],
                                 main:['confirm_compute_node_disabled_task','custom_main_task'],
                                 post:['custom_post_task']
host_auto_failure_recovery_tasks = pre:['disable_compute_service_task'],
                                   main:['prepare_HA_enabled_instances_task'],
                                   post:['evacuate_instances_task','custom_post_task']
host_rh_failure_recovery_tasks = pre:['custom_pre_task','disable_compute_service_task'],
                                 main:['prepare_HA_enabled_instances_task',
                                 'evacuate_instances_task']
                                 post:['custom_post_task']

Need to add entry point for each task in setup.cfg so that these tasks can be loaded dynamically using stevedore during creation of a recovery workflow.

For example, Masakari setup.cfg will have following entry points:

For each entry point in setup.cfg should have the full class path as mentioned in below example:

masakari.task_flow.tasks =
    stop_instance_task = masakari.engine.drivers.taskflow.instance_failure:StopInstanceTask

masakari.task_flow.tasks =
    disable_compute_service_task = <full_class_path_of_task>
    prepare_HA_enabled_instances_task = <full_class_path_of_task>
    evacuate_instances_task = <full_class_path_of_task>
    stop_instance_task = <full_class_path_of_task>
    start_instance_task = <full_class_path_of_task>
    confirm_instance_active_task = <full_class_path_of_task>
    disable_compute_node_task = <full_class_path_of_task>
    confirm_compute_node_disabled_task = <full_class_path_of_task>

If operator wants to configure customized tasks in a Third Party library, then they will need to follow below guidelines to associate newly added tasks with the respective recovery workflows in Masakari:

First make sure required Third Party Library is installed on the Masakari engine node.
Configure custom task in Third Party Library’s setup.cfg as below:

For example, Third Party Libraries setup.cfg will have following entry points

masakari.task_flow.tasks =
    custom_pre_task = <custom_task_class_path_from_third_party_library>
    custom_main_task = <custom_task_class_path_from_third_party_library>
    custom_post_task = <custom_task_class_path_from_third_party_library>

Note:: Entry point in Third Party Library’s setup.cfg should have same key as in Masakari setup.cfg for respective failure recovery.

If there are any configuration parameters required for custom task, then add them into masakari-custom-recovery-methods.conf under the same group/section where they are registered in Third Party Library. Operator will be responsible to generate masakari configuration file by themselves.
Operator should ensure output of each task should be made available to the next tasks needing them.

Alternatives

For recovery from failures, instead of fully configurable task flow, one can add custom tasks at the start or after completion of predefined existing workflow.

One can customized recovery workflow in masakari-custom-recovery-methods.conf as below and Masakari will inject these custom tasks at start or end of the predefined workflow as per requirement.

For example,

[taskflow_driver_recovery_flows]
instance_failure_recovery_tasks = ['custom_pre_task','custom_main_task']
process_failure_recovery_tasks = ['custom_pre_task']
host_auto_failure_recovery_tasks = ['custom_pre_task','custom_main_task']
host_rh_failure_recovery_tasks = ['custom_pre_task']

custom_pre_task and custom_main_task will be executed at the start or end of the existing ‘instance_failure’ workflow.

Note:: For host failure having recovery method as rh, developer should add custom task in nested flow so that it will execute once.

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

A new config file ‘masakari-custom-recovery-methods.conf’ will be added, where ‘taskflow_driver_recovery_flows’ section need to be update for customized recovery workflows. If an operator doesn’t want any customization to any of the recovery workflows, then there will be no impact as it will load the default tasks for each recovery workflow. For example,

[taskflow_driver_recovery_flows]
instance_failure_recovery_tasks = pre:['custom_pre_task','stop_instance_task'],
                                  main:['start_instance_task','custom_main_task'],
                                  post:['confirm_instance_active_task','custom_post_task']
process_failure_recovery_tasks = pre:['disable_compute_node_task'],
                                 main:['confirm_compute_node_disabled_task','custom_main_task'],
                                 post:['custom_post_task']
host_auto_failure_recovery_tasks = pre:['disable_compute_service_task'],
                                   main:['prepare_HA_enabled_instances_task'],
                                   post:['evacuate_instances_task','custom_post_task']
host_rh_failure_recovery_tasks = pre:['custom_pre_task','disable_compute_service_task'],
                                 main:['prepare_HA_enabled_instances_task',
                                 'evacuate_instances_task']
                                 post:['custom_post_task']

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

bhagyashris <bhagyashri.shewale@nttdata.com>

Work Items

Implement customize task flow execution.
Add unit tests for the coverage.
Add documentation guide to describe how to configure customizable workflows.

Dependencies

None

Testing

None

Documentation Impact

Add documentation guide to describe how to configure customizable workflows.

References

https://etherpad.openstack.org/p/masakari-recovery-method-customization http://eavesdrop.openstack.org/meetings/masakari/2018/masakari.2018-07-03-03.00.log.html

History

None

Implement RESERVED_HOST recovery action for host failure workflow

Tue, 10 Apr 2018 00:00:00

https://blueprints.launchpad.net/masakari/+spec/implement-recovery-methods

This spec talks about adding RESERVED_HOST recovery action for host failure workflow. In masakari each failover segment has recovery_method defined for it, so that if any of the host within that failover segment goes down then recovery action will be executed to evacuate “HA_Enabled” or all instances depending on “evacuate_all_instances” configuration option from that host based on recovery_method.

What is RESERVED_HOST?

In each failover segment operators will keep some hosts as reserved by disabling the compute service on those hosts and setting “reserved” property of that host as “True”. As a result, these hosts are not selected by the nova scheduler while provisioning new instances or for performing any other instance actions such as resize, migration etc. These hosts can be used by masakari engine for evacuating the instances from the failed host. Once the reserved host is used for evacuating the instances it is no longer treated as reserved and nova scheduler can use that host for scheduling the instances.

Problem description

Masakari provides a driver interface for implementing the workflows synchronously or asynchronously. Whoever wants to implement the workflow can inherit the masakari driver and implement the workflows.

For implementing the RESERVED_HOST recovery action masakari engine should provide list of reserved hosts associated with its failover segment to the driver. Its then job of the driver to execute the workflow and use this list for evacuating the instances from failed host. One of the task of the workflow is to enable the compute service on reserved host so that instances can be evacuated on that host. At the same time “reserved” property of that host needs to be set to False. There is a possibility for multiple host failures under one failover segment may take the same reserved host and start the recovery workflows at the same time. To avoid this situation, lock with current reserved host name will be acquired on each of the task and that reserved host will be skipped if the lock acquired and evacuation will be done on the next reserved host from the list.

Use Cases

Operator may want to execute host_failure workflow using ‘RESERVED_HOST’ recovery method.

Proposed change

Masakari engine should execute the workflows synchronously only. Masakari engine will load all the drivers. Whoever is going to implement the new driver, it should be the responsibility of that driver to get the result of workflow and send it back to the masakari engine. If someone wants to add a driver which will execute the workflow on a different host and not on the same host where masakari engine is running then they will need to design that driver in such a way that workflow will execute on any host in asynchronous way and send back the result to the masakari engine, so that masakari engine will set the notification status to “ERROR” or “FINISHED” based on the results.

To implement “reserved_host” recovery method, we need to implement lock mechanism over reserved host so that masakari-engine don’t use same reserved host for multiple failure notifications. There are two ways to implement the lock mechanism:

Use oslo_concurrency.lockutils file based lock: Easy to implement, but cannot manage lock among multiple nodes.
Implement lock mechanism using Tooz: Operator would want to deploy multiple masakari-engine services on different nodes. For this purpose we recommend to use distributed locking mechanism provided by Tooz library. By default Tooz would be configured to use file locks, so everything will work as oslo_concurrency lock mechanism. If operator would want to run multiple masakari-engine services he/she would need to configure Tooz backend service and set it in masakari.conf. Currently most reliable Tooz backends are ZooKeeper and Redis.

As of now, in masakari file based lock (oslo_concurrency.lockutils) is already used. Same mechanism will be used to acquire lock on reserved host. Tooz support will be added later and all the existing locks will be migrated to use Tooz locking mechanism.

Pros:

No need to change current masakari engine implementation.
It’s easy to implement other recovery actions such as “AUTO_PRIORITY” and “RH_PRIORITY” with this design.

Alternatives

Execute the workflows asynchronously i.e. either workflow can be executed on same host where masakari engine is running or on a different host altogether. In this case masakari engine might not get the results from the workflow execution and will not able to update notification status and set reserved hosts to False.

To achieve this masakari engine will generate a callback URL based on the notification_id and pass it to the driver. Sample callback URL will be like,

http://<host>:<port>/v1/notification/<notification_id>

Driver will further pass this URL and the required information such as reserved host list, host name etc. to the workflow. Workflow will be responsible to call the masakari using callback URL with notification status and reserved hosts used by workflow as a body of the request.

Once this request is received by masakari, then based on the notification_id it will map it to the notification from the database table and update the status of the notification accordingly. Also masakari will get the list of used reserved hosts in request body, so it will loop through it and set those host’s “reserved” property as False.

Rest API can be:

method: PUT
URL: URL that is passed to the workflow(contains notification_id)
Body:
result {
    notification_status: status of the notification, either 'error' or
    'finished', used_reserved_hosts: list of actually used reserved_hosts
    by workflow
}

Pros:

Better approach to implement new workflow drivers.

Cons:

The other service which is going to request masakari using REST api should have required admin credentials to call the API.
Need to change current driver (taskflow) implementation to adopt this design.
Need to modify PUT api to incorporate this change.

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

None

Developer impact

None

Implementation

Assignee(s)

Primary assignee:: Dinesh Bhor <dinesh.bhor@nttdata.com> Abhishek Kekane <abhishek.kekane@nttdata.com>

Work Items

Implement RESERVED_HOST recovery_method for host_failure recovery in synchronous way for taskflow driver
Add unit tests for the coverage

Dependencies

None

Testing

None

Documentation Impact

None

References

http://eavesdrop.openstack.org/meetings/masakari/2016/masakari.2016-12-13-04.02.log.html http://eavesdrop.openstack.org/meetings/masakari/2017/masakari.2017-02-07-04.01.log.html

Example Spec - The title of your blueprint

Tue, 10 Apr 2018 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Pike	Introduced

Example Spec - The title of your blueprint

Tue, 10 Apr 2018 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Pike	Introduced

Example Spec - The title of your blueprint

Tue, 10 Apr 2018 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Rocky	Introduced

Example Spec - The title of your blueprint

Tue, 10 Apr 2018 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Rocky	Introduced

The title of your blueprint

Tue, 10 Apr 2018 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Introduction paragraph – why are we doing anything?

Problem description

A detailed description of the problem.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Include where in the masakari tree hierarchy this will reside.

Alternatives

This is an optional section, where it does apply we’d just like a demonstration that some thought has been put into why the proposed approach is the best one.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>

Can optionally can list additional ids if they intend on doing substantial implementation work on this blueprint.

Milestones

Target Milestone for completion:: Juno-1

Work Items

Dependencies

Include specific references to specs and/or blueprints in masakari, or in other projects, that this one either depends on or is related to.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Example Spec - The title of your blueprint

Wed, 22 Nov 2017 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Queens	Introduced

Example Spec - The title of your blueprint

Wed, 22 Nov 2017 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Queens	Introduced

Contributing to: masakari-specs

Tue, 20 Jun 2017 00:00:00

If you would like to contribute to the development of OpenStack, you must follow the steps in this page:

http://docs.openstack.org/infra/manual/developers.html

If you already have a good understanding of how the system works and your OpenStack accounts are set up, you can skip to the development workflow section of this documentation to learn how changes to OpenStack should be submitted for review via the Gerrit tool:

http://docs.openstack.org/infra/manual/developers.html#development-workflow

Pull requests submitted through GitHub will be ignored.

Bugs should be filed on Launchpad, not GitHub:

https://bugs.launchpad.net/masakari

Introspective Instance Monitoring

Tue, 30 May 2017 00:00:00

This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode

https://blueprints.launchpad.net/masakari/+spec/introspective-instance-monitoring

Currently, Masakari instance monitoring is strictly non-intrusive black-box type monitoring through qemu and libvirt. There are however a number of internal instance/VM faults (kernel scheduling and IO, application health), that if detected by Masakari, could be recovered by existing Masakari auto-recovery mechanisms; increasing the overall availability of the instance/VM. This blueprint introduces the capability of performing introspective instance monitoring of VMs, in order to detect, report and optionally recover VMs from internal VM faults. Specifically, VM Heartbeat Monitoring via the QEMU Guest Agent is introduced by this spec, in order to indirectly detect some of these internal VM faults.

Problem description

Currently, Masakari instance monitoring is a strictly non-intrusive black-box type monitoring through qemu and libvirt. This detects a number of faults for which Masakari’s auto-recovery mechanisms can be used to recover the instance/VM.

However, there are a number of internal instance/VM faults not detected by this black-box monitoring, that if detected by Masakari, could be recovered by these same Masakari auto-recovery mechanisms. This includes faults such as hung Guest OS, failure of the Guest OS to schedule Application process(es), failure to route basic IO within the Guest, Application-specific process failures or data corruption, etc. . The exact scope of the proposed monitoring of this blueprint is described at the end of the ‘Proposed change’ section.

Monitoring of Internal instance/VM faults requires that the Guest VM supports software to respond to this monitoring. In the following proposal, the Guest VM must support the QEMU Guest Agent. Because not all VMs will support this software, the monitoring of internal instance/VM faults, by the OpenStack Host, must be optionally enabled per VM or per VM image.

Proposed change

This blueprint introduces introspective instance monitoring; specifically, VM Heartbeat Monitoring via the QEMU Guest Agent. Any VM Heartbeat fault will be reported through the Masakari instance-alerter to registered API drivers (e.g. masakari-api).

The high-level architecture for Introspective Instance Monitoring is shown below:

+--------------------+   instance  +-------------+    + - - - - - - +
| instance-alerter   |<------------|  Masakari   |    |             |
|- - - - - - - - - - |   fault     |     VM      |      F U T U R E
| driver abstraction |             |  Heartbeat  |    |             |
|       layer        |             |    Agent    |
+--------------------+             +-------------+    + - - - - - - +
           |    |                         ^                  ^
  other <--+    |                         |                  |
  apis          |                         | +----------------+
                v                         | |
+--------------------+                    | |
|    masakari-api    |                    v v
+--------------------+             +-------------+
         |                         |  Libvirtd   |
         v                         +-------------+
+--------------------+                    ^
|   masakari-engine  |                    | unix socket
+--------------------+                    v
         |                         +-------------+
         | (recovery)              |    QEMU     |
         v                         +-------------+
+--------------------+                    ^
|                    |                    |
|      OpenStack     |       +--------------------------------------+
|                    |       | VM         | virtio serial device    |
+--------------------+       |            v                         |
                             |       +--------------------+         |
                             |       |   QEMU             |         |
                             |       |   Guest Agent      |         |
                             |       |   ( guest-ping{} ) |         |
                             |       +--------------------+         |
                             |                                      |
                             |         +-------------+              |
                             |       +-------------+ |              |
                             |       |             | |              |
                             |       | Application | |              |
                             |       |             | +              |
                             |       +-------------+                |
                             +--------------------------------------+

VM Heartbeat and Healthcheck Monitoring will leverage the QEMU feature, Guest Agent [1], for both the transport level communication between OpenStack Host and the Guest VM, and the built-in guest ping command (guest-ping{}). A QEMU Guest Agent daemon, built as part of QEMU, is installed and run inside the Guest and implements support for QMP commands that are sent to the guest. Specifically the QEMU Guest Agent daemon connects to a virtio-serial device (/dev/virtio-ports/org.qemu.guest_agent.0), feeds the input to a QMP JSON parser, and when a command is received, invokes the QAPI generated dispatch routine. In the case of VM Heartbeat Monitoring, the QEMU Guest Agent command, ‘guest-ping’, will be used as the heartbeat challenge request from the Host.

On the host, OpenStack Nova already supports an image property, hw_qemu_guest_agent, that can be used to specify that the VM should be created with the QEMU guest agent virto-serial-interface. The Masakari VM Heartbeat Agent will discover VMs with hw_qemu_guest_agent enabled by monitoring the files representing the socket identifiers for the QEMU Guest Agents’ virtual-serial-interfaces.

libvirt-qemu provides a virDomainQemuAgentCommand() for sending commands to a selected VM’s QEMU guest agent. This command opens the unix socket to the VM’s virtio-serial-interface, sends the command, waits to receive the response and closes the socket. The command fails if the unix socket is openned by another process, i.e. another process is sending a command to the same VM.

Masakari VM Heartbeat Agent will leverage virDomainQemuAgentCommand() provided by libvirtd to send the heartbeat challenge requests (i.e. the QEMU Guest Agent’s guest-ping command) to the VM(s) and report any detected faults to the masakari instance-alerter.

The Masakari VM Heartbeat Agent, on the host, will initiate VM Heartbeating as soon as it discovers the VM has QEMU Guest Agent communication enabled. However, in order to deal with arbitrary boot times for VMs/Guests, which may delay the Guests ability to start responding to the heartbeat challenges, the Masakari VM Heartbeat Agent will not enable reporting of heartbeat failures until after the first successful heartbeat response is received from the VM/Guest.

This functionality will support a flag in masakari.conf for overall enabling/disabling of introspective-instance-monitoring. It will also support parameters for configuring default heartbeat period and default consecutive heartbeat miss threshold (before declaring fault); in future, flavor extraspecs could be used for VMs to specify specific values for these.

At a high-level, the scope of this heartbeat monitoring is that the QEMU Guest Agent is running within the VM. However, just the fact that a Heartbeat message can get from the Host to the QEMU Guest Agent inside the VM and back, inherently validates that a lot of basic Guest Kernel functionality is working; i.e. the Guest OS is not hung or failed, the QEMU heartbeat message was properly routed through basic linux socket IO, etc. . In the future, the heartbeating can be extended to do more than just reply/ack the message … i.e. basic sanity / health tests on key applications within the VM can be done.

Alternatives

Could simply leverage the virtual hardware watchdog of QEMU/KVM [2] for Instance monitoring.

However, VM Heartbeat Monitoring:

provides notification of the Heartbeat status to higher-level cloud entities through instance-alerter, such as Masakari, Mistral and/or Vitrage,
- which depending on the backend can result in VM auto-recovery (Masakari) or deduced-state updates in Nova for the VM and resulting Aodh Event generation due to the VM state change (Vitrage).
in the future can be extended to provide a higher-level (i.e. application-level) heartbeating
- i.e. if the Heartbeat requests are being answered by the Application running within the VM
in the future can be extended to provide more than just heartbeating, as the Application can use it to trigger a variety of audits,
in the future can be extended to provide a mechanism for the Application within the VM to report a Health Status / Info back to the Host / Cloud.

Limitation

Only VMs supporting the QEMU Guest Agent can be monitored by the functionality of this proposal.

Implementation

Assignee(s)

Primary assignee:: greg-waines

Milestones

Target Milestone for completion:: Rocky-2

Work Items

Masakari VM Heartbeat Agent on the Compute
- discovery of VMs with QEMU Guest Agent communication enabled,
- high-level logic for Heartbeat / Healthcheck monitoring,
- reporting of faults to masakari instance-alerter.
tox and/or tempest test suite updates
masakari documentation updates

Dependencies

requires that VMs are installed with and running the QEMU Guest Agent [1] built as part of QEMU.

References

[1] http://wiki.qemu.org/Features/GuestAgent

[2] https://libvirt.org/formatdomain.html#elementsWatchdog

Masakari Spec Lite

Fri, 20 Jan 2017 00:00:00

Please keep this template section in place and add your own copy of it between the markers. Please fill only one Spec Lite per commit.

<Title of your Spec Lite>

link:: <Link to the blueprint.>
problem:: <What is the driver to make the change.>
solution:: <High level description what needs to get done. For example: “We need to add client function X.Y.Z to interact with new server functionality Z”.>
impacts:: <All possible *Impact flags (same as in commit messages) or ‘None’.>

Optionals (please remove this line and fill or remove the rest until End):

how:: <More technical details than the high level overview of solution if needed.>
alternatives:: <Any alternative approaches that might be worth of bringing to discussion.>
timeline:: <Estimation of the time needed to complete the work.>
reviewers:: <If reviewers has been agreed for the functionality, list them here.>
assignee:: <If known, list who is going to work on the feature implementation here>

End of Template

Add periodic task to clean up workflow failure

link:

https://blueprints.launchpad.net/masakari/+spec/add-periodic-tasks

problem:

Due to some unknown circumstances, there is a possibility few of the notifications might get into ‘error’ status or it might remain in ‘new’ status forever. There should be some way to retrieve such notifications and process them to completion.

solution:

Add a periodic task “process_unfinished_notifications’ which will execute at regular interval as defined by the new config option “process_unfinished_notifications_interval” in seconds. Default value for this option will be 120 seconds, however operator can set this interval value as per the requirement. Inside this periodic task, it will retrieve all notifications which are in ‘error’ or ‘new’ status and then execute recovery action workflow to process all of them. The notifications which are in ‘new’ status will be picked up based on a new config option ‘retry_notification_new_status_interval’. Default value for this option will be 60 seconds, however operator can set this interval value as per the requirement. Each notification has ‘generated_time’ field, if this time + retry_notification_new_status_interval value (in seconds) is greater than or equal to the current system time, then all such notifications in ‘new’ status will be picked up by this periodic task. Also, the notifications in ‘error’ status will be picked up too.

Let’s understand the transition state of notification for different statuses for success case: notification current status error -> running -> finished notification current status new-> running -> finished If the workflow execution fails, then the transition state of notification would be: notification current status error -> running -> failed notification current status new-> running -> failed

Note: One important point to take note of is if the original notification status is ‘new’ then it won’t be retried again if the workflow fails to process it in the periodic task. It’s status will be directly set to ‘failed’. The operator needs to take corrective action for all notifications which are in ‘failed‘ state manually.

alternatives:

Add two periodic tasks ‘process_error_notifications’ and ‘process_queued_notifications’ to process notifications which are in ‘error’ and ‘new’ status respectively. These periodic tasks will be called at regular intervals as defined by two new config options “process_error_notification_interval’ and ‘process_queued_notifications_interval’. The logic for retrieval of notifications which are in ‘error’ and ‘new’ statuses will be exactly same as above solution. The only difference would be in the notification status upon its completion as explained below.

Transition state of notification for different statuses for success case. notification current status error (process_error_notifications) -> running -> finished notification current status new (process_queued_notifications) -> running -> finished

If the workflow execution fails, then the transition state of notification would be: notification current status error (process_error_notifications) -> running -> failed notification current status new (process_queued_notifications) -> running -> error This means that the notification will be again eligible for reprocessing during the next cycle of ‘process_error_notifications’ periodic task.

impacts:

None

timeline:

Expected to be merged within the Ocata time frame.

reviewers:

sam47priya@gmail.com, kajinamit@nttdata.co.jp, tushar.vitthal.patil@gmail.com

assignee:

Abhishek Kekane

End of Add periodic task to clean up workflow failure

OpenStack Masakari Specifications

Tue, 10 Jan 2017 00:00:00

This git repository is used to hold approved design specifications for additions to the Masakari project. Reviews of the specs are done in gerrit, using a similar workflow to how we review and merge changes to the code itself.

The layout of this repository is:

specs/<release>/

Where there are two sub-directories:

specs/<release>/approved: specifications approved but not yet implemented specs/<release>/implemented: implemented specifications

The lifecycle of a specification

Developers proposing a specification should propose a new file in the approved directory. masakari-core will review the change in the usual manner for the OpenStack project, and eventually it will get merged if a consensus is reached. At this time the Launchpad blueprint is also approved. The developer is then free to propose code reviews to implement their specification. These reviews should be sure to reference the Launchpad blueprint in their commit message for tracking purposes.

Once all code for the feature is merged into Masakari, the Launchpad blueprint is marked complete. As the developer of an approved specification it is your responsibility to mark your blueprint complete when all of the required patches have merged.

Periodically, someone from masakari-core will move implemented specifications from the approved directory to the implemented directory. Individual developers are also welcome to propose this move for their implemented specifications. It is important to create redirects when this is done so that existing links to the approved specification are not broken. Redirects aren’t symbolic links, they are defined in a file which sphinx consumes. An example is at specs/ocata/redirects.

This directory structure allows you to see what we thought about doing, decided to do, and actually got done. Users interested in functionality in a given release should only refer to the implemented directory.

Example specifications

You can find an example spec in specs/ocata-template.rst.

Working with gerrit and specification unit tests

For more information about working with gerrit, see http://docs.openstack.org/infra/manual/developers.html#development-workflow

To validate that the specification is syntactically correct (i.e. get more confidence in the Jenkins result), please execute the following command:

$ tox

After running tox, the documentation will be available for viewing in HTML format in the doc/build/ directory.

Free software: Apache license
Documentation: http://docs.openstack.org/developer/masakari-specs

Example Spec - The title of your blueprint

Tue, 10 Jan 2017 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Ocata	Introduced

Example Spec - The title of your blueprint

Tue, 10 Jan 2017 00:00:00

Include the URL of your launchpad blueprint:

https://blueprints.launchpad.net/masakari/+spec/example

Some notes about the masakari-spec and blueprint process:

Not all blueprints need a spec. For more information see https://wiki.openstack.org/wiki/Masakari-specs
The aim of this document is first to define the problem we need to solve, and second agree the overall approach to solve that problem.
This is not intended to be extensive documentation for a new feature. For example, there is no need to specify the exact configuration changes, nor the exact details of any DB model changes. But you should still define that such changes are required, and be clear on how that will affect upgrades.
You should aim to get your spec approved before writing your code. While you are free to write prototypes and code before getting your spec approved, its possible that the outcome of the spec review process leads you towards a fundamentally different solution than you first envisaged.
But, API changes are held to a much higher level of scrutiny. As soon as an API change merges, we must assume it could be in production somewhere, and as such, we then need to support that API change forever. To avoid getting that wrong, we do want lots of details about API changes upfront.

Some notes about using this template:

Your spec should be in ReSTructured text, like this template.
Please wrap text at 79 columns.
The filename in the git repository should match the launchpad URL, for example a URL of: https://blueprints.launchpad.net/masakari/+spec/awesome-thing should be named awesome-thing.rst
Please do not delete any of the sections in this template. If you have nothing to say for a whole section, just write: None
For help with syntax, see http://sphinx-doc.org/rest.html
To test out your formatting, build the docs using tox and see the generated HTML file in doc/build/html/specs/<path_of_your_file>
If you would like to provide a diagram with your spec, ascii diagrams are required. http://asciiflow.com/ is a very nice tool to assist with making ascii diagrams. The reason for this is that the tool used to review specs is based purely on plain text. Plain text will allow review to proceed without having to look at additional files which can not be viewed in gerrit. It will also allow inline feedback on the diagram itself.
If your specification proposes any changes to the Masakari REST API such as changing parameters which can be returned or accepted, or even the semantics of what happens when a client calls into the API, then you should add the APIImpact flag to the commit message.

Problem description

A detailed description of the problem. What problem is this blueprint addressing?

Use Cases

What use cases does this address? What impact on actors does this change have? Ensure you are clear about the actors in each use case: Developer, End User, Deployer etc.

Proposed change

Here is where you cover the change you propose to make in detail. How do you propose to solve this problem?

If this is one part of a larger effort make it clear where this piece ends. In other words, what’s the scope of this effort?

Alternatives

Data model impact

Questions which need to be addressed by this section include:

What new data objects and/or database schema changes is this going to require?
What database migrations will accompany this change.
How will the initial set of new data objects be generated, for example if you need to take into account existing instances, or modify other existing data describe how that will work.

REST API impact

Each API method which is either added or changed should have the following

Specification for the method
- A description of what the method does suitable for use in user documentation
- Method type (POST/PUT/GET/DELETE)
- Normal http response code(s)
- Expected error http response code(s)
  - A description for each possible error code should be included describing semantic errors which can cause it such as inconsistent parameters supplied to the method, or when an instance is not in an appropriate state for the request to succeed. Errors caused by syntactic problems covered by the JSON schema definition do not need to be included.
- URL for the resource
  - URL should not include underscores, and use hyphens instead.
- Parameters which can be passed via the url
- JSON schema definition for the request body data if allowed
  - Field names should use snake_case style, not CamelCase or MixedCase style.
- JSON schema definition for the response body data if any
  - Field names should use snake_case style, not CamelCase or MixedCase style.
Example use case including typical API samples for both data supplied by the caller and the response
Discuss any policy changes, and discuss what things a deployer needs to think about when defining their policy.

Reuse of existing predefined parameter types such as regexps for passwords and user defined names is highly encouraged.

Security impact

Describe any potential security impact on the system. Some of the items to consider include:

Does this change touch sensitive data such as tokens, keys, or user data?
Does this change alter the API in a way that may impact security, such as a new way to access sensitive information or a new way to login?
Does this change involve cryptography or hashing?
Does this change require the use of sudo or any elevated privileges?
Does this change involve using or parsing user-provided data? This could be directly at the API level or indirectly such as changes to a cache layer.
Can this change enable a resource exhaustion attack, such as allowing a single API interaction to consume significant server resources? Some examples of this include launching subprocesses for each connection, or entity expansion attacks in XML.

Notifications impact

Please specify any changes to notifications. Be that an extra notification, changes to an existing notification, or removing a notification.

Other end user impact

Aside from the API, are there other ways a user will interact with this feature?

Does this change have an impact on python-masakariclient? What does the user interface there look like?

Performance Impact

Describe any potential performance impact on the system, for example how often will new code be called, and is there a major change to the calling pattern of existing code.

Examples of things to consider here include:

A periodic task might look like a small addition but if it calls conductor or another service the load is multiplied by the number of nodes in the system.
Scheduler filters get called once per host for every instance being created, so any latency they introduce is linear with the size of the system.
A small change in a utility function or a commonly used decorator can have a large impacts on performance.
Calls which result in a database queries (whether direct or via conductor) can have a profound impact on performance when called in critical sections of the code.
Will the change include any locking, and if so what considerations are there on holding the lock?

Other deployer impact

Discuss things that will affect how you deploy and configure OpenStack that have not already been mentioned, such as:

What config options are being added? Should they be more generic than proposed (for example a flag that other hypervisor drivers might want to implement as well)? Are the default values ones which will work well in real deployments?
Is this a change that takes immediate effect after its merged, or is it something that has to be explicitly enabled?
If this change is a new binary, how would it be deployed?
Please state anything that those doing continuous deployment, or those upgrading from the previous release, need to be aware of. Also describe any plans to deprecate configuration values or features. For example, if we change the directory name that instances are stored in, how do we handle instance directories created before the change landed? Do we move them? Do we have a special case in the code? Do we assume that the operator will recreate all the instances in their cloud?

Developer impact

Discuss things that will affect other developers working on OpenStack, such as:

If the blueprint proposes a change to the driver API, discussion of how other hypervisors would implement the feature is required.

Implementation

Assignee(s)

Who is leading the writing of the code? Or is this a blueprint where you’re throwing it out there to see who picks it up?

If more than one person is working on the implementation, please designate the primary author and contact.

Primary assignee:: <launchpad-id or None>
Other contributors:: <launchpad-id or None>

Work Items

Dependencies

Include specific references to specs and/or blueprints in Masakari, or in other projects, that this one either depends on or is related to.
If this requires functionality of another project that is not currently used by Masakari (such as nova, or masakari-monitors, python-masakariclient), document that fact.
Does this feature require any new library dependencies or code otherwise not included in OpenStack? Or does it depend on a specific version of library?

Testing

Is this untestable in gate given current limitations (specific hardware / software configurations available)? If so, are there mitigation plans (3rd party testing, gate enhancements, etc).

Documentation Impact

References

Links to mailing list or IRC discussions
Links to notes from a summit session
Links to relevant research, if appropriate
Related specifications as appropriate (e.g. if it’s an EC2 thing, link the EC2 docs)
Anything else you feel it is worthwhile to refer to

History

Optional section intended to be used each time the spec is updated to describe new design, API or any database schema updated. Useful to let reader understand what’s happened along the time.

Revisions
Release Name	Description
Ocata	Introduced