Queens Project Priorities

List of priorities the Watcher drivers team is prioritizing in Queens.

Priority Owner
Baremetal data model in Watcher Li Canwei
Check the strategy requirements Alexander Chadin
Notifications for action plan cancel Aditi Sharma
Watcher Planner Selector Alexander Chadin
Watcher Strategy Selector Aditi Sharma
Workload Characterization and QoS Susanne Balle
Extend node status suzhengwei
Exclude project by audit scope Aditi Sharma
Audit scoper for storage data model Hidekazu Nakamura
Define a tailored Scoper for each CDM Hidekazu Nakamura
Register and Document Policy in Code Yumeng Bao
Workload characterization grammar Susanne Balle
Zone migration strategy Hidekazu Nakamura
Watcher API validation using JSON Aditi Sharma
Compute CDM include all instances suzhengwei
Display input parameters of strategy Alexander Chadin
Add name for audit suzhengwei
Cluster maintance strategy suzhengwei
Multiple datasources for strategies Aditi Sharma
Replace voplutuous with JSON Schema Yumeng Bao

Baremetal data model in Watcher

For a data center with large amount of VMs and physical hosts,the total power consumption is tremendous. When workload is not heavy, Watcher can be used to reduce power consumption by triggering a request to power off some idle hosts without VMs. And when the workload increases Watcher will trigger a “power on” request to fullfill the service requirements.

Check the strategy requirements

Running of strategy requires different type of data resources to compute a solution. During the strategies loading, the decision engine should automatically check if all requirements are achieved.

Notifications for action plan cancel

Notifications needs to be added to action and action plan for new operation action plan cancel.

Watcher Planner Selector

This component is responsible for selecting an appropriate planner for a given strategy depending on several factors (e.g list of actions used by the strategy or user request).

Watcher Strategy Selector

There may be several strategies applying for a given optimization goal. Currently , if the admin didn’t specify a strategy Watcher select the first strategy available in the list. If Watcher intends to be used in real infrastructure it need a more robust way to select the strategy for a given goal. The strategy selector component will enable watcher to automatically decide which strategy to use.

Workload Characterization and QoS

Based on the defined workload characteristics we should be able to apply Quality of Services to applications. An example would be leveraging technologies like Intel RDT.

This opens up several application optimization possibilities (use cases like NFV etc.) and also ensures efficient use of cloud resources. Scope of this blueprint is to build a QoS strategy using Intel RDT and workload grammar.

Extend node status

We can get a node status through CDM (cluster data model) in watcher. Most of strategies rely on the node’s status. But the existing status just meets existing strategies. We need to extend nodes status description for new strategies.

Exclude project by audit scope

As of Now, Watcher can exclude instances, compute_nodes, host_aggregates and instance_metadata by audit scope. As an administrator, I want to exclude instances of a specific project from Watcher optimization. This bp proposes to add exclude project feature to audit scope.

Audit scoper for storage data model

Since storage data model was added in Pike cycle, we need audit scoper for storage data model as compute data model has.

Define a tailored Scoper for each CDM

Storage cluster data model was introduced in Pike cycle. Since the model is different from compute data model, current single CDM scoper does not work for the model.

Register and Document Policy in Code

This blueprint tracks the work for moving default policies and documentation into code. This is OpenStack-wide goals for Queens release. https://governance.openstack.org/tc/goals/queens/policy-in-code.html

Storage workload balance

As of now, Watcher optimizes only compute nodes. Storage optimization is also important feature for centralized storage (non distributed storage). This spec will add Storage Workload Balance Strategy to balance the storage resource. And we can use existing goal (workload_balancing) and action(volume_migrate) for storage workload balance.

Workload characterization grammar

As we run several workloads in cloud, we should be able to characterize such workloads as input to watcher for ensuring Application QoS, placements and consolidation.

An example of workload characterization is a weighted combination of CPU, Memory or any other resource attributes like High IOPs, Network latency etc.

Scope of this blueprint is to come up with a grammar structure for defining workload character.

Zone migration strategy

There are thousands of physical servers and storage running various kinds of workloads in the Cloud system. Administrator have to migrate instances and block storage for hardware maintenance once a quarter or so. It requires operators manually to watch workloads, choose instances to migrate and migrate for each instances and block storage for efficiently, with minimum downtime. Watcher can be used to do this task automatically.

Watcher API validation using JSON

Cureently watcher uses different methods to validate api, which causes many bugs and few operations are possible which should not be allowed like a cloud admin can delete “ongoing” actionplan and audit. To have more cleaner and same approach for all operations we should have a unified way of validating the api, which can be done using JSON.

Compute CDM include all instances

When building compute CDM, Watcher excludes the instances excluded in the scope. It has negative impact to Watcher.

  • To some strategies, it get incorrectly workload of the compute nodes, because the excluded instances was not calculated in.
  • To server consolidation, it disables the nodes which has excluded instances running.

Display input parameters of strategy

As of now in watcher, it is difficult for users to know what input-parameters are required by each strategy when creating an audit. This blueprint will add a column to display input parameters of each strategy in results of “watcher strategy list”.

Add name for audit

Adding name for audit entity would be more friendly to end users to interact with audit. This blueprint suggests to add name property to audit entity, so it will be easy for users to retrieve an andit by name.

Cluster maintance strategy

Sometimes we need to maintain compute nodes, update hardware and software, and so on. But we don’t want user’s application to be interrupted. This issue imports one goal and strategy for manually maintaining without user’s application interruption.

Multiple datasources for strategies

We need an abstraction layer with a minimal subset of metrics needed by existing strategies (it will deal with gnocchi & monasca). If we have multiple metrics collection backend deployed, the strategy will need to give the back-end it wants to use / alternatively we could define a backend priority in configuration. We will need a map between metrics names used by strategies and correspond names in backends.

Multiple global efficacy indicator

As of now global efficacy indicator is a single entity. It is useful for the strategies which optimizes only one resource like server consolidation works only with instances. There can be some future strategies which optimizes many resources (volume, instance, network) for them it is necessary to calculate global efficacy for each resource. This blueprint will implement multiple global efficacy indicator.

Replace voplutuous with JSON Schema

In this blueprint, we will replace voplutuous with JSON-schema to validate efficacy indicator. Since we want to remove voluptuous and use JSONSchema as our only JSON validation tool to keep consistency in Watcher.