https://blueprints.launchpad.net/fuel/+spec/cgroups
Support daemon resource control by means of cgroups kernel feature.
General OS doesn’t activate any protection by default against taking all hardware’s memory or CPU. So there is a necessity to allocate resources between competing processes, e.g. at the peak time CPU computing resources should be distributed by the specified rules.
By using cgroups, cloud operator gains fine-grained control over allocating, prioritizing, denying, managing, and monitoring system resources. Hardware resources can be appropriately divided up among tasks and services, increasing overall efficiency. Service set what is supposed to be moved under cgroups control:
all OpenStack services
- middleware
- rabbitmq
- mysql/galera
- mongodb
- ceph services
User will be able to move all services described above under cgroups resources control(we specified only openstack related set of services in provided list, but, user is able to move any service what he want under the cgroup control). User should prepare special configuration JSON string for each service what supposed to be moved under the cgroups control(cgroups utils will be installed even if no cgroup’s settings are specified).
User will be able to specify limits via API/CLI (one more field will be brought into yaml with cluster’s settings structure by user). New puppet task to apply cgroups configuration on target nodes will be added as well.
As there is one source of limits for all cluster nodes we should support relative values for specific resources(like RAM). Format of proposed relative expressions is (user should put it into respective resource’s field):
%percentage_value, minimal_value, maximum_value
It means that:
- percentage value(% of total memory) will be calculated and then clamped to keep value within the range( percentage value will be used if total node’s RAM lower than minimal range value)
- minimal value will be taken if node’s RAM lower than minimal value
- maximum value will be taken if node’s RAM upper than maximum value
Example: %20, 2G, 20G
For other limits (e.g. disk I/O, share of CPU time) the native relative values will be accepted.
User will be able to extend list of limits using cluster settings yaml file(download/upload via CLI).
None
None
New hidden section cgroups should be added into openstack.yaml file under ‘general’ group to make cgroups settings configurable after the cluster is deployed. User will be able to download/upload cluster’s settings file to override default cgroups settings(add new services and settings).
Example of a new structure what’s supposed to be added into openstack.yaml file by (the nesting level - [‘editable’][‘additional_components’]):
cgroups:
metadata:
group: general
label: Cgroups configuration
always_editable: true
weight: 90
restrictions:
- condition: "true"
action: "hide"
...
Example of services what should be added under cgroups control:
mysqld:
label: mysqld
type: text
value: '{"memory":{"memory.swappiness":0, "memory.soft_limit_in_bytes":"%5, 10, 20"}}}'
rabbitmq:
label: rabbitmq
type: text
value: '{"memory":{"memory.swappiness":0}}'
cinder:
label: cinder
type: text
value: '{"blkio":{"blkio.weight":500}}'
keystone:
label: keystone
type: text
value: '{"cpu":{"cpu.shares":70}}'
neutron:
label: neutron
type: text
value: '{"memory":{"memory.soft_limit_in_bytes":"%total, min, max"}}'
...
Cgroups limits per service will be described in json format into ‘text’ fields. Format will be explicitly described in feature’s documentation.
None
None
None
Cloud operator should add services that are supposed to be moved under cgroups control into cluster’s settings file via CLI(into cgroups section), data from corresponding section will be included into node’s astute yaml file automatically during the serialization process.
A new cgroups puppet module should be implemented which will be used by main task to configure given limits for services on the cluster nodes. Module should be able to get input data from hiera structure then validate and apply it.
Task will be run on post deploment stage:
id: cgroups
type: puppet
version: 2.0.0
role: '*'
requires: [post_deployment_start]
required_for: [post_deployment_end]
parameters:
puppet_manifest: .../osnailyfacter/modular/cgroups/cgroups.pp
puppet_modules: /etc/puppet/modules
timeout: 3600
cwd: /
Limit CPU utilization by nice(1), for limiting memory allocation rely upon service configuration/runtime constrains itself.
From life cycle management perspective, cloud operator will be able to change cgroups settings for the deployed cluster in following way:
change service’s limits in cluster’s settings via CLI/API
- run ‘hiera’ and ‘cgroups’ tasks on the cluster via CLI
(fuel node –node-id ID1, ID2 –tasks hiera,cgroups)
None
None
With emploing cgroups kernel feature hardware resources can be appropriately divided up among tasks and services, increasing overall efficiency.
None
None
None
This feature should be described in the documentation.
None
In order to verify the quality of new feature, automatic system tests will be expanded by the cases listed below:
The tests that are described above should pass.