Resource Meta-data Caching¶
This was discussed at the YVR summit as follow up to issues raised during the Operators feedback session.
Ceilometer polls regularly via the compute-agent to get resource meta-data’ for all instances. This polling adds load to Nova, Neutron, and Keystone APIs, which at scale causes a significant performance impact. This meta-data does not change very often, and the load is likewise often unnecessary.
Lets take advantage of the changes-since parameter in the nova api call. We’ll start by creating an in-memory cache to store the resource meta-data on the initial polling and the time of that polling. Then the compute agent will use the changes-since parameter in all subsequent pollings. This will reduce the amount of data queried and returned by the Nova API. The results of this call will update the cache, and the cache will be used to return the data. Also, we should populate the instance flavor and image data in the cache as well.
Data model impact¶
REST API impact¶
Other end user impact¶
Should provide greater overall scalability due to the reduction in API load and reduced traffic flows.
Other deployer impact¶
- Primary assignee:
- Other contributors:
- Ongoing maintainer:
Implement an in-memory cache in the compute agent.
Change compute-agent to populate an in-memory cache.
Change compute-agent to use the changes-since parameter in the Nova API call .
Change compute-agent to record the polling timestamp in the cache.
Change compute-agent to cache the flavor and image metadata as well.
We will need to add tests for meta-data cache flag, API changed-since queries, compute-agent cache control, and image/flavor caching.
We will need to document the changes to the polling process, the new meta-data cache config option, and create new documentation that describes the caching work-flow.