https://blueprints.launchpad.net/cinder/+spec/extend-limit-implementations
Currently, the pagination keys, like limit, marker, sort key and sort direction and the next link generation, have been implemented for the volume list operations, but not available in other list operations. This blueprint implements the pagination keys and the next link generation for other items, like snapshots, volume transfers, consistency group, consistency group snapshots and backups.
The pagination keys mentioned in the above section have been implemented in the list operations for volumes. This feature can be extended to other cinder concepts. The reason why snapshots, volume transfers, consistency groups, consistency group snapshots, and backups are being targeted is because these entities have relatively high frequencies to query and it is more probable for them to reach a large amount comparing to other entities, like type, availability zone, etc.
The next link generation is already available in volume list, so it is straightforward to implement it for other cinder concepts. Please refer to the _get_collection_links function in the cinder.api.common.ViewBuilder class.
Suppose the maximum page size 1000 and there are more than 1000 volume snapshots. If there is no pagination key marker implemented for snapshot, the maximum snapshots the user can query from the API is 1000. The only way to query more than 1000 snapshots is to increase the default maximum page size, which is a limitation to the cinder list operations. If no next link is generated, it is not possible to retrieve more items than the maximum page size. Even if the next link is generated in this case, it will still return the same results as the previous request without the key marker honored. In conclusion, both the pagination keys and the next link generation are necessary to add for the cinder entities.
The change depends on some work items, which have been put in the “Dependencies” section.
The list APIs for snapshots, volume transfers, consistency group, consistency group snapshots and backups will support the following parameters:
Multiple sort keys and sort directions will be implemented as they are done for volumes in the patch “GET volumes API sorting REST/volume/DB updates”, https://review.openstack.org/#/c/141915/.
The change is compliant with the syntax for client sorting arguments defined in the patch “CLI Sorting Argument Guidelines”, https://review.openstack.org/#/c/145544/.
The caller can specify these parameters in the request to query the list of the items, e.g. /snapshots?limit=5&marker=snapshot_id&sort=key1:asc,key2:desc,key3:asc
If there are more items available to query in further requests, the next link should be correctly generated for snapshots, volume transfers, consistency group, consistency group snapshots and backups.
It is possible that we define different pagination keys, other than limit, marker, sort information to implement the pagination for other items in Cinder.
However, to keep the consistency in code change, it is better to follow the pattern that volume list does.
Another alternative is cinder sticks with no pagination for other entities except volumes. However, the default maximum page size 1000 can be exceeded, by some entities, like snapshots, in the production environment. It will cause issues in other entities in future.
None
The following existing v2 GET APIs will support the new sorting parameters:
{items} will be replaced by the appropriate entities as follows:
The existing API needs to support the following new Request Parameters for the above cinder concepts:
Parameter | Style | Type | Description |
---|---|---|---|
limit | query | integer | Key used to determine the number of returned items in the response |
marker | query | string | Key used to determine the item, where to start the query |
sort | query | list | A comma-separated list of sort keys, with the sort directions optionally appended to each sort key |
In the event that an invalid pagination key is specified then a “badRequest” error response (code 400) will be returned.
The next link will be put in the response returned from cinder if it is necessary.
For snapshots, it replies:
{
"snapshots": [<List of snapshots>],
"snapshots_links": [{'href': '<next_link>', 'rel': 'next'}]
}
For volume transfers, it replies:
{
"transfers": [<List of transfers>],
"transfers_links": [{'href': '<next_link>', 'rel': 'next'}]
}
For consistency group, it replies:
{
"consistencygroups": [<List of consistencygroups>],
"consistencygroups_links": [{'href': '<next_link>', 'rel': 'next'}]
}
For consistency group snapshots, it replies:
{
"cgsnapshots": [<List of cgsnapshots>],
"cgsnapshots_links": [{'href': '<next_link>', 'rel': 'next'}]
}
For backups, it replies:
{
"backups": [<List of backups>],
"backups_links": [{'href': '<next_link>', 'rel': 'next'}]
}
None
None
The cinderclient should be updated to accept limit, marker, sort information, in snapshots, volume transfers, consistency group, consistency group snapshots and backups.
The user will be able to specify pagination keys, like limit, marker, sort information to list snapshots, volume transfers, consistency group, consistency group snapshots and backups.
None
The deployer should be aware that the flag osapi_max_limit can set the maximum number of items that a collection resource returns in ONE single response, but it will not limit the number of items returned for the cinderclient request any longer.
After the pagination keys and the next link generation are implemented for the cinder entities, the cinderclient request can retrieve more items than the flag osapi_max_limit sets, because it can fetch the items multiple times via the next link with the marker key until all the items are returned if no limit key is set or the number of items equals to limit if the limit key is set.
None
Since the logic code of next link generation is finished in a common class. We do not need to repeat the work any more.
Therefore, I see the following work items:
Both unit and Tempest tests need to be created to ensure that snapshots, volume transfers, consistency group, consistency group snapshots and backups support the pagination keys of limit, marker, and sort information, and the next link generation is available if necessary.
The API documentation will need to be updated to accept limit, marker, sort key and sort direction, in snapshots, volume transfers, consistency group, consistency group snapshots and backups as it does for volumes.
None
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.