Enhance List Operations with Pagination Keys¶

Alternatives¶

It is possible that we define different pagination keys, other than limit, marker, sort information to implement the pagination for other items in Cinder.

However, to keep the consistency in code change, it is better to follow the pattern that volume list does.

Another alternative is cinder sticks with no pagination for other entities except volumes. However, the default maximum page size 1000 can be exceeded, by some entities, like snapshots, in the production environment. It will cause issues in other entities in future.

Data model impact¶

None

REST API impact¶

The following existing v2 GET APIs will support the new sorting parameters:

• /v2/{tenant_id}/{items}
• /v2/{tenant_id}/{items}/detail

{items} will be replaced by the appropriate entities as follows:

• For snapshots:
  • /v2/{tenant_id}/snapshots
  • /v2/{tenant_id}/snapshots/detail
• For volume transfers:
  • /v2/{tenant_id}/os-volume-transfer
  • /v2/{tenant_id}/os-volume-transfer/detail
• For consistency group:
  • /v2/{tenant_id}/consistencygroups
  • /v2/{tenant_id}/consistencygroups/detail
• For consistency group snapshots:
  • /v2/{tenant_id}/cgsnapshots
  • /v2/{tenant_id}/cgsnapshots/detail
• For backups:
  • /v2/{tenant_id}/backups
  • /v2/{tenant_id}/backups/detail

The existing API needs to support the following new Request Parameters for the above cinder concepts:

In the event that an invalid pagination key is specified then a “badRequest” error response (code 400) will be returned.

• For the limit key, it returns a message like “Invalid input received: Invalid limit key”.
• For the marker key, it returns a message like “Invalid input received: Invalid marker key”.
• For the sort information, it returns a message like “Invalid input received: Invalid sort key” or “Invalid input received: Invalid sort direction” depending on different circumstances.

The next link will be put in the response returned from cinder if it is necessary.

• For snapshots, it replies:

  {
      "snapshots": [<List of snapshots>],
      "snapshots_links": [{'href': '<next_link>', 'rel': 'next'}]
  }
  

• For volume transfers, it replies:

  {
      "transfers": [<List of transfers>],
      "transfers_links": [{'href': '<next_link>', 'rel': 'next'}]
  }
  

• For consistency group, it replies:

  {
      "consistencygroups": [<List of consistencygroups>],
      "consistencygroups_links": [{'href': '<next_link>', 'rel': 'next'}]
  }
  

• For consistency group snapshots, it replies:

  {
      "cgsnapshots": [<List of cgsnapshots>],
      "cgsnapshots_links": [{'href': '<next_link>', 'rel': 'next'}]
  }
  

• For backups, it replies:

  {
    "backups": [<List of backups>],
    "backups_links": [{'href': '<next_link>', 'rel': 'next'}]
  }
  

Security impact¶

None

Notifications impact¶

None

Other end user impact¶

The cinderclient should be updated to accept limit, marker, sort information, in snapshots, volume transfers, consistency group, consistency group snapshots and backups.

The user will be able to specify pagination keys, like limit, marker, sort information to list snapshots, volume transfers, consistency group, consistency group snapshots and backups.

Performance Impact¶

None

Other deployer impact¶

The deployer should be aware that the flag osapi_max_limit can set the maximum number of items that a collection resource returns in ONE single response, but it will not limit the number of items returned for the cinderclient request any longer.

After the pagination keys and the next link generation are implemented for the cinder entities, the cinderclient request can retrieve more items than the flag osapi_max_limit sets, because it can fetch the items multiple times via the next link with the marker key until all the items are returned if no limit key is set or the number of items equals to limit if the limit key is set.

Developer impact¶

None

Assignee(s)¶

Primary assignee:

Vincent Hou (sbhou@cn.ibm.com)

Other contributors:

None

Work Items¶

Since the logic code of next link generation is finished in a common class. We do not need to repeat the work any more.

Therefore, I see the following work items:

• Add the code to call the common functions to get the limit, marker and sort parameters processed for snapshots, volume transfers, consistency group, consistency group snapshots and backups.
• Add the code to do the database query with the paginations keys for snapshots, volume transfers, consistency group, consistency group snapshots and backups.
• Add the support of the paginations keys for snapshots, volume transfers, consistency group, consistency group snapshots and backups in cinderclient.
• Modify the existing APIs to support passing the limit, marker, and sort information from the API layer to the database layer.
• Unit tests to ensure that these pagination keys and the next link generation is supported for snapshots, volume transfers, consistency group, consistency group snapshots and backups.