Scrub Images in Parallel¶
This change proposal introduces a way for the scrubber to scrub images in parallel when delayed delete is enabled.
As of today, when delayed delete is enabled, images are being scrubbed serially while the image locations, if multiple, are being scrubbed in parallel. For the general case, this may not achieve much performance gain as the number of images is likely to be more than the number of image locations per image. Consequently, the scrubber can fall behind when the number of pending_delete images increase.
This change will attempt to parallelize image scrubbing while leaving image locations to be scrubbed serially.
A new config option would be introduced to offer the flexibility to choose between serial or parallel scrubbing. Also, using this config option, one can regulate the degree of parallelism to a desired level.
One can run multiple scrubbers to keep up with the increase in pending_delete images. However, in that case, scrubbers may race as they all get the same set of images from the registry. This may not be a bad option if one can ignore all the errors in the logs. Nevertheless, scrubbing images serially is still inefficient.
Data model impact¶
REST API impact¶
As mentioned in the performance impact section, while setting the degree of parallelism one should take into account any rate-limits enforced by the backend store. If the degree of parallelism is set to go beyond the rate-limits, an attacker may be able to force the scrubber hit the rate-limits by creating and deleting several images in a short span of time.
Other end user impact¶
When using parallel scrubbing one must take into account any rate-limits on the backend store. Depending on extent of parallelism desired, scrubber may hit the rate-limits of the backend store and may eventually slowdown or fail.
Other deployer impact¶
Deployers would have to set the new configuration option introduced with this change proposal to be able to scrub images in parallel.
- Primary assignee:
- Other contributors:
- Core reviewer(s):
- Other reviewer(s):
Use eventlet to parallelize image scrubbing
Monkey-patch required modules for eventlet
Test on devstack
The new configuration option would require documentation.