Support Cinder Volume Multi-attach¶
Currently, Nova only allows a volume to be attached to a single host or instance. There are times when a user may want to be able to attach the same volume to multiple instances.
Currently, Nova only allows a volume to be attached to one instance and or host at a time. Nova makes an assumption in a number of places that assumes the limitation of a single volume to a single instance.
cinderclient only has volume as a parameter to the detach() call. This makes the assumption that a volume is only attached once.
nova assumes that if a volume is attached, it can’t be attached again. see nova/volume/cinder.py: check_attach()
Allow users to share volumes between multiple guests using either read-write or read-only attachments. Clustered applications with two nodes where one is active and one is passive. Both require access to the same volume although only one accesses actively. When the active one goes down, the passive one can take over quickly and has access to the data.
The Changes needed in Nova are related to attach time and detach time.
At attach time, nova has to remove the assumption that it can only attach a volume if it’s not ‘in-use’. A Cinder volume can now be attached if it’s ‘available’ and/or ‘in-use’. Cinder will only allow a volume to be attached more than once if it’s ‘shareable’ flag is set on the volume at create time.
At detach time, nova needs to pass a new parameter to the cinderclient to tell cinder which specific attachment it’s requesting cinder to detach. Since a volume can be attached to an instance and/or a host, a new attachment uuid is added at detach time. Passing only an instance uuid is insufficient. The attachment_id will be optional in the cinderclient. If it isn’t passed in and there are multiple attachments, then cinder will fail because it won’t know which attachment to detach. By default libvirt assumes all disks are exclusively used for a single guest. If you want to share disks between instances, you need to tell libvirt when configuring the guest XML for that disk. Libvirt can reject the request to avoid problems with data consistency e.g. host level I/O caching we need to use cache=none.
The only alternative is for a user to clone a volume and attach the clone to the second instance. The downside to this is any changes to the original volume don’t show up in the mounted clone.
Data model impact¶
REST API impact¶
In the libvirt driver, the disk is given a shared SELinux label, and so that disk has no longer strong sVirt SELinux isolation.
Other end user impact¶
The command line will now allow you to call nova volume-attach for a volume to multiple instances.
Other deployer impact¶
Any time new code is added to Nova that requires a call to detach a volume, the developer must get the volume attachment uuid for the instance. This information is embedded in the cinder volume volume_attachments list.
Based on the work from Walter Boring and Charlie Zhou. Agreed with Walter to start the work again.
- Primary assignee:
Update the use of cinderclient to extract the new list of volume attachments when Nova fetches a volume.
Update all calls to cinderclient.detach() to include the attachment uuid.
Libvirt volume driver.
This requires a new version of the python-cinderclient. The changes in the client include the new detach API. https://blueprints.launchpad.net/python-cinderclient/+spec/multi-attach-volume
This also requires a patch in cinder to support the ability to attach to multiple instances. https://blueprints.launchpad.net/cinder/+spec/multi-attach-volume
We’ll have to add new Tempest tests to support the new Cinder volume shareable flag. The new cinder shareable flag is what allows a volume to be attached more than once or not. Have to look into a tempest test for attaching the same volume to multiple instances.
We will have to update the docs to discuss the new ability to attach a volume to multiple instances if the cinder shareable flag is set on a volume.
This is the cinder wiki page that discusses the approach to multi-attach https://wiki.openstack.org/wiki/Cinder/blueprints/multi-attach-volume