Support SRIOV interface attach and detach¶
Nova supports booting servers with SRIOV interfaces. However, attaching and detaching an SRIOV interface to an existing server is not supported as the PCI device management is missing from the attach and detach code path.
SRIOV interfaces cannot be attached or detached from an existing nova server.
As an end user I need to connect my server to another neutron network via an SRIOV interface to get high throughput connectivity to that network direction.
As an end user I want to detach an existing SRIOV interface as I don’t use that network access anymore and I want to free up the scarce SRIOV resource.
In the compute manager, during interface attach, the compute needs to generate
InstancePCIRequest for the requested port if the vnic_type of the port
indicates an SRIOV interface. Then run a PCI claim on the generated PCI request
to check if there is a free PCI device, claim it, and get a
object. If this is successful then connect the PCI request to the
RequestedNetwork object and call Neutron as today with that
RequestedNetwork. Then call the virt driver as of today.
If the PCI claim fails then the interface attach instance action will fail but the instance state will not be set to ERROR.
During detach, we have to recover the PCI request from the VIF being destroyed then from that, we can get the PCI device that we need to unclaim in the PCI tracker.
Note that detaching an SRIOV interface succeeds today from API user perspective. However, the detached PCI device is not freed from resource tracking and therefore leaked until the nova server is deleted or live migrated. This issue will be gone when the current spec is implemented. Also as a separate bugfix SRIOV detach will be blocked on stable branches to prevent the resource leak.
There is a separate issue with SRIOV PF detach due to the way the libvirt domain XML is generated. While the fix for that is needed for the current spec, it also needed for the existing SRIOV live migration feature because that also detaches the SRIOV interfaces during the migration. So the SRIOV PF detach issue will be fixed as an independent bugfix of the SRIOV live migration feature and the implementation of this spec will depend on that bugfix.
Data model impact¶
REST API impact¶
Other end user impact¶
There will be an extra neutron call during interface attach as well as
additional DB operations. The
interface_attach RPC method is synchronous
today, so this will be an end user visible change.
Other deployer impact¶
- Primary assignee:
- Feature liaison:
change the attach and detach code path
add unit and functional tests
Tempest test cannot be added since the upstream CI does not have SRIOV devices. Functional tests with libvirt driver will be added instead.
remove the limitation from the API documentation