Add support for InfiniBand SR-IOV VIF Driver

https://blueprints.launchpad.net/nova/+spec/vif-driver-ib-passthrough

Adding support for InfiniBand SR-IOV vif type allows Virtual PCI device (VF) to be directly mapped to the guest, allowing higher performance and advanced features such as RDMA (remote direct memory access).

Problem description

Till Juno Release (including) InfiniBand (IB) SR-IOV networking was possible via out of the tree nova vif driver with neutron Mellanox Ml2 Mechanism Driver. Since Juno Ethernet SR-IOV device vNIC is supported by nova, and may be leveraged for IB SR-IOV device. IB SR-IOV vif plugging logic should be added to LibvirtGenericVIFDriver.

Use Cases

InfiniBand (IB) is popular, open standard, high performance and extreme efficiency interconnect protocol. To enable Big Data, High Performance Computing (HPC) and other similar use cases the guest requires direct access to IB NIC device.

Project Priority

None

Proposed change

This change adds vif_driver support for vif_type VIF_TYPE_IB_HOSTDEV as part of the GenericLibvirtVifDriver. Currently, there is no standard API to set InfiniBand vGuid (equivalent for Ethernet mac address), therefore special driver utility will be used to set it. This utility is already used in case of mlnx_direct vif_type.

In the neutron VIF_TYPE_HOSTDEV is already supported by ML2 Mellanox Mechanism Driver and enables networking configuration of SR-IOV Virtual Functions on IB Fabric. The vif_type name should be renamed to VIF_TYPE_IB_HOSTDEV to indicate specific InfiniBand vif_type.

Alternatives

Currently there is no valid alternative.

Data model impact

None

REST API impact

None

Security impact

Execution of the dedicated ‘ebrctl’ utility requires the use of sudo.

Notifications impact

None

Other end user impact

None

Performance Impact

None

Other deployer impact

To use this feature the deployer must use Infiniband enabled network adapters. Infiniband Subnet Manager should be running to enable IB Fabric. The ‘ebrctl’ utility should be installed on compute nodes.

Deployers need to be aware of the limitations: * There will be no smooth upgrade path from the out of tree solution [3] to the current solution. * The out of tree solution [3] will be deprecated.

Developer impact

This blueprint will have no developer impact.

Implementation

Assignee(s)

Primary assignee:

Moshe Levi

Other contributors:

Irena Berezovsky

Work Items

  • Add vif_type ib_hostdev support to LibvirtGenericVIFDriver

Dependencies

proprietary ‘ebrctl’ library is required. This is installed as part of the Driver OFED package.

Testing

Documentation Impact

No documentation changes for Nova are anticipated. VIF_TYPE_IB_HOSTEV will be automatically enabled by Neutron where appropriate.