Refactor ISCSIDriver to support other iSCSI transports besides TCP

https://blueprints.launchpad.net/nova/+spec/add-open-iscsi-transport-support

Currently open-iscsi in openstack does not allow the use of offload iscsi transports (cxgb4i, qla4xx, bnx2i etc) or other software iscsi transports (like iser). This blueprint aims to add support for this

Problem description

open-iscsi has hardware iSCSI transport support. The interface argument specifies the hardware iSCSI interface to use for the operation. Any hardware vendors that currently provide hardware interfaces for open-iscsi cannot use them in openstack as the interfaces (iface) argument is not currently supported in the iSCSI driver.

While this effort is being done by Chelsio, the approach is currently to make it generic enough for all supported HBAs (Host Bus Adapter).

Use of such iSCSI hardware transports requires providing a corresponding interface file (referred to as iface), which can be autogenerated via iscsiadm but need correctness checks or can also be generated on the fly, given the 3-tuple of ipaddress, hwaddress & transport name to be used. The iface format itself is generic and does not use any propreitary fields irrespective of transport being used.

This also addresses a second problem with the current transport support in nova, namely the iSER driver. It is currently a full copy of the normal open-iscsi driver code with an extra parameter setting transport to ‘iser’

This will also enable use of iSER directly through the iSCSI driver, though deprecating the iSER driver is not the aim of this spec, and will probably start with iSER disabled, (and also because iSER requires target support unlike hardware transports) Previous relevant specs are here :

https://review.openstack.org/#/c/86637/

https://blueprints.launchpad.net/nova/+spec/add-transport-support-to-iscsi

https://blueprints.launchpad.net/cinder/+spec/add-transport-support-to-iscsi

A follow up blueprint is expected to deprecate the separate iSER driver and just roll functionality into using transport support.

Use Cases

User has an HBA that supports hardware iSCSI installed in his system. Using open-iscsi’s hardware transport support allows for iSCSI communication at higher speeds and reduced CPU usage.

Current transports supported by open-iscsi are :

  • Chelsio : cxgb3i, cxgb4i

  • QLogic : qla4xxx, bnx2i (formerly Broadcom)

  • Emulex : be2iscsi, ocs

  • Mellanox & others : iser

Use of any of these transports requires the appropriate hardware to be present in the system, but iSER will not be supported at the very start.

Project Priority

None

Proposed change

On the initiator side the only difference between using software iSCSI and hardware iSCSI transport is the interface flag (–interface=[iface])

e.g. “iscsiadm -m discovery -t st -p ip:port -I <iface file>”

iface files can automatically be generated for all hardware present on a system that is supported by open-iscsi by ‘iscsiadm -m iface’. The default format for iface file names is in the form <transport_name.hw_address>

The default iface name is ‘default’. Not using the -I parameter (which is how open-iscsi is currently being used in nova) is the same as specifying

‘iscsiadm -m discovery -t st -p ip:port -I default’

Required changes are :

  • A new parameter to specify which transport should be used while doing iscsi discovery/login, falling back to TCP if operation does not succeed, or if parameter is not provided (-I default)

  • An alternative (but lot more complicated ) approach might be to figure out iface name from source ip address. ipaddress can be used to figure out hwaddress and pci dev id. pci dev id provides transport name, iface names are in the format <transportname>.<hwaddress>

  • Optional/unresearched : Further down the line, it might make sense to see if networking services can be integrated to provide a networking interface for open-iscsi transports, but its out of the scope of this spec.

Other related changes :

  • Verify hardware exists before attempting to use said transport. The best way to accomplish this is checking PCI vendor ids/MAC address if nova would be open to accepting such code.

Alternatives

Use manual discovery of iscsi-targets, and provide -I parameter when doing discovery. Transport names are written in the resultant records generated by open-iscsi, however this does not guarantee hardware transports will be used, as this can be overridden with depending on how cli params are constructed.

Data model impact

None

REST API impact

None

Security impact

None

Notifications impact

None

Other end user impact

Performance Impact

iSCSI performance is improved drastically, no extra commands are added to driver, except perhaps one to generate the default iface files : ‘iscsiadm -m iface’

Other deployer impact

This can be used as a stepping stone to eventually replace & deprecate LibvirtISERVolumeDriver as ‘iscsiadm -I iser’ can be used for iSER support without requiring a separate driver.

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

anish7 <anish@chelsio.com>

iSER driver deprecation & followup spec Assignee :

Work Items

  • Add -I parameter to all iscsiadm login/discovery commands in ‘nova/virt/libvirt/volume.py’

  • Add transport parameter to config options, which defaults to software iscsi i.e. ‘-I default’

Dependencies

None as such. However, the minimum information required for using a tansport if it is not already configured is a 3 tuple of ip, transport name & hardware address. Functionality to this effect would be good to have for automatic configuration of transports.

Testing

Current iSCSI tests will work just fine. Will be testing on Chelsio hardware, and any other transport supporting hardware that can be obtained.

Documentation Impact

New transport parameter will need relevant documentation.

References