Port NUMA affinity policy

https://bugs.launchpad.net/neutron/+bug/1886798

Add a new parameter to “port” object setting the NUMA affinity policy.

Problem Description

Currently Nova allows to define the “numa_affinity_policy” for PCI devices. Those PCI devices could be, for example, network cards. When a port is created using this physical device (an SR-IOV port with VNIC type “direct”, “direct-physical”, “macvtap” or “virtio-forwarder”), Nova will select the host to spawn the virtual machine depending on the host NUMA topology availability and the NUMA nodes associated to this PCI network device [1].

However this filtering cannot be done currently for other backends than SR-IOV. For example, Open vSwitch with DPDK will run the PMD threads attached to specific CPU cores (and to specific NUMA nodes) [2]. Nova represents this information assigning a set of NUMA nodes to each defined physical or tunneled network [3]. If the port NUMA affinity policy is provided, Nova will enforce it during the scheduling.

Proposed Change

The goal of this spec is to create a “numa_affinity_policy” parameter applicable to all ports. This information will be provided to Nova that will use it while scheluding the virtual machine, independently of the network backend.

Of course, Nova should have the network backend NUMA information topology. For example, since [3], Nova can be statically configured with the physical networks NUMA location. With the port “numa_affinity_policy” parameter, the Nova scheduler will filter those hosts matching the required policy.

This spec covers the missing cases defined in [4]. Defining the NUMA affinity policy per port (interface), includes all type of ports, not only the SR-IOV interfaces. It also avoids defining the NUMA affinity in the Nova flavor; instead of this, a generic flavor can be used with ports with different policies.

This spec proposes to add a new parameter to the “port” object. This parameter, “numa_affinity_policy”, will be an string defining the NUMA affinity policy, based on the PCINUMAAffinityPolicy [5] enum defined in Nova. That field could have different values: “required”, “legacy” and “preferred”.

This information will be populated in the “port” dictionary when informing to Nova. This parameter will be added as a new extension in the port dictionary, in a new field called “numa_affinity_policy”. By default, the value of this new parameter will be “None”, to keep backwards compatibility.

Data Model Impact

A new table, “portnumaaffinitypolicy”, will be created:

Attribute Name

Type

CRUD

Default Value

Validation/Conversion

port_id

uuid-str

R

numa_affinity_policy

str

CRU

None

enum (including None)

This child table will depend on the “ports” table. Each row will have a 1:1 relationship with a “port” row and will be deleted when the “port” row is deleted too.

REST API Impact

The parameter “numa_affinity_policy” in the “port” API:

NUMA_AFFINITY_POLICY_VALUES = (None, 'required', 'preferred', 'legacy')

RESOURCE_ATTRIBUTE_MAP = {
    'port': {
        'numa_affinity_policy': {
            'allow_post': True,
            'allow_put': True,
            'validate': {'type:values': NUMA_AFFINITY_POLICY_VALUES}
            'default': None,
            'is_visible': True}
    }
}

The parameter can be updated only if the port is not bound. That check does not depend on the API but on the server.

Security Impact

None

Performance Impact

None

Operators CLI Impact

An additional parameter will be added to the OSC “port” CLI interface, in the create, set and unset commands.

For logging resource:

openstack port create [--numa-policy-required | --numa-policy-preferred |
                       --numa-policy-legacy]

openstack port set [--numa-policy-required | --numa-policy-preferred |
                    --numa-policy-legacy]

openstack port unset --numa-policy

Implementation

Assignee(s)

Primary assignee:

Rodolfo Alonso Hernandez <ralonsoh@redhat.com> (IRC: ralonsoh)

Testing

  • Unit Test

  • Functional test

  • API test

Documentation Impact

User Documentation

  • Add CLI usage into the networking guide for operator.

References