Floating IPs for Routed Networks¶
This specification proposes the implementation of floating IPs for Routed Networks.
Neutron Routed Networks are composed of individual L2 segments stitched together by L3 routers in the operator infrastructure. The operator has the option to route the fixed IP addresses of the ports in such a network to the external world, but this might not be feasible for IPv4 without a large enough externally routable address range.
Routed Networks were implemented in the first place to support a large number of ports / VMs, beyond the constraints of networks based on a single L2 broadcast domain. Operators want to be able to support a number of ports / VMs as large as possible and route a subset of them externally, without being limited by the availability IPv4 address ranges.
This specification proposes to enable the association of ports in a Routed Network with floating IP addresses (floating IPs). This will enable operators and their users to create ports in routed networks with fixed IP addresses in private ranges and assign to them scarce publicly routable IPv4 addresses only when needed, through the association of floating IPs. This functionality will require the changes described in the following sections.
Creation of subnets spanning segments¶
To support the use of floating IP’s on a routed provider network, there needs to be a way to create a subnet that is not bound to a segment. In its current state, a subnet cannot be created on a network without specifying a segment when one or more subnets on the network are already bound to a segment.
A new subnet service type of
network:routed will be introduced. When a
subnet supports this service_type it can be associated to the network of a
routed provider network rather than having to be associated to one of the
segments that comprises the routed network. This provides a way of expressing
the notion of a subnet that spans the segments of a routed network. As such,
this not only supports the use of floating IP’s on routed networks but also
allows various neutron backends to handle these subnets in innovative ways.
The current definition of a routed provider network is a network in which
each subnet is associated to a different segment. This definition will change
subtly and include networks where 1 or more subnets with
service_types is associated with the network but none of the individual
segments comprising it.
To enable this, minor changes to
neutron/db/ipam_backend_mixin.py that allow for subnets with this special
service type to exist outside of any segment.
Service subnets and floating IPs for routed networks¶
Service subnets enable operators to define valid port types for each subnet on a network, without limiting networks to one subnet or manually creating ports with a specific subnet ID. Using this feature, operators can ensure that ports for instances and router interfaces, for example, always use different subnets.
To allow floating IP’s to be created on a routed network, an operator will
leverage this functionality when creating the floating IP subnet. To create a
floating IP subnet that can be used on a routed provider network, the subnet
should be created with two
network:routed allows the subnet to be pinned to
the network rather than a particular segment, and
allows for floating IP’s be allocated from the subnet.
No changes are anticipated to enable this to work properly. Documentation will be updated to explain the workflow for operators. No change to the current workflow for creating/associating/disassociating/deleting floating IP’s will be introduced.
DVR and routed provider networks¶
No changes to the current operation of DVR need to be made in support of
these changes. Changes to documentation will be made to reflect the fact that
subnets on segments where you wish to have floating IP gateway ports built
should be created with the
network:floatingip_agent_gateway should be used to indicate that compute
nodes can reach the routed external network on a given segment. For the
network node connectivity, a
should be used to indicate centralized router ports will be instantiated on
the proper segment(s).
Because only the ToR device providing connectivity to the network segment would know how to reach the floating IP (DVR will send a gratuitous ARP), connectivity from outside the segment must be provided host routes that steer traffic to the appropriate segment. This can be achieved by enabling neutron-dynamic-routing, enabling it to announce FIP reachability to ToR routers, and ensuring ToR routers propogate the FIP host route with an appropriate next-hop. Documentation will be added to explain how this can be set up by an operator.
Floating IPs advertisement with BGP Dynamic Routing¶
BGP Dynamic Routing is a Neutron Stadium project that enables advertisement
of self-service (private) network prefixes to physical routers that support
BGP, thus removing the conventional dependency on static routes. A
speaker advertises the self-service network prefixes to associated
peers, which represent the BGP capable physical routers in the operator
In its current state, neutron-dynamic-routing determines next-hops for floating IP’s by identifying the relevant router and floating IP agent gateway ports and mapping floating IP’s to their proper endpoint. In this way, it is completely agnostic of segments and is capable of properly discovering and announcing the correct next-hop for a /32 host route of a floating IP.
This change is not expected to pose any new security concerns.
Other End User Impact¶
No client impacts are anticipated. The network guide will be updated with proper documentation and example configurations.
For some operators, there is a desire to allow for floating IP’s to be associated to a port without first connecting a router to an external network and a tenant network. This can be implemented in many ways. One way is by relying on a cloud-init-like process to set the FIP as a loopback address inside a VM and rely on a BGP process to announce the FIP with the fixed IP as the next-hop address for the floating IP. With the implementation described in previous sections in place, a similar scheme would be the next logical iteration of routed provider networks.
Introduce the new
network:routedsubnet service type.
Refactor network segment integrity checks to allow subnets with a
network:routedto be associated directly to a network.
There are currently no tempest tests for routed provider networks. However, some tests are being developed and tests that cover floating IP creation on routed provider networks can eventually be added to these tests.
The following tests can be added to the suite of scenario tests being developed in https://review.opendev.org/#/c/665155/ :
Floating IP subnets can be created on a routed provider network
Floating IP’s can be created on a routed provider network
Floating IP’s from the same subnet can be associated to ports across different segments
Tests in neutron-dynamic-routing to assert proper route discovery on routed provider networks
Document new behavior for creating floating IP subnets
Document how to use in conjunction with neutron-dynamic-routing
Example configurations of routed networks, ToR router BGP config, and neutron-dynamic-routing.
A new section to the Neutron devref will be added describing the implementation of floating IPs for routed networks.