Dynamic Advertising Routes for Public Ranges

Blueprint Link

The goal of this blueprint is to add dynamic routing capability to OpenStack deployment. This feature would allow Neutron to advertise its own routes to an external router using the BGP protocol.

Problem Description

This is a new feature and this section will describe use cases where dynamic routing may be useful.

Expose External Networks Dynamically

Allow Neutron to dynamically announce to external uplink routers.

An OpenStack cloud would run a routing protocol (for example, BGP) against at least one router in each uplink network provider. By announcing network prefixes to those peers, the Neutron network would be reachable by the rest of the internet via both paths. If the link to an uplink provider broke, the failure information would propagate to routers further up the stream, keeping the cloud reachable through the remaining healthy link.

Sample Topology

../../_images/bgp-sample-topology-1.png

Please note that in this diagram, the red gateway router is actually a linux machine with a software BGP speaker and a software router. This blueprint proposes to configure this BGP speaker through the Neutron API (see below).

This also would be a valid scenario for an internal cloud, where the uplink router is actually one’s provider router and the routes are advertised using iBGP.

Routed Model for Floating IPs on External Network

This use case will allow an external network with a large public IP space to possibly span more than one L2 network. This could allow for improved scale and isolation between AZs while maintaining the appearance of a large external network where floating IPs and networks can float freely.

This use case may also include announcing public networks behind Neutron routers to an upstream router. For example, this will be useful for IPv6 networks when using the subnet-allocation mechanism to allocate routable IPv6 prefixes to tenant networks. It does not require learning routes from the upstream router.

Address Scopes blueprint adds a new L3 context to make public routable tenant networks (among other advantages). Address Scopes may leverage this development to advertise the new routable subnets to uplink routers.

Sample Topology

../../_images/bgp-sample-topology-2.png

The topology of this use case can be seen as a generalization of the previous one, with a multi-homed OpenStack installation and leverage the fact that a floating IP can be seen as a /32 network.

Proposed Change

Overview

A new system that dynamically advertises routes to other peers outside the OpenStack deployment is proposed. From the Neutron API, cloud administrator should be able to define these peers as well as the way to interact with them.

Routing Peer

A system that supports dynamic routing must be able to advertise its own routes to its peers. It order to achieve this goal the system must know the list of its peers and be able to trust the information that it receives from them.

Route Advertisement

Routes added manually by administrator will be advertised to external dynamic routing peers.

Dynamic Routing System

In default implementation a new system will be used to manage dynamic routing information at the edge of OpenStack deployment. Dynamic peering will not be performed by each individual Neutron router due to scaling concerns. Such an approach could create an inordinately high number of peering relationships. Instead, the model proposed sets up a speaker to represent the Neutron deployment as a whole to external routers.

This system will allow different implementations (develop your own BGP speaker implementation, for instance) to fit into third party requirements.

IPv6 Impact

The implementation must be able to exchange IPv4 and IPv6 routes. This blueprint may be even more important with IPv6 to complete routing from outside the cloud to tenant networks running IPv6.

Solution Proposed

Reference implementation will employ a dynamic routing agent (dr_agent). This agent will have direct connectivity with configured peers and it will be responsible to update the dynamic routes of some defined routers. Another spec will be filled to define the details of this implementation.

So, through the Neutron API, cloud administrator should be able to define the peer connections for each agent according to his needs.

The cloud administrator will be able to execute the following new actions, through the Neutron API:

  1. Create a BGPSpeaker entity with configuration options of the BGP connection. First approach will only need the local_as attribute. Future implementations, such as the policy support for dynamic routing, will need to add more attributes here. The implementation driver need to read this configuration options an establish BGP sessions to its peers.

  2. Create a RoutingPeer entity.

  3. Associate a RoutingPeer entity to a BGPSpeaker. The BGPSpeaker implementation driver will be able to connect to this RoutingPeer

  4. Associate a Router to a BGPSpeaker. That means the router will have dynamic routing capabilities that will let it advertise its routes

  5. Associate a Network to a BGPSpeaker. All the routers with external gateways attached to this network will use the same BGPSpeaker attributes.

At this point, Neutron will compare the address scope of the subnets to which assigned routers (step 5 implies all the routers of the external network) have their external gateways connected to the internal subnets of the tenants to see if they belong to the same address scope. If so, the BGP speaker implementation will advertise these tenant subnets to its configured RoutingPeers. Floating IPs will be included implicitly since they are allocated from the external network.

Considerations

Alternatives

Multi-homed clouds can be handled using classic networking infrastructure, configuring manually the vendor router with BGP outside the OpenStack deployment. This is limited. It can’t handle the routed floating ip model proposed above.

Data Model Impact

This document proposes modifying data objects and schema in the following way. For a quick glance of the Data Object Model, check out this etherpad.

Data Object Changes

Three new data model classes will be added: BGPSpeaker, RoutingPeer and AdvertiseRoute.

We will need the binding entities:

  • RoutingPeerBGPSpeakerBinding to associate peers to BGPSpeaker.

  • RouterBGPSpeakerBinding to associate routers BGPSpeaker.

  • NetworkBGPSpeakerBinding to associate networks to BGPSpeaker.

New BGPSpeaker class will contain the following attributes:

  • id: UUID of the entity.

  • local_as: Local AS value.

Now we only need these values. In the future, more advanced configuration options of BGPSpeaker will be able to be added here.

New RoutingPeer class that represents a peer connection will contain the following attributes:

  • id: UUID of the entity

  • ip: IP Address of the peer.

  • remote_as: Remote Peer’s AS value.

  • auth: Authentication data of the connection that can be serialized as a

    dictionary. { type: ‘MD5’, password: ‘234a23d10234’ } could be a simple example and first approach.

Another data object called AdvertiseRoute will be created extending the Route entity and associated to a router. Will have the following attributes:

  • nexthop: IP address of the next hop.

  • destination: CIDR prefix.

  • router_id: UUID of the Routing Instance.

RoutingPeerBGPSpeakerBinding is an n-to-n relationship between bgp peer and bgp speaker and only will have:

  • peer_id: UUID of the BGPPeer

  • bgpspeaker_id: UUID of the Dynamic Routing Agent

RouterBGPSpeakerBinding is an n-to-1 relationship with the attributes:

  • router_id: UUID of the Router

  • bgpspeaker_id: UUID of the Dynamic Routing Agent.

NetworkBGPSpeakerBinding is an n-to-1 relationship with the attributes:

  • network_id: UUID of the Network

  • bgpspeaker_id: UUID of the Dynamic Routing Agent.

RouterBGPSpeakerBinding it will be used when you want to propagate a single router’s AdvertiseRoutes. This option is suitable for the Expose External Networks Dynamically use case, explained above. It can be seen as adding dynamic routing capabilities to a Router

NetworkBGPSpeakerBinding is the Routed Model for Floating IPs on External Network use case: in this case you propagate tenant routes (IPv6) or Floating IPs into an upstream router. These routes are more dynamic to assign, because it is up to the tenant use to set them. You will want any router attached to a network, to automatically be added to propagate its routes.

REST API Impact

API endpoints should be implemented according to the Solution Proposed section.

Security Impact

This feature will allow an external system to manipulate routing information within Neutron network. The external system should be trusted and may be authenticated using a shared secret.

Dynamic routing may only be configured by the system administrator.

Notifications Impact

A notification should be provided when connectivity of control channel over which routes are exchanged is interrupted

Other End User Impact

The following CLI commands will be added to manage dynamic routing specification for connecting OpenStack to outside networks:

  • bgp-speaker-list: List configured bgpspeakers.

  • bgp-speaker-show: Show detailed bgpspeaker configuration.

  • bgp-speaker-create Create new bgpspeaker connection.

  • bgp-speaker-update: Update bgpspeaker specification.

  • bgp-speaker-delete: Delete bgpspeaker specification.

  • bgp-speaker-peer-add: Associate a peer to a bgpspeaker.

  • bgp-speaker-peer-list: List Peers on bgpspeaker

  • bgp-speaker-peer-remove: Remove peers on bgpspeaker.

  • bgp-speaker-network-add: Associate a network to a bgpspeaker.

  • bgp-speaker-network-list: List networks on bgpspeaker

  • bgp-speaker-network-remove: Remove networks on bgpspeaker.

  • bgp-speaker-router-add: Associate a peer to a bgpspeaker.

  • bgp-speaker-router-list: List routers on bgpspeaker

  • bgp-speaker-router-remove: Remove router on bgpspeaker.

  • bgp-peer-list: List configured peers.

  • bgp-peer-show: Show detailed peer configuration.

  • bgp-peer-create Create new peer connection.

  • bgp-peer-update: Update peer specification.

  • bgp-peer-delete: Delete peer specification.

  • router-advertiseroutes-list: List advertise routes.

Horizon Requirements

A new screen will be added to configure gateway configuration for connecting OpenStack to outside networks. This screen will allow routes and peer configuration to be added to gateway configuration.

An external network will have an option to be linked to a routing instance.

Usage Example

Configure 2 uplinks for the routing instance serving an external network to advertise its routes and update the discovered ones.

Sample configuration using Neutron CLI commands:

neutron bgp-speaker-create --local-as 12345

neutron bgp-peer-create --ip 123.23.43.4 --remote-as=1234

neutron bgp-speaker-peer-add peer1 (previous step created peer. should be an
uuid, but modified for the sake of understanding)

neutron bgp-speaker-network-add bgpspeaker1 network1 (same here, with uuids)

Performance Impact

This feature describes an out of band mechanism to negotiate routing configuration. This feature should not have a performance impact on Neutron network.

Other Deployer Impact

This feature would have to explicitly enabled and configured before it will take effect. There are no changes to configuration files.

Developer Impact

This change does not affect current developments or any plugin development.

Neutron API exposed is agnostic of the exchange routing protocol used. If another developer want to provide other driver than BGP with exabgp, only the dr_agent part will be affected with new code.

Community Impact

This change does not impact community.

Alternatives

Taking into account the use case (BGP connectivity), we think that the agent approach is the only one that fits into Neutron. Although maybe the functionality would be solved using another entities and workflow.

Implementation

Assignee(s)

This is a pre-liminary contributor list

Primary assignee:

tidwellr vikram

Other contributors:

devvesa YAMAMOTO

Work Items

  • Create the dr_agent, exposing the API and implemented with the chosen BGP speaker. (BGP Comparison)

  • Model tables and API resources.

  • Periodically scheduled process to communicate with dr_agent.

  • Testing.

  • Devstack.

  • Documentation.

Dependencies

Depending on the implementation, new system library or python library will need to be installed.

Testing

Tempest Tests

Dynamic routing testing may be performed in an isolated environment. An external autonomous system may be simulated with an instance of BGP capable software router (for example, quagga).

The following dynamic routing scenarios could be tested:

Verify that when BGP is enabled on the gateway and one peer is configured the agent establishes BGP session with the peer, receives a list of routes, and submits advertised routes to the peer.

Verify that when BGP is disabled on the gateway and one peer is configured the dr_agent establishes no BGP sessions.

Verify that when BGP is enabled on the agent and 3 BGP peer connections are configured, the agent establishes 3 BGP sessions, one to each of the configured peers.

When 2 or more peers are configured, verify that BGP implementation is able to detect when the BGP session is interrupted the routes received from that BGP session are automatically removed from the routing table.

Functional Tests

Full top-down Neutron API internal logic must be developed by mocking the agent.

API Tests

All API exposed endpoints by Neutron extensions must be tested.

Documentation Impact

New documentation for the whole functionality.

User Documentation

User documentation explaining the functionality must be provided.

Developer Documentation

Developer documentation about how to develop a new driver for the dr_agent must be provided.

References

Previous work:

Links and helpers: