Multiple DHCP agents in Neutron

https://blueprints.launchpad.net/fuel/+spec/fuel-multiple-dhcp-agents

In FUEL 6.0 and before HA network solution was based on one DHCP agent, which was switchable between controllers.

This blueprint describes a way of using multiple DHCP agents instead of single. It is required for network scalability and neutron performance improvements.

Problem description

When virtual network in Neutron is created, it is scheduled to the DHCP-agent (to one of alive if we had multiple agents using random selection). In MOS version 6.0 and before Neutron server didn’t monitor life cycle of agent serving the network. There was only one DHCP agent in a cluster and if the DHCP-agent service stopped or connectivity with a node containing the DHCP agent was lost, Neutron server didn’t reschedule networks. So HA network solution was based on the custom script which used API calls. Also in case of large number of networks with DHCP server the only agent was overloaded.

Proposed change

Neutron supports multiple DHCP agents, which can be used in two ways. At first, all created networks can be distributed between available DHCP agents. Also each network can be served with more than one DHCP agent simultaneously. These functions give us a possibility to build more efficient and reliable HA solution. Neutron server automatically monitors DHCP agents lifecycle. In case of failure of all DHCP agents serving the network it will be unscheduled from the dead agents and scheduled to alive agents. So external rescheduling solutions aren’t needed anymore.

Multiple DHCP agents will be enabled for clusters by default. There’re three possible ways to configure number of DHCP agents per network: 1. each network is served by only one agent. This is the fastest solution from the point of view of network performance, because all networks are distributed and every agent does only the necessary work. HA is based on agents rescheduling. 2. each network is served by two agents. In this case the failure of one agent is competely transparent for clients so it’s better HA solution. There’s slight performance impact on large number of networks due to redundancy. 3. each network is served by 3 or more agents. It’s the most reliable solution from clients’ point of view because rescheduling is required only in case of changes in controller nodes configuration. Due to high redundancy there will be performance degradation in case of large number of networks.

The best solution will be selected after testing of all possible variants.

The Pacemaker OCF script for Neutron DHCP agents has all necessary functionality so no changes are required.

This feature allows to have faster and more reliable DHCP service for instances. Also it allows to effectively distribute virtual networks between all available agents to improve network performance on large environments.

Alternatives

None

Data model impact

None

REST API impact

None

Upgrade impact

None

Security impact

None

Notifications impact

None

Other end user impact

None

Performance Impact

• Time delays for answers from DHCP servers will decrease.
• Network scalability will grow.
• Load on a separate controller will be decreased.
• Customers will get a possibility to add any number of nodes with started neutron agents and network scalability will grow.

Other deployer impact

None

Developer impact

None

Implementation

• In astute.yaml we have following options:
  • quantum_settings/L3/multiple_dhcp_agents (default=true)
  • quantum_settings/L3/dhcp_agents_per_network (default=3)
• cluster::neutron::dhcp classes got “multiple_agents” option enabled by default, which allows to configure agents for running in multiple-agent mode
• cluster::neutron::dhcp got “agents_per_net” option, which describes amount of dhcp-agents to serve each network.

Backward compatibility

Using “multiple_agents” option for OCF script we can manipulate behavior of DHCP agent. Moreover, for using old-style behavior of DHCP agent we should decrease clone size for corresponded Pacemaker resources to “1”.

Work Items

• Update Puppet manifests to enable multiple DHCP agents
• Add necessary patches to Neutron for network rescheduling mechanism

Assignee(s)

Sergey Kolekonov Eugene Nikanorov

Dependencies

None

Documentation Impact

New Neutron-server behavior in case of dead DHCP agents should be reflected in documentation to correctly debug possible problems.

References

None

Testing

• Deploy HA cluster
• All instances must get IP addresses correctly via DHCP even in case of whole controller failure or particular cases such as message broker failures