Add support for multiple RabbitMQ clusters¶

date:: 2016-07-11 21:00
tags:: rabbitmq, messaging, notifications, scalability

Larger deployments may wish to provision multiple RabbitMQ clusters such that each cluster is deployed on its own set of hosts.

Such functionality would allow a deployer to configure one or more additional component and container skeletons to add inventory groups to be used for the deployment of additional clusters.

Problem description¶

The current playbook and roles assume a single inventory group: rabbitmq_all that is deployed on the shared-infra_hosts infrastructure. The inventory group name is hardcoded throughout and the playbook makes the assumption that only one cluster will ever be needed.

Proposed change¶

Modify the rabbitmq_server role to be more configurable with respect to the inventory group(s) that it operates upon

Modify the rabbitmq-install play to be more configurable with respect to the inventory group it operates upon

Alternatives¶

I’m not aware of alternative ways for the project to address this need.

Playbook/Role impact¶

Initial impacts will be to the rabbitmq_server role and the rabbitmq-install play. However, I expect that additional impacts may exist within other roles such that they would need to change to be more configurable with respect to the inventory group they expect to use for rabbit hosts, or the variables they use to identify which rabbit hosts they should connect to.

Upgrade impact¶

Unclear on how upgrades would be impacted. To my knowledge, custom inventory extensions are not currently handled in the upgrade automation.

Security impact¶

No unique security impacts. The existing RabbitMQ security posture will be maintained, though additional secrets may be required.

Performance impact¶

None expected/anticipated.

End user impact¶

End users will have increased flexibility in defining their deployment architecture.

Deployer impact¶

The goal is for the deployer impact to be negligible due to the opt-in nature of the changes discussed.

Developer impact¶

This change will add some additional complexity for developers, but it should be minimal.

Dependencies¶

None

Implementation¶

Assignee(s)¶

Primary assignee:: travis-truman (automagically)

Work items¶

rabbitmq_server role modifications for inventory group configurability

rabbitmq-install play modifications for inventory group configurability

Documentation explaining how to create additional RabbitMQ cluster groups

Other role modifications to support cluster connectivity configurability

Testing¶

This should be able to be tested within the rabbitmq_server role functional tests given some changes to the test inventory.

Documentation impact¶

An appendix should be added that explains to deployers how to configure their environment for RabbitMQ multiple cluster support.

References¶