Kafka Driver Revisions for Messaging Notifications¶
This specification proposes changes to the existing kafka driver that allows notification message transport over the Apache Kafka distributed streaming platform 1. The blueprint for the original driver implementation can be found here 2 and the spec for the implementation can be found here 3. The goal of the changes described by this specification is to transition the driver from ‘experimental’ to ‘supported’ status in order to encourage driver adoption. The driver will (continue to) only support notification messaging as the use of the kafka server is non-optimal for RPC messaging patterns. Thus, this kafka driver is intended for deployment in hybrid messaging configurations where RPC messaging will be provided by a separate messaging backend.
The original kafka driver was introduced during the mitaka release cycle. Adoption of the driver has been limited due to a number of factors such as its ‘experimental’ designation and its intended use for notification messaging only. Meanwhile, the kafka server has experienced widespread adoption and is frequently included in application architectures to provide accurate analytics in cloud monitoring systems. This success and the progression of configuration frameworks to easily enable hybrid messaging deployments in OpenStack is a catalyst to revise this driver and provide active maintenance and support going forward.
Kafka hybrid oslo.messaging deployment:
+------------+ +----------+ | RPC Caller | | Notifier | +-----+------+ +----+-----+ | | | | v v +-------------------+ +-------------------+ | RPC | | Notification | | Messaging Backend | | Messaging Backend | | (amqp:// or | | (kafka://) | | rabbit://) | | | +--------+----------+ +--------+----------+ | | | | v v +------------+ +------+-------+ | RPC | | Notification | | Server | | Server | +------------+ +--------------+
The revision to the kafka driver is not a major rework. A number of issues need to be resolved in order to support Notifications over a kafka server messaging backend:
virtual host (vhost) emulation
encryption and authentication
functional and integration testing
The driver should be updated to support the latest kakfa server software release. As of this writing, the latest versions are:
scala version - 2.12
kafka version - 1.0.0
kafka-python version - 1.35
Virtual Hosts (vhost) Emulation¶
Currently, only the rabbitmq messaging backend supports vhosts contained in the transport url 4. Since the kafka server architecture does not natively support vhosts, the kafka driver revision will emulate vhost support by adding the virtual host name to the topic created on the kafka server. This will effectively create a private topic per virtual host that is configured for use.
Related to this change the devstack kafka plugin will need to be updated so that the kafka backend does not fail.
Encryption and Authentication¶
The Apache Kafka allows clients to connect over SSL. By default, SSL is disabled and the kafka driver will be updated to enable it via configuration. This release will support server authentication and client authentication will be planned for a future release and will be dependent upon client library capabilties. The configuration for SSL will be the same for both producer and consumer:
In section [oslo_messaging_kafka]:
ssl - attempt to connect via ssl
ssl_ca_file - file containing the trusted CA’s digital certificate
SASL may be used with PLAINTEXT or SSL as the transport layer when there is a username and password present in the transport_url. The SASL configuration support is currently for PLAIN authentication only. The following configuration options will be provided by the driver:
In section [oslo_messaging_kafka]:
sasl_mechanisms - space separated list of acceptable SASL mechanisms
The revision to the kafka driver will include updates to a number of driver functional aspects to incorporate new features and to enhance driver support-ability:
config options - update the driver configuration options to include new security options as well as remove deprecated options removed from the oslo messaging library 5.
logging - add additional info, warning, debug messages to the driver to help operational and debugging tasks when deploying the driver
check python client - check for installed library dependencies
connection management - review and identify any simplification that would benefit driver operation and support
ack/requeue message - investigate support of manual message commit in order to support message requeue following notify message dispatch
Presently, there are alternative oslo messaging drivers that can be used for different messaging backends. With hybrid messaging support, there is the flexibility to optimally align the messaging backend with the RPC or Notification communication patterns provided by the oslo messaging library. The objective to support and maintain the kafka driver should enhance the overall value of oslo messaging by providing users messaging backend alternatives that best suit their operational objectives and needs.
The alternative is to deprecate this driver and support a single messaging backend for notifications (e.g. rabbit broker).
Impact on Existing APIs¶
The existing API should not require any changes. The changes to the kafka driver will preserve compatibility with existing experimental kafka deployments and will not affect other oslo.messaging drivers.
With the additional support of authentication and encryption, there will be an expansion of the security model provided by the driver through its use of the python client library and its interactions with the kafka server for message exchange.
Any performance impact should be limited to the users of the kafka driver for messaging notifications. Users of other drivers such as rabbitmq and amqp 1.0 will not be affected. Any performance changes realized in the kafka driver update may be due to:
changes to the underlying kafka protocol in the new server version
New configuration items for authentication and security will be added as detailed above. The default value for these options will be determined as the driver is updated and revised.
To be considered as supported, any new features added to oslo.messaging that must be implemented via driver modification should be implemented in the kafka driver as well. In the circumstance when a new feature requires behaviors/capabilities that cannot be provided by kafka, clients attempting to use the feature will cause a NotImplementedError exception to be raised. Additionally, the absence of supported functionality must be documented and included in the release notes.
The kafka server will be used as the messaging backend for notifications in testing. An alternative backend such as rabbit or amqp 1.0 should be used as the messaging backend for RPC.
The kafka driver tests should be expanded as necessary for the new features and capabilities in the update and the driver must pass all tests.
The driver must pass the following gate checks with deployed in a hybrid messaging configuration (e.g. when kafka is configured as the notification backend):
The zookeeper, kafka, jdk and client will need to be avabilable in the CI environment in order to fully test this driver.
Target Milestone for completion: queens
Update environment for latest software release updates and dependencies
Implement virtual hosts support
Implement SSL and SASL integration
Implement new configuration items
Functional test integration
Update devstack plugin
Upstream CI integration
Send announcement to openstack-discuss following release
The kafka driver is expected to be adopted in hybrid messaging deployments as the notification messaging backend. Adoption is likely in environments that already have kafka servers broadly deployed (e.g. operational benefit) or where the characteristics of the kafka server best suit the information analytics requirements.
Anticipated API Stabilization¶
The kafka driver documentation will be added to the libary. This documentation will follow the style of documentation provided by the other drivers and should include the following topics:
theory of operation (overview) of the Apache Kafka messaging backend
driver options overview
kafka server operations
platforms and software
The driver revision will require no additional dependencies.
This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode