Neutron LBaaS TLS - Specification¶
BP https://blueprints.launchpad.net/neutron/+spec/lbaas-ssl-termination
Terminating TLS connections on the load balancer is a capability expected from modern load balancers and incorporated into many applications. This capability enables better certificate management and improved application based load balancing e.g. cookie-based persistency, L7 Policies appliance, etc.
Problem Description¶
No TLS offloading capability is available for Neutron LBaaS
Proposed Change¶
Note: This document is referencing to the new LBaaS objects model proposed at https://review.openstack.org/#/c/89903
Note: This document does not consider flavors framework proposed at https://review.openstack.org/#/c/90070 Before the flavors framework is in place, specific back-end driver which does not support TLS capabilities should throw an exception stating a lack of TLS support once it gets request for listener with TLS configuration. This document specifies a “core” feature set that every back-end implementing TLS capabilities must comply. TLS capabilities of various back-end implementations may differ in future releases, thus flavors aspect should definitely be part of TLS capabilities specification
Note: Horizon project aspect is not a part of this specification.
Tenant will manage his TLS certificates using Barbican. Certificates will be stored in Barbican secure containers.
Barbican is in charge of containers life cycle management, containers classification and validation. LBaaS TLS requires a specific container type (TLS). Only container of this type will be listed to the tenant for selection while configuring listener’s TLS containers to use. Any invalid container usage will raise an error.
Barbican will also manage list of interested consumers for each container. See spec at https://review.openstack.org/#/c/99516
Neutron LBaaS (a consumer, according to Barbican’s terminology) will not use a regular GET request for container resource in order to get the container. Instead, it will use a POST request to container’s consumers resource (http://admin-api/v1/containers/{container-uuid}/consumers) with following info:
- {
“type”: “LBaaS”, “URL”: “https://lbaas.myurl.net/loadbalancers/<lb_id>/”
}
Note: We might want to use specific Listener URL instead of Loadbalancer’s one, like “https://lbaas.myurl.net/lbaas/listeners/<listener_id>”
As a response, it will get containers data like container’s resource GET request was used. Barbican, in its turn will store consumer’s (LBaaS instance URL) data in its database so this info can be used for getting all consumers of a specific TLS container.
As a result, Neutron LBaaS TLS implementation is required to:
Use only POST request to container’s consumers resource in order to get the container’s data.
Perform DELETE request to container’s consumers resource when stop using a container.
Barbican TLS container will contain PEM encoded data. Specific back-end implementation might convert the certificates data to other format such as DER, if needed.
In addition to existing HTTP, HTTPS and TCP, new protocol, TERMINATED_HTTPS will be added for listener creation
For tenant, creating listener with TERMINATED_HTTPS as a protocol means desire to offload incoming encrypted traffic. New configuration options will be available for listener’s configuration including:
Default TLS container id for TLS termination
TLS containers list for SNI
In case when specific back-end is not able to support TLS capabilities, its driver should throw an exception. The exception message should state that this specific back-end (provider) does not support listeners with TLS offloading. The clear sign of listener’s requirement for TLS offloading capabilities is its TERMINATED_HTTPS protocol.
New module will be developed in Neutron LBaaS for Barbican TLS containers interactions. The module will be used by Neutron LBaaS front-end API code and providers’ driver code. The module will be used for validation and data extraction from default TLS container and SNI containers. This module represents the only legitimate API for Barbican TLS containers interactions. See ‘SNI certificates list management’ section for detailed module specification.
When creating listener with TERMINATED_HTTPS protocol:
Front-end TLS offloading is always enabled - hard coded as a default behaviour for listener with TERMINATED_HTTPS protocol
Tenant must supply default TLS container for front-end offloading. Not supplying a container is an invalid configuration.
TLS supported protocols and cipher suites for termination will be set to sane values by each back-end’s specific code
SNI certificates list is optional and not mandatory to specify
SNI certificates list specifying is described in “SNI certificates list management” section
Certificate intermediate chain will be stored as a part of Barbican’s TLS container
Back-end re-encryption will not be supported in first phase
Front-end client authentication and back-end server authentication will not be supported in first phase
When updating listener with TERMINATED_HTTPS protocol:
In TLS configuration domain, default TLS container ID for front-end offloading and SNI container IDs list are values that may be changed
In case when defaut TLS container ID is replaced for the listener, back-end implementation should ensure a lack of a downtime on LB appliance.
Same for changing SNI container IDs list, back-end should avoid LB downtime.
HA-Proxy LBaaS implementaion and other LBaaS implementations should be modified to support this specification.
With stated above, following is a description of a basic tenant use case - creating listener with TLS offloading:
Tenant cteates Barbican TLS container with a certificate.
Tenant creates listener with TERMINATED_HTTPS as a listener protocol and specifies the Barbican TLS container ID as a default TLS container for front-end offloading
As a result, listener created, offloading encrypted traffic on front-end with default tenant’s TLS certificate, not re-encrypting traffic to the back-end.
Requirements from Barbican¶
Tenant should be able to create and delete TLS containers using Barbican
Ability to store TLS certificates in Barbican containers that contain the TLS certificate itself, its private key and optionaly, intermediate chain
Creating TLS container with:
Certificate : PEM text field
Private_key: PEM text_field
(extracted) Private_key_pass_phrase : text field
Intermediates: PEM text field (optional) This field is a concatination of PEM encoded certificate blocks in specific order
Delete TLS certificate optional: Check if certificate is in use by any consumer and warn before deleting. Barbican’s BP discussing this feature: https://review.openstack.org/#/c/99516/
Get TLS container, including private key in PEM encoded PKCS1 or PKCS7 formats, by container id
Get TLS certificate in pem encoded x509 format, by container id
Restrictions¶
TLS settings are only available for listeners having TERMINATED_HTTPS as a protocol. In other cases TLS settings will be disabled and have None or empty values. There should be a meaningfull error message to a user explaining the exact reason of a failure in case of an invalid configuration.
Listener protocol is immutable. Changing the protocol will require radical re-configuration of provider’s back-end system, which seems to be not justified for this use case. Tenant should create new listener.
While updating existing TLS certificate, name and description are only values allowed to be modified. Creating new TLS container and using it instead of the old one will be easier option than re-configuring LBaaS back-end with modified container, at least in first phase.
SNI certificates list management¶
For SNI functionality, tenant will supply list of TLS containers in specific order. In case when specific back-end is not able to support SNI capabilities, its driver should throw an exception. The exception message should state that this specific back-end (provider) does not support SNI capability. The clear sign of listener’s requirement for SNI capability is a none empty SNI container ids list. However, reference implementation must support SNI capability.
New separate module will be developed in Neutron LBaaS for Barbican TLS containers interactions. The module will use service account for Barbican API interation. The module will have API for: * Ensuring Barbican TLS container existence (used by LBaaS front-end API)
Validating Barbican TLS container (used by LBaaS front-end API) This API will also “register” LBaaS as a container’s consumer in Barbican’s repository.
Extracting SubjectCommonName and SubjectAltName information from certificates’ X509 (used by LBaaS front-end API) As for now, only dNSName and directoryName types will be extracted from SubjectAltName sequence, while directoryName type usage is an issue for further discussion.
Extracting certificate’s data from Barbican TLS container (used by provider/driver code)
Unregistering LBaaS as a consumer of the container when container is not used by any listener any more (used by LBaaS front-end API)
The module will use pyOpenSSL and PyASN1 packages. Only this new common module should be used by Neutron LBaaS code for Barbican containers interactions.
Front-end LBaaS API (plugin) code will use a new developed module for validating Barbican TLS containers. Driver, in its turn, can extract SubjectCommonName and SubjectAltName information from certificates’ X509 via the common module API and use it for its specific SNI implementation.
Note:
Specific back-end driver does not have to use SubjectAltName information. Furthermore, specific driver may throw an exception saying SubjectAltName is not supported by its provider
Any specific driver implementation may extract host names info from certificates using the mentioned above common module API only, if needed.
SNI conflicts
Employing the order of certificates list is not a common requirement for all back-end implementations. The order of SNI containers list may be used by specific back-end code, like Radware’s, for specifying priorities among certificates. Order is meant to be a hint to resolve conflicts when 2 or more certificates match the DNS name requested in the SNI client hello. Specific backends might choose to ignore this order and might employ their own mechanisms to choose one among the clashing certificates. For ex. NetScaler employs the best match algorithm and does not require order for conflict resolution. It’s also possible that specific driver throws an exception saying there is a collision and this specific SNI setup will not be supported by the back-end.
Data Model Impact¶
Data model changes
lbaas_listeners table will be modified with new
default_tls_container_id (nullable string 36) - Barbican’s TLS container id
New lbaas_sni table will be created for storing ordered list of TLS containers associated to a listener for SNI capabilities. Association objects is composed of:
id (immutable string 36) - generated object id
listener_id (string 36) - associated listener id
tls_container_id (string 36) - associated Barbican TLS container id
position - (integer) index for preserving the order
Required database migration
add new columns to lbaas_isteners table
create new lbaas_sni table
New data initial set
New columns for lbaas_listeners table’s existing entries will be set to defaults
REST API Impact¶
Listener Attributes
Attribute Name |
Type |
Access |
Default Value |
Validation/ Conversion |
Description |
|---|---|---|---|---|---|
default-tls- container-id |
UUID |
RW,tenant |
NULL |
UUID |
default TLS cert id to use for offloading |
sni_container _ids |
UUID list |
RW,tenant |
NULL |
UUID list |
ordered list of TLS containers to use for SNI . |
Functions
create_listener
Creates new listener
Request POST /v2.0/lbaas/listeners Accept: application/json { “listener”:{ <…usual listener parameters>, “protocol”: “TERMINATED_HTTPS” “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4j”, “sni_container_ids”: None } }
Response { “listener”:{ “id”: “8604a0de-7f6b-409a-a47c-a1cc7bc77b2e” <…usual listener parameters>, “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4c”, “sni_container_ids”: None “tenant_id”:”6b96ff0cb17a4b859e1e575d221683d3” } }
create_listener (with SNI list)
Creates new listener
Request POST /v2.0/lbaas/listeners Accept: application/json { “listener”:{ <…usual listener parameters>, “protocol”: “TERMINATED_HTTPS” “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4j”, “sni_container_ids”: [5404a0de-7f6b-409a-a47c-a1ccgbc77b3j, 1206a0de-7f6b-409a-a47c-a1ccgbc7bgf3] } }
Response { “listener”:{ “id”: “8604a0de-7f6b-409a-a47c-a1cc7bc77b2e” <…usual listener parameters>, “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4c”, “sni_container_ids”:[5404a0de-7f6b-409a-a47c-a1ccgbc77b3j, 1206a0de-7f6b-409a-a47c-a1ccgbc7bgf3] “tenant_id”:”6b96ff0cb17a4b859e1e575d221683d3” } }
update_listener
Updates VIP listener
Request PUT /v2.0/lbaas/listeners/<listener-id> Accept: application/json { “listener”:{ <…usual listener parameters>, “protocol”: “TERMINATED_HTTPS” “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4c”, “sni_container_ids”: None } }
Response { “listener”:{ “id”: “8604a0de-7f6b-409a-a47c-a1cc7bc77b2e” <…usual listener parameters>, “default_tls_container_id”: “7804a0de-7f6b-409a-a47c-a1cc7bc77b4c”, “sni_container_ids”: None “tenant_id”:”6b96ff0cb17a4b859e1e575d221683d3” } }
Security Impact¶
Following are security requirements:
Retrieving TLS container from Barbican to LBaaS plugin/driver must be secured
Sending TLS container contents from driver to back-end system must be secured
Storing secrets on neutron server is prohibited
Back-end systems may need to ensure secured store for secrets to meet certain security compliance requirements
Notifications Impact¶
None
CLI Impact¶
Listener creation with TERMINATED_HTTPS protocol (default behavior) lb-listener-create –protocol TERMINATED_HTTPS –protocol-port 443 –default_tls_container_id 9a96ff0cb17a4b859e1e575d2216cd23 …<usual CLI options>
Listener creation with TERMINATED_HTTPS protocol and SNI certificates list lb-listener-create –protocol TERMINATED_HTTPS –protocol-port 443 –default_tls_container_id 9a96ff0cb17a4b859e1e575d2216cd23 –sni_container_ids list=true 6b96ff0cb17a4b859e1e575d221683d3 4596ff0cb17a4b859e1e575d22168ba1 …<usual CLI options>
Other End User Impact¶
None
Performance Impact¶
When updating listener without modifying TLS settings (default container id or SNI list) - Barbican API should not be used for retrieving container content which was not actually changed. This will prevent unnecessary resources consumption when, for example, members are added to the pool used by listener. It means that each Barbican TLS container will be validated only once for a listener while it’s still in use by this listener.
IPv6 Impact¶
None
Other Deployer Impact¶
Barbican is required to be deployed and functional in order this feature to work.
New dependencies are added for neutron, pyOpenSSL and PyASN1. These are required by new module for Barbican TLS containers interactions.
Developer Impact¶
None
Community Impact¶
This change has been in review since Juno. Much discussion has taken place over IRC and the mailing list.
Alternatives¶
None
Implementation¶
Assignee(s)¶
Primary assignee:
- Other contributors:
Barbican TLS containers interactions module - https://launchpad.net/~carlos-garza
Work Items¶
Develop new module for BArbican TLS containers interactions using pyOpenSSL and PyASN1 packages.
Implement changes in LBaaS DB schema v2
Implement changes in LBaaS extension v2
Implement all required CLI changes
Implement all required unit testing
Implement all required tempest testing
Make integration with Barbican certificates storage API Detailed specificatio of how Barbican’s API for containers should be used is at https://review.openstack.org/#/c/99516
Modifying LBaaS HA-Proxy driver to support TLS capability Detailed specification of this work item is at https://review.openstack.org/#/c/100931
Use HA-Proxy version 1.5
Implement horizon part of this spec, not as part of Juno release.
Dependencies¶
Barbican API requirements
Neutron LBaaS API v2 with new listener object implemented
New dependencies will be added for neutron, pyOpenSSL and PyASN1. These are required by new module for Barbican TLS containers interactions.
Testing¶
Tempest Tests¶
listener unit testing domains
REST API and attributes validation tests
DB mixin and schema tests
LBaaS Plugin with mocked driver end-to-end tests
Specific driver tests for each existing driver supporting TLS offloading
Tempest tests
CLI tests
New listener creation with TERMINATED_HTTPS as a protocol
No default TLS container for termination supplied. Check error generation
Default TLS container for termination supplied. Test expected default configuration took place.
Default TLS container supplied. SNI TLS containers list was supplied Test expected configuration took place
Update existing listener with TERMINATED_HTTPS as a protocole
Change default TLS container. Test expected configuration
Add/Modify SNI containers list. Test expected configuration
CLI tests should test inconsistency issues such as:
No default offloading TLS container specified when creating listener with TERMINATES_HTTPS protocole
Functional Tests¶
As above.
API Tests¶
As above.
Documentation Impact¶
User Documentation¶
Neutron CLI should be modified with updated listener commands with TLS options
Developer Documentation¶
Neutron API should be modified with new listener TLS attributes
