Port Forwarding API

https://blueprints.launchpad.net/neutron/+spec/port-forwarding

Port forwarding is a common feature in networking and more specifically in PaaS and SaaS cloud systems which aim at reusing the same public IP for different clients that use different VMs for their services.

This is especially relevant for deployments which lack a large number of public IPs they can assign.

Common use case for this feature is a client requesting a specific service, where the serving platform (PaaS, SaaS) allocates a VM to run the service and then allocates a client port to access this service. This means that various clients use the same public IP, but the TCP/UDP destination port is used to distinguish between the end point VMs.

Example, a mapping for web servers:

• client1 172.24.4.2:4001 TCP => maps to 10.0.0.2 port 80 TCP (VM1)

• client2 172.24.4.2:4002 TCP => maps to 10.0.0.3 port 80 TCP (VM2)

This spec will focus on port forwarding based on Floating IPs. A future spec will be submitted for port forwarding based on routers external gateway interface.

Problem Description

• In environments constrained with limited IPs, operators would like to reuse public IPs instead of assigning to each VM its own public IP (Floating IP).

• Docker supports a port-mapping feature and hence a big eco-system of automation orchestration and management plugins leverage it. We would like to make Neutron compatible for these tools and systems and provide a similar API [1].

Proposed Change

Introduce port forwarding API and implementation to Floating IPs.

The user can define various port forwarding rules on the Floating IPs containing the internal/client port and the external/destination port, connected with the VM they want to expose. And users would be allowed to create port forwarding rules only on a “free” Floating IP, i.e. a Floating IP which is not directly associated with a Fixed IP of a tenant’s VM.

We will have four deployment variants:

a) Legacy Router: For the generic Router deployment, the port forwarding rules would be installed in the Router namespace on the network node and forwarding functions would be performed.

b) HA Router: For the HA Router deployment, the port forwarding rule will be installed in both the ACTIVE and the BACKUP Router namespaces on the network nodes they are located on.

c) DVR: As [2] has been merged, we now have the ability to create centralized Floating IPs in a DVR supported deployment. This helps in mapping the Compute nodes where the destination VM is present with the centralized FIP for Port forwarding. This mechanism will centralize a floating IP not only when it is associated with a port bound to a host with the dvr_no_external option enabled, but also when port forwarding attributes are added to it.

d) DVR + HA: If the created router is not an HA router, then we can proceed with option (c). While if the router is an HA router, then we will use the centralized HA router to install the port forwarding rules.

In all deployment variants, the port forwarding entry NATs a specific Floating IP:Port and protocol to a specific Neutron port (and a private IP that is attached to this port). That means it will maintain a mapping like “FIP:extport protocol” to “Neutron Port Fixed IP:intport protocol”. So if a Neutron Port contains multiple Fixed IPs, then it will be allowed to create multiple port forwarding entries for a particular Neutron port, with different external ports, such as:

• FIPX:EXTPORTX PROTOCOLA => Neutron PortQ Fixed IPA:INTPORTA PROTOCOLA (VM1)

• FIPX:EXTPORTY PROTOCOLA => Neutron PortQ Fixed IPB:INTPORTA PROTOCOLA (VM1)

However, the same Fixed IP:intport socket cannot be mapped with different protocols.

If the Neutron port is deleted, the port forwarding entries that match this port are also deleted. Same is applicable in case the Floating IP is deleted.

Data Model Impact

The following new table is added as part of the port forwarding feature:

CREATE TABLE port_forwarding (
    id CHAR(36) NOT NULL PRI KEY,
    floating_ip_id VARCHAR(36) NOT NULL,
    external_port INT NOT NULL,
    internal_neutron_port_id VARCHAR(36) NOT NULL FOREIGN KEY,
    protocol CHAR(4) NOT NULL,
    socket VARCHAR(20) NOT NULL,
    CONSTRAINT floating_ip_id_external_port_constraint UNIQUE (floating_ip_id, external_port),
    CONSTRAINT internal_neutron_port_id_socket_constraint UNIQUE (
        internal_neutron_port_id, socket)
);

The socket column will store the string like ‘Fixed IP:Port’.

Note

This table lacks project_id, as the owner of this port_forwarding must be the owner of associated Floating IP. So there is a project_id check for preventing association of Floating IP to internal Neutron Port if their project_id are different. Also, allow the association that Floating IP/internal Neutron Port exists on a shared network for admin users in different project_id cases, such as FloatingIP from a shared public network created by a admin user and a Neutron Port from a particular internal tenant network created by the tenant user, then admin user want a association of them which have different project_ids. For general users, there is only the same project_id case.

Sub Resource Extension

Neutron floatingips will be extended with a sub resource port_forwarding, it will contain some fields to expose the port_forwarding assigned to the floatingip resource.

For this new feature, a new service plugin will be introduced, and the following methods will be added:

• ‘create_floatingip_port_forwarding()’

• ‘delete_floatingip_port_forwarding()’

• ‘get_floatingip_port_forwarding()’

• ‘get_floatingip_port_forwardings()’

For update operation, we will extend the function in the future if possible. But for now, we will just support create/delete/get functions.

So the attributes map of new sub resource would be like:

SUB_RESOURCE_ATTRIBUTE_MAP = {
    'port_forwarding': {
        'parent': {'collection_name': 'floatingips',
                    'member_name': 'floatingip'},
        'parameters': {
            'external_port': {'allow_post': True, 'allow_put': False,
                                 'convert_to':
                                     convert_validate_port_value,
                                 'is_visible': True},
            'internal_port': {'allow_post': True, 'allow_put': False,
                                 'convert_to':
                                     convert_validate_port_value,
                                 'is_visible': True},
            'internal_ip_address': {'allow_post': True,
                                        'allow_put': False,
                                        'validate': {
                                        'type:ip_address_or_none': None},
                                        'is_visible': True},
            'protocol': {'allow_post': True, 'allow_put': False,
                           'validate': {
                              'type:values': constants.IPTABLES_PROTOCOL_MAP.keys()},
                           'is_visible': True,
                           'convert_to': converters.convert_to_protocol},
            'internal_port_id': {'allow_post': True,
                                    'allow_put': False,
                                    'validate': {'type:int':None},
                                    'is_visible': True},
        }
    }
}

REST API Impact

The idea is to extend the Floating IP Rest API with a new extension floating_ip_port_forwarding with the below defined attributes.

Floating IP extension

Attribute Name	Type	CRUD	Default Value	Description
port_forwardings	List	R	None	The associated ‘port-forwarding’ sub resource with the particular Floating IP resource.

The Floating IP extension definition would be expanded as :

RESOURCE_ATTRIBUTE_MAP = {
     'floatingips': {
         'port_forwardings': {'allow_post': False,
                                 'allow_put': False,
                                 'is_visible': True, 'default': None}
     }
}

This new field will be exposed in the response during GET/POST/PUT requests of Floating IP resource. That means users can not change the forwarding resources through CRU FloatingIP, only can create the forwardings one by one with the new port forwarding API which will be introduced below.

For example, GET a Floating IP:

GET /v2.0/floatingips/<floatingip-uuid>

{
    "floatingip": {
         "floating_network_id": "376da547-b977-4cfe-9cba-275c80debf57",
         "router_id": "d23abc8d-2991-4a55-ba98-2aaea84cc72f",
         "fixed_ip_address": "",
         "floating_ip_address": "172.24.4.228",
         "project_id": "4969c491a3c74ee4af974e6d800c62de",
         "tenant_id": "4969c491a3c74ee4af974e6d800c62de",
         "status": "ACTIVE",
         "port_id": "",
         "id": "2f245a7b-796b-4f26-9cf9-9e82d248fda7",
         "port_forwardings": [
             {
                 "internal_ip_address": "10.0.0.3",
                 "protocol": "tcp",
                 "internal_port": "22",
                 "external_port": "7001"
             },
             {
                 "internal_ip_address": "192.168.4.32",
                 "protocol": "tcp",
                 "internal_port": "22",
                 "external_port": "7002"
             }
         ]
    }
}

For the new sub resource ‘port_forwarding’, a new url will be introduced:

• /v2.0/floatingips/<floatingip-uuid>/port_forwardings

List Port Forwardings

GET /v2.0/floatingips/<floatingip-uuid>/port_forwardings

{
    "port_forwardings": [
         {
             "id": "ae34051f-aa6c-4c75-abf5-50dc9ac99ef3",
             "external_port": "7003",
             "internal_port": "22",
             "internal_ip_address": "10.0.0.10",
             "protocol": "tcp",
             "internal_port_id": "b930d7f6-ceb7-40a0-8b81-a425dd994ccf"
         },
         {
             "id": "915a14a6-867b-4af7-83d1-70efceb146f9",
             "external_port": "7004",
             "internal_port": "22",
             "internal_ip_address": "10.0.0.11",
             "protocol": "tcp",
             "internal_port_id": "0c56df5d-ace5-46c8-8f4c-45fa4e334d18"
         }
    ]
}

Response Parameters

Parameter	Style	Type	Description
port_forwardings	plain	xsd:list	A list of port_forwarding objects

More parameters see Show Port Forwarding

Show Port Forwarding

GET /v2.0/floatingips/<floatingip-uuid>/port_forwardings/<port-forwarding-id>

{
    "port_forwarding": {
         "id": "ae34051f-aa6c-4c75-abf5-50dc9ac99ef3",
         "external_port": "7003",
         "internal_port": "22",
         "internal_ip_address": "10.0.0.10",
         "protocol": "tcp",
         "internal_port_id": "b930d7f6-ceb7-40a0-8b81-a425dd994ccf"
     }
}

Response Parameters

Parameter	Style	Type	Description
port_forwarding	plain	xsd:dict	A port_forwarding object
id	plain	xsd:string	The ID of port_forwarding object
external_port	plain	xsd:string	The exposed external protocol port number
internal_port	plain	xsd:string	The port forwarding mapped internal protocol port number
internal_ip_address	plain	xsd:string	The IP Address from the fixed ips of a particular port
protocol	plain	xsd:string	The traffic protocol type, such as TCP or UDP. Default value is ‘TCP’.
internal_port_id	plain	xsd:string	The Neutron internal Port ID.

Create Port Forwarding

POST /v2.0/floatingips/<floatingip-uuid>/port_forwardings

{
    "port_forwarding": {
         "external_port": "7233",
         "internal_port": "22",
         "internal_port_id": "b930d7f6-ceb7-40a0-8b81-a425dd994ccf"
     }
}

Response parameters

see Show Port Forwarding

{
    "port_forwarding": {
         "id": "f8a44de0-fc8e-45df-93c7-f79bf3b01c95",
         "external_port": "7233",
         "internal_port": "22",
         "internal_ip_address": "192.168.43.33",
         "protocol": "tcp",
         "internal_port_id": "b930d7f6-ceb7-40a0-8b81-a425dd994ccf"
     }
}

Delete Port Forwarding

DELETE /v2.0/floatingips/<floatingip-uuid>/port_forwardings/<port-forwarding-id>

This operation does not accept a request body and does not return a response body.

Effects on Existing Floating IP APIs

Slight adjustment to existing Floating IP APIs:

• Create a Floating IP just the same with current behavior.

• Associate a Neutron internal port with a Floating IP, if the requested url is the same as previous, the Floating IP will work as 1:1 DNAT like current behavior. If the request with the new url, that means the request needs a port-forwarding function towards the Floating IP.

• Get a Floating IP resource will be the same as before if the Floating IP resource had already associated with a neutron port for 1:1 DNAT. If a Floating IP resource contains more than 1 port-forwarding sub resource, it is better to show the port-forwardings summary in the Floating IP response body to distinguish which Floating IP resource is available for different requirements, such as 1:1 DNAT, port-forwarding.

Command Line Client Impact

Openstack Client would have additional options portforwarding for Floating IP CLI, which would define the port-forwarding characteristics.

Security Impact

Port forwarding is similar in nature to centralized DNAT, so should not pose additional security implications. But if users actually want to use Port forwarding, they must make sure to allow the associated ingress Security Group towards the internal ip which is used by the neutron port of their VMs.

Notifications Impact

Depends on the implementation spec

Other End User Impact

None

Performance Impact

Performance testing must be conducted to see what is the overhead of enabling this feature, of course that if the feature is disabled no performance impact should be noticed.

IPv6 Impact

IPv6 is not supported

Other Deployer Impact

Deployer will be able to leverage port forwarding for a unique way to reach a private VM/Container without wasting new public IPs

Developer Impact

Future SNAT distribution plans should take port forwarding into consideration. Kuryr can leverage port forwarding for feature compatibility with Docker port mapping.

Alternatives

Users can use an external VM that provide this NAT capability or assign a new Floating IP for each VM.

Implementation

Assignee(s)

Primary assignees:

reedip <reedip.banerjee@nectechnologies.in>

Other contributors:

gal-sagie <gal.sagie@gmail.com>

tian-mingming <tian.mingming@h3c.com>

zhaobo <zhaobo6@huawei.com>

Work Items

1. API Implementation

2. DB Implementation

3. Reference implementation

4. Tests

5. Documentation

Dependencies

None

Testing

Tempest Tests

Need to add tempest tests

Functional Tests

Need to add functional tests

API Tests

Need to add API tests

Fullstack Tests

Need to add Fullstack tests.

Documentation Impact

User Documentation

Needs user documentation

Developer Documentation

Needs devref documentation

References

[1]

https://docs.docker.com/engine/userguide/networking/default_network/binding/

[2]

https://review.openstack.org/#/c/485333/20

[3]

https://ask.openstack.org/en/question/75190/neutron-port-forwarding-qrouter-vms/

[4]

http://www.gossamer-threads.com/lists/openstack/dev/34307

[5]

http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-for-router-td46639.html

[6]

http://openstack.10931.n7.nabble.com/Neutron-port-forwarding-from-gateway-to-internal-hosts-td32410.html

[7]

https://blueprints.launchpad.net/neutron/+spec/neutron-ovs-dvr