Networking Guide

Create a network to give your instances a place to communicate with each other and receive IP addresses using DHCP. A network can also be integrated with external networks in your Red Hat OpenStack Platform deployment or elsewhere, such as the physical network. This integration allows your instances to communicate with outside systems. For more information, see Bridge the physical network.

When creating networks, it is important to know that networks can host multiple subnets. This is useful if you intend to host distinctly different systems in the same network, and would prefer a measure of isolation between them. For example, you can designate that only webserver traffic is present on one subnet, while database traffic traverse another. Subnets are isolated from each other, and any instance that wishes to communicate with another subnet must have their traffic directed by a router. Consider placing systems that will require a high volume of traffic amongst themselves in the same subnet, so that they don’t require routing, and avoid the subsequent latency and load.

1. In the dashboard, select Project > Network > Networks.

2. Click +Create Network and specify the following:

3. Click the Next button, and specify the following in the Subnet tab:

4. Click Next to specify DHCP options:

5. Click Create.

The completed network is available for viewing in the Networks tab. You can also click Edit to change any options as needed. Now when you create instances, you can configure them now to use its subnet, and they will subsequently receive any specified DHCP options.

Advanced network options are available for administrators, when creating a network from the Admin view. These options define the network type to use, and allow tenants to be specified:

1. In the dashboard, select Admin > Networks > Create Network > Project. Select a destination project to host the new network using Project.

2. Review the options in Provider Network Type:

Click Create Network, and review the Project’s Network Topology to validate that the network has been successfully created.

To allow traffic to be routed to and from your new network, you must add its subnet as an interface to an existing virtual router:

1. In the dashboard, select Project > Network > Routers.

2. Click on your virtual router’s name in the Routers list, and click +Add Interface. In the Subnet list, select the name of your new subnet. You can optionally specify an IP address for the interface in this field.

3. Click Add Interface.

Instances on your network are now able to communicate with systems outside the subnet.

There are occasions where it becomes necessary to delete a network that was previously created, perhaps as housekeeping or as part of a decommissioning process. In order to successfully delete a network, you must first remove or detach any interfaces where it is still in use. The following procedure provides the steps for deleting a network in your project, together with any dependent interfaces.

1. In the dashboard, select Project > Network > Networks. Remove all router interfaces associated with the target network’s subnets. To remove an interface: Find the ID number of the network you would like to delete by clicking on your target network in the Networks list, and looking at the its ID field. All the network’s associated subnets will share this value in their Network ID field.

2. Select Project > Network > Routers, click on your virtual router’s name in the Routers list, and locate the interface attached to the subnet you would like to delete. You can distinguish it from the others by the IP address that would have served as the gateway IP. In addition, you can further validate the distinction by ensuring that the interface’s network ID matches the ID you noted in the previous step.

3. Click the interface’s Delete Interface button.

Select Project > Network > Networks, and click the name of your network. Click the target subnet’s Delete Subnet button.

4. Select Project > Network > Networks, and select the network you would like to delete.

5. Click Delete Networks.

In a previous release, after deleting a project, you might have noticed the presence of stale resources that were once allocated to the project. This included networks, routers, and ports. Previously, these stale resources had to be been manually deleted, while also being mindful of deleting them in the correct order. This has been addressed in Red Hat OpenStack Platform 12, where you can instead use the neutron purge command to delete all the neutron resources that once belonged to a particular project.

For example, to purge the neutron resources of test-project prior to deletion:

# openstack project list
+----------------------------------+--------------+
| ID                               | Name         |
+----------------------------------+--------------+
| 02e501908c5b438dbc73536c10c9aac0 | test-project |
| 519e6344f82e4c079c8e2eabb690023b | services     |
| 80bf5732752a41128e612fe615c886c6 | demo         |
| 98a2f53c20ce4d50a40dac4a38016c69 | admin        |
+----------------------------------+--------------+

# neutron purge 02e501908c5b438dbc73536c10c9aac0
Purging resources: 100% complete.
Deleted 1 security_group, 1 router, 1 port, 1 network.

# openstack project delete 02e501908c5b438dbc73536c10c9aac0

Subnets are the means by which instances are granted network connectivity. Each instance is assigned to a subnet as part of the instance creation process, therefore it’s important to consider proper placement of instances to best accommodate their connectivity requirements. Subnets are created in pre-existing networks. Remember that tenant networks in OpenStack Networking can host multiple subnets. This is useful if you intend to host distinctly different systems in the same network, and would prefer a measure of isolation between them. For example, you can designate that only webserver traffic is present on one subnet, while database traffic traverse another. Subnets are isolated from each other, and any instance that wishes to communicate with another subnet must have their traffic directed by a router. Consider placing systems that will require a high volume of traffic amongst themselves in the same subnet, so that they don’t require routing, and avoid the subsequent latency and load.

You can delete a subnet if it is no longer in use. However, if any instances are still configured to use the subnet, the deletion attempt fails and the dashboard displays an error message. This procedure demonstrates how to delete a specific subnet in a network:

In the dashboard, select Project > Network > Networks, and click the name of your network. Select the target subnet and click Delete Subnets.

OpenStack Networking provides routing services using an SDN-based virtual router. Routers are a requirement for your instances to communicate with external subnets, including those out in the physical network. Routers and subnets connect using interfaces, with each subnet requiring its own interface to the router. A router’s default gateway defines the next hop for any traffic received by the router. Its network is typically configured to route traffic to the external physical network using a virtual bridge.

1. In the dashboard, select Project > Network > Routers, and click +Create Router.

2. Enter a descriptive name for the new router, and click Create router.

3. Click Set Gateway next to the new router’s entry in the Routers list.

4. In the External Network list, specify the network that will receive traffic destined for an external location.

5. Click Set Gateway. After adding a router, the next step is to configure any subnets you have created to send traffic using this router. You do this by creating interfaces between the subnet and the router.

You can delete a router if it has no connected interfaces. This procedure describes the steps needed to first remove a router’s interfaces, and then the router itself.

1. In the dashboard, select Project > Network > Routers, and click on the name of the router you would like to delete.

2. Select the interfaces of type Internal Interface. Click Delete Interfaces.

3. From the Routers list, select the target router and click Delete Routers.

Interfaces allow you to interconnect routers with subnets. As a result, the router can direct any traffic that instances send to destinations outside of their intermediate subnet. This procedure adds a router interface and connects it to a subnet. The procedure uses the Network Topology feature, which displays a graphical representation of all your virtual router and networks and enables you to perform network management tasks.

1. In the dashboard, select Project > Network > Network Topology.

2. Locate the router you wish to manage, hover your mouse over it, and click Add Interface.

3. Specify the Subnet to which you would like to connect the router. You have the option of specifying an IP Address. The address is useful for testing and troubleshooting purposes, since a successful ping to this interface indicates that the traffic is routing as expected.

4. Click Add interface.

The Network Topology diagram automatically updates to reflect the new interface connection between the router and subnet.

You can remove an interface to a subnet if you no longer require the router to direct its traffic. This procedure demonstrates the steps required for deleting an interface:

1. In the dashboard, select Project > Network > Routers.

2. Click on the name of the router that hosts the interface you would like to delete.

3. Select the interface (will be of type Internal Interface), and click Delete Interfaces.

You can use procedures in this section to manage your IP address allocation in OpenStack Networking.

# neutron subnet-create --name SUBNET_NAME --enable_dhcp=False --allocation_pool start=IP_ADDRESS,end=IP_ADDRESS --gateway=IP_ADDRESS NETWORK_NAME CIDR

# neutron subnet-create --name public_subnet --enable_dhcp=False --allocation_pool start=192.168.100.20,end=192.168.100.100 --gateway=192.168.100.1 public 192.168.100.0/24

# nova floating-ip-associate INSTANCE_NAME IP_ADDRESS

# nova floating-ip-associate corp-vm-01 192.168.100.20

# neutron floatingip-create public
+---------------------+--------------------------------------+
| Field               | Value                                |
+---------------------+--------------------------------------+
| fixed_ip_address    |                                      |
| floating_ip_address | 192.168.100.20                       |
| floating_network_id | 7a03e6bc-234d-402b-9fb2-0af06c85a8a3 |
| id                  | 9d7e2603482d                         |
| port_id             |                                      |
| router_id           |                                      |
| status              | ACTIVE                               |
| tenant_id           | 9e67d44eab334f07bf82fa1b17d824b6     |
+---------------------+--------------------------------------+

# neutron port-list
+--------+------+-------------+--------------------------------------------------------+
| id     | name | mac_address | fixed_ips                                              |
+--------+------+-------------+--------------------------------------------------------+
| ce8320 |      | 3e:37:09:4b | {"subnet_id": "361f27", "ip_address": "192.168.100.2"} |
| d88926 |      | 3e:1d:ea:31 | {"subnet_id": "361f27", "ip_address": "192.168.100.5"} |
| 8190ab |      | 3e:a3:3d:2f | {"subnet_id": "b74dbb", "ip_address": "10.10.1.25"}|
+--------+------+-------------+--------------------------------------------------------+

# neutron floatingip-associate 9d7e2603482d 8190ab

OpenStack Networking supports one floating IP pool per L3 agent. Therefore, scaling out your L3 agents allows you to create additional floating IP pools.

The procedure below enables you to bridge your virtual network to the physical network to enable connectivity to and from virtual instances. In this procedure, the example physical eth0 interface is mapped to the br-ex bridge; the virtual bridge acts as the intermediary between the physical network and any virtual networks. As a result, all traffic traversing eth0 uses the configured Open vSwitch to reach instances. Map a physical NIC to the virtual Open vSwitch bridge (for more information, see Chapter 12, Configure Bridge Mappings):

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex
ONBOOT=yes

Configure the virtual bridge with the IP address details that were previously allocated to eth0:

# vi /etc/sysconfig/network-scripts/ifcfg-br-ex
DEVICE=br-ex
DEVICETYPE=ovs
TYPE=OVSBridge
BOOTPROTO=static
IPADDR=192.168.120.10
NETMASK=255.255.255.0
GATEWAY=192.168.120.1
DNS1=192.168.120.1
ONBOOT=yes

You can now assign floating IP addresses to instances and make them available to the physical network.

When planning your OpenStack deployment, you might begin with a number of these subnets, from which you would be expected to allocate how the individual addresses will be used. Having multiple subnets allows you to segregate traffic between systems into VLANs. For example, you would not generally want management or API traffic to share the same network as systems serving web traffic. Traffic between VLANs will also need to traverse through a router, which represents an opportunity to have firewalls in place to further govern traffic flow.

You would typically allocate separate VLANs for the different types of network traffic you will host. For example, you could have separate VLANs for each of these types of networks. Of these, only the External network needs to be routable to the external physical network. In this release, DHCP services are provided by the director.

The following systems will consume IP addresses from your allocated range:

These virtual resources consume IP addresses in OpenStack Networking. These are considered local to the cloud infrastructure, and do not need to be reachable by systems in the external physical network:

This example shows a number of networks that accommodate multiple subnets, with each subnet being assigned a range of IP addresses:

This procedure lists all the ports attached to a particular router, then demonstrates how to retrieve a port’s state (DOWN or ACTIVE).

1. View all the ports attached to the router named r1:

#  neutron router-port-list r1

Example result:

+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| id                                   | name | mac_address       | fixed_ips                                                                            |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
| b58d26f0-cc03-43c1-ab23-ccdb1018252a |      | fa:16:3e:94:a7:df | {"subnet_id": "a592fdba-babd-48e0-96e8-2dd9117614d3", "ip_address": "192.168.200.1"} |
| c45e998d-98a1-4b23-bb41-5d24797a12a4 |      | fa:16:3e:ee:6a:f7 | {"subnet_id": "43f8f625-c773-4f18-a691-fd4ebfb3be54", "ip_address": "172.24.4.225"}  |
+--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+

2. View the details of each port by running this command against its ID (the value in the left column). The result includes the port’s status, indicated in the following example as having an ACTIVE state:

# neutron port-show b58d26f0-cc03-43c1-ab23-ccdb1018252a

Example result:

+-----------------------+--------------------------------------------------------------------------------------+
| Field                 | Value                                                                                |
+-----------------------+--------------------------------------------------------------------------------------+
| admin_state_up        | True                                                                                 |
| allowed_address_pairs |                                                                                      |
| binding:host_id       | node.example.com                                                      |
| binding:profile       | {}                                                                                   |
| binding:vif_details   | {"port_filter": true, "ovs_hybrid_plug": true}                                       |
| binding:vif_type      | ovs                                                                                  |
| binding:vnic_type     | normal                                                                               |
| device_id             | 49c6ebdc-0e62-49ad-a9ca-58cea464472f                                                 |
| device_owner          | network:router_interface                                                             |
| extra_dhcp_opts       |                                                                                      |
| fixed_ips             | {"subnet_id": "a592fdba-babd-48e0-96e8-2dd9117614d3", "ip_address": "192.168.200.1"} |
| id                    | b58d26f0-cc03-43c1-ab23-ccdb1018252a                                                 |
| mac_address           | fa:16:3e:94:a7:df                                                                    |
| name                  |                                                                                      |
| network_id            | 63c24160-47ac-4140-903d-8f9a670b0ca4                                                 |
| security_groups       |                                                                                      |
| status                | ACTIVE                                                                               |
| tenant_id             | d588d1112e0f496fb6cac22f9be45d49                                                     |
+-----------------------+--------------------------------------------------------------------------------------+

Perform this step for each port to retrieve its status.

The ping command is a useful tool for analyzing network connectivity problems. The results serve as a basic indicator of network connectivity, but might not entirely exclude all connectivity issues, such as a firewall blocking the actual application traffic. The ping command works by sending traffic to specified destinations, and then reports back whether the attempts were successful.

Ping tests are most useful when run from the machine experiencing network issues, so it may be necessary to connect to the command line via the VNC management console if the machine seems to be completely offline.

For example, the ping test command below needs to validate multiple layers of network infrastructure in order to succeed; name resolution, IP routing, and network switching will all need to be functioning correctly:

$ ping www.redhat.com

PING e1890.b.akamaiedge.net (125.56.247.214) 56(84) bytes of data.
64 bytes from a125-56.247-214.deploy.akamaitechnologies.com (125.56.247.214): icmp_seq=1 ttl=54 time=13.4 ms
64 bytes from a125-56.247-214.deploy.akamaitechnologies.com (125.56.247.214): icmp_seq=2 ttl=54 time=13.5 ms
64 bytes from a125-56.247-214.deploy.akamaitechnologies.com (125.56.247.214): icmp_seq=3 ttl=54 time=13.4 ms
^C

You can terminate the ping command with Ctrl-c, after which a summary of the results is presented. Zero packet loss indicates that the connection was timeous and stable:

--- e1890.b.akamaiedge.net ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 13.461/13.498/13.541/0.100 ms

In addition, the results of a ping test can be very revealing, depending on which destination gets tested. For example, in the following diagram VM1 is experiencing some form of connectivity issue. The possible destinations are numbered in blue, and the conclusions drawn from a successful or failed result are presented:

1. The internet - a common first step is to send a ping test to an internet location, such as www.redhat.com.

2. Physical router - This is the router interface designated by the network administrator to direct traffic onward to external destinations.

3. Neutron router - This is the virtual SDN (Software-defined Networking) router used by Red Hat OpenStack Platform to direct the traffic of virtual machines.

4. Physical switch - The physical switch’s role is to manage traffic between nodes on the same physical network.

5. VM2 - Attempt to ping a VM on the same subnet, on the same Compute node.

OpenStack Networking is able to trunk VLAN networks through to the SDN switches. Support for VLAN-tagged provider networks means that virtual instances are able to integrate with server subnets in the physical network.

To troubleshoot connectivity to a VLAN Provider network, attempt to ping the gateway IP designated when the network was created. For example, if you created the network with these commands:

# neutron net-create provider --provider:network_type=vlan --provider:physical_network=phy-eno1 --provider:segmentation_id=120
# neutron subnet-create "provider" --allocation-pool start=192.168.120.1,end=192.168.120.253 --disable-dhcp --gateway 192.168.120.254 192.168.120.0/24

Then you’ll want to attempt to ping the defined gateway IP of 192.168.120.254

If that fails, confirm that you have network flow for the associated VLAN (as defined during network creation). In the example above, OpenStack Networking is configured to trunk VLAN 120 to the provider network. This option is set using the parameter --provider:segmentation_id=120.

Confirm the VLAN flow on the bridge interface, in this case it’s named br-ex:

# ovs-ofctl dump-flows br-ex

 NXST_FLOW reply (xid=0x4):
  cookie=0x0, duration=987.521s, table=0, n_packets=67897, n_bytes=14065247, idle_age=0, priority=1 actions=NORMAL
  cookie=0x0, duration=986.979s, table=0, n_packets=8, n_bytes=648, idle_age=977, priority=2,in_port=12 actions=drop

# neutron agent-list
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| id                                   | agent_type         | host                  | alive | admin_state_up |
+--------------------------------------+--------------------+-----------------------+-------+----------------+
| a08397a8-6600-437d-9013-b2c5b3730c0c | Metadata agent     | rhelosp.example.com   | :-)   | True           |
| a5153cd2-5881-4fc8-b0ad-be0c97734e6a | L3 agent           | rhelosp.example.com   | :-)   | True           |
| b54f0be7-c555-43da-ad19-5593a075ddf0 | DHCP agent         | rhelosp.example.com   | :-)   | True           |
| d2be3cb0-4010-4458-b459-c5eb0d4d354b | Open vSwitch agent | rhelosp.example.com   | :-)   | True           |
+--------------------------------------+--------------------+-----------------------+-------+----------------+

In OpenStack Networking, all tenant traffic is contained within network namespaces. This allows tenants to configure networks without interfering with each other. For example, network namespaces allow different tenants to have the same subnet range of 192.168.1.1/24 without resulting in any interference between them.

To begin troubleshooting a tenant network, first determine which network namespace contains the network:

1. List all the tenant networks using the neutron command:

# neutron net-list
+--------------------------------------+-------------+-------------------------------------------------------+
| id                                   | name        | subnets                                               |
+--------------------------------------+-------------+-------------------------------------------------------+
| 9cb32fe0-d7fb-432c-b116-f483c6497b08 | web-servers | 453d6769-fcde-4796-a205-66ee01680bba 192.168.212.0/24 |
| a0cc8cdd-575f-4788-a3e3-5df8c6d0dd81 | private     | c1e58160-707f-44a7-bf94-8694f29e74d3 10.0.0.0/24      |
| baadd774-87e9-4e97-a055-326bb422b29b | private     | 340c58e1-7fe7-4cf2-96a7-96a0a4ff3231 192.168.200.0/24 |
| 24ba3a36-5645-4f46-be47-f6af2a7d8af2 | public      | 35f3d2cb-6e4b-4527-a932-952a395c4bb3 172.24.4.224/28  |
+--------------------------------------+-------------+-------------------------------------------------------+

In this example, we’ll be examining the web-servers network. Make a note of the id value in the web-server row (in this case, its 9cb32fe0-d7fb-432c-b116-f483c6497b08). This value is appended to the network namespace, which will help you identify in the next step.

2. List all the network namespaces using the ip command:

# ip netns list
qdhcp-9cb32fe0-d7fb-432c-b116-f483c6497b08
qrouter-31680a1c-9b3e-4906-bd69-cb39ed5faa01
qrouter-62ed467e-abae-4ab4-87f4-13a9937fbd6b
qdhcp-a0cc8cdd-575f-4788-a3e3-5df8c6d0dd81
qrouter-e9281608-52a6-4576-86a6-92955df46f56

In the result there is a namespace that matches the web-server network id. In this example it’s presented as qdhcp-9cb32fe0-d7fb-432c-b116-f483c6497b08.

3. Examine the configuration of the web-servers network by running commands within the namespace. This is done by prefixing the troubleshooting commands with ip netns exec (namespace). For example:

a) View the routing table of the web-servers network:

# ip netns exec qrouter-62ed467e-abae-4ab4-87f4-13a9937fbd6b route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.24.4.225    0.0.0.0         UG    0      0        0 qg-8d128f89-87
172.24.4.224    0.0.0.0         255.255.255.240 U     0      0        0 qg-8d128f89-87
192.168.200.0   0.0.0.0         255.255.255.0   U     0      0        0 qr-8efd6357-96

b) View the routing table of the web-servers network:

# ip netns exec qrouter-62ed467e-abae-4ab4-87f4-13a9937fbd6b route -n

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.24.4.225    0.0.0.0         UG    0      0        0 qg-8d128f89-87
172.24.4.224    0.0.0.0         255.255.255.240 U     0      0        0 qg-8d128f89-87
192.168.200.0   0.0.0.0         255.255.255.0   U     0      0        0 qr-8efd6357-96

# ip netns exec qrouter-62ed467e-abae-4ab4-87f4-13a9937fbd6b tcpdump -qnntpi any icmp

# ip netns exec qrouter-62ed467e-abae-4ab4-87f4-13a9937fbd6b  ping www.redhat.com

tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
IP (tos 0xc0, ttl 64, id 55447, offset 0, flags [none], proto ICMP (1), length 88)
    172.24.4.228 > 172.24.4.228: ICMP host 192.168.200.20 unreachable, length 68
	IP (tos 0x0, ttl 64, id 22976, offset 0, flags [DF], proto UDP (17), length 60)
    172.24.4.228.40278 > 192.168.200.21: [bad udp cksum 0xfa7b -> 0xe235!] UDP, length 32

This procedure creates flat provider networks that can connect instances directly to external networks. You would do this if you have multiple physical networks (for example, physnet1, physnet2) and separate physical interfaces (eth0 -> physnet1, and eth1 -> physnet2), and you need to connect each Compute node and Network node to those external networks.

type_drivers = vxlan,flat
flat_networks =*

2. Create a flat external network and associate it with the configured physical_network. Creating it as a shared network will allow other users to connect their instances directly to it:

neutron net-create public01 --provider:network_type flat --provider:physical_network physnet1 --router:external=True --shared

3. Create a subnet within this external network using neutron subnet-create, or the OpenStack Dashboard. For example:

neutron subnet-create --name public_subnet --enable_dhcp=False --allocation_pool start=192.168.100.20,end=192.168.100.100 --gateway=192.168.100.1 public01 192.168.100.0/24

4. Restart the neutron-server service to apply this change:

# systemctl restart neutron-server.service

1. Create the Open vSwitch bridge and port. This step creates the external network bridge (br-ex) and adds a corresponding port (eth1):

i. Edit /etc/sysconfig/network-scripts/ifcfg-eth1:

DEVICE=eth1
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none

ii. Edit /etc/sysconfig/network-scripts/ifcfg-br-ex:

DEVICE=br-ex
TYPE=OVSBridge
DEVICETYPE=ovs
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none

2. Restart the network service for the changes to take effect:

# systemctl restart network.service

3. Configure the physical networks in /etc/neutron/plugins/ml2/openvswitch_agent.ini and map the bridge to the physical network:

bridge_mappings = physnet1:br-ex

4. Restart the neutron-openvswitch-agent service on the Network and Compute nodes for the changes to take effect:

systemctl restart neutron-openvswitch-agent

# Name of bridge used for external network traffic. This should be set to
# empty value for the linux bridge
external_network_bridge =

2. Restart neutron-l3-agent for the changes to take effect:

systemctl restart neutron-l3-agent.service

1. Create a new instance.

2. Use the Networking tab in the dashboard to add the new instance directly to the newly-created external network.

qbr269d4d73-e7		8000.061943266ebb	no		qvb269d4d73-e7
							tap269d4d73-e7

  Bridge br-int
        fail_mode: secure
            Interface "qvof63599ba-8f"
        Port "qvo269d4d73-e7"
            tag: 5
            Interface "qvo269d4d73-e7"

    Bridge br-int
        fail_mode: secure
       Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}

    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port br-ex
            Interface br-ex
                type: internal

# ovs-ofctl show br-ex
OFPT_FEATURES_REPLY (xid=0x2): dpid:00003440b5c90dc6
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE

 2(phy-br-ex): addr:ba:b5:7b:ae:5c:a2
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

# ovs-ofctl dump-flows br-ex
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=4703.491s, table=0, n_packets=3620, n_bytes=333744, idle_age=0, priority=1 actions=NORMAL
 cookie=0x0, duration=3890.038s, table=0, n_packets=13, n_bytes=1714, idle_age=3764, priority=4,in_port=2,dl_vlan=5 actions=strip_vlan,NORMAL
 cookie=0x0, duration=4702.644s, table=0, n_packets=10650, n_bytes=447632, idle_age=0, priority=2,in_port=2 actions=drop

ovs-ofctl show br-int
OFPT_FEATURES_REPLY (xid=0x2): dpid:00004e67212f644d
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
 15(int-br-ex): addr:12:4e:44:a9:50:f4
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

# ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=5351.536s, table=0, n_packets=12118, n_bytes=510456, idle_age=0, priority=1 actions=NORMAL
 cookie=0x0, duration=4537.553s, table=0, n_packets=3489, n_bytes=321696, idle_age=0, priority=3,in_port=15,vlan_tci=0x0000 actions=mod_vlan_vid:5,NORMAL
 cookie=0x0, duration=5350.365s, table=0, n_packets=628, n_bytes=57892, idle_age=4538, priority=2,in_port=15 actions=drop
 cookie=0x0, duration=5351.432s, table=23, n_packets=0, n_bytes=0, idle_age=5351, priority=0 actions=drop

# grep bridge_mapping /etc/neutron/plugins/ml2/openvswitch_agent.ini
bridge_mappings = physnet1:br-ex

# neutron net-show provider-flat
...
| provider:physical_network | physnet1
...

# neutron net-show provider-flat
...
| provider:network_type     | flat                                 |
| router:external           | True                                 |
...

    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "eth1"
            Interface "eth1"

# ip a
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000

This procedure creates VLAN provider networks that can connect instances directly to external networks. You would do this if you want to connect multiple VLAN-tagged interfaces (on a single NIC) to multiple provider networks. This example uses a physical network called physnet1, with a range of VLANs (171-172). The network nodes and compute nodes are connected to the physical network using a physical interface on them called eth1. The switch ports to which these interfaces are connected must be configured to trunk the required VLAN ranges.
The following procedures configure the VLAN provider networks using the example VLAN IDs and names given above.

[ml2]
type_drivers = vxlan,flat,vlan

2. Configure the network_vlan_ranges setting to reflect the physical network and VLAN ranges in use. For example:

[ml2_type_vlan]
network_vlan_ranges=physnet1:171:172

3. Restart the neutron-server service to apply the changes:

systemctl restart neutron-server

4. Create the external networks as a vlan-type, and associate them to the configured physical_network. Create it as a --shared network to let other users directly connect instances. This example creates two networks: one for VLAN 171, and another for VLAN 172:

neutron net-create provider-vlan171 \
			--provider:network_type vlan \
			--router:external true \
			--provider:physical_network physnet1 \
			--provider:segmentation_id 171 --shared

neutron net-create provider-vlan172 \
			--provider:network_type vlan \
			--router:external true \
			--provider:physical_network physnet1 \
			--provider:segmentation_id 172 --shared

5. Create a number of subnets and configure them to use the external network. This is done using either neutron subnet-create or the dashboard. You will want to make certain that the external subnet details you have received from your network administrator are correctly associated with each VLAN. In this example, VLAN 171 uses subnet 10.65.217.0/24 and VLAN 172 uses 10.65.218.0/24:

neutron subnet-create \
			--name subnet-provider-171 provider-171 10.65.217.0/24 \
			--enable-dhcp \
			--gateway 10.65.217.254 \

neutron subnet-create \
			--name subnet-provider-172 provider-172 10.65.218.0/24 \
			--enable-dhcp \
			--gateway 10.65.218.254 \

1. Create an external network bridge (br-ex), and associate a port (eth1) with it:

/etc/sysconfig/network-scripts/ifcfg-eth1

DEVICE=eth1
TYPE=OVSPort
DEVICETYPE=ovs
OVS_BRIDGE=br-ex
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none

/etc/sysconfig/network-scripts/ifcfg-br-ex:

DEVICE=br-ex
TYPE=OVSBridge
DEVICETYPE=ovs
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none

2. Reboot the node, or restart the network service for the networking changes to take effect. For example:

# systemctl restart network

3. Configure the physical networks in /etc/neutron/plugins/ml2/openvswitch_agent.ini and map bridges according to the physical network:

bridge_mappings = physnet1:br-ex

4. Restart the neutron-openvswitch-agent service on the network nodes and compute nodes for the changes to take effect:

systemctl restart neutron-openvswitch-agent

# Name of bridge used for external network traffic. This should be set to
# empty value for the linux bridge
external_network_bridge =

2. Restart neutron-l3-agent for the changes to take effect.

systemctl restart neutron-l3-agent

3. Create a new instance and use the Networking tab in the dashboard to add the new instance directly to the newly-created external network.

# brctl show
bridge name	bridge id		STP enabled	interfaces
qbr84878b78-63		8000.e6b3df9451e0	no		qvb84878b78-63
							tap84878b78-63

qbr86257b61-5d		8000.3a3c888eeae6	no		qvb86257b61-5d
							tap86257b61-5d

                options: {peer=phy-br-ex}
        Port "qvo86257b61-5d"
            tag: 3

            Interface "qvo86257b61-5d"
        Port "qvo84878b78-63"
            tag: 2
            Interface "qvo84878b78-63"

    Bridge br-int
        fail_mode: secure
       Port int-br-ex
            Interface int-br-ex
                type: patch
                options: {peer=phy-br-ex}

    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port br-ex
            Interface br-ex
                type: internal

# ovs-ofctl show br-ex
 4(phy-br-ex): addr:32:e7:a1:6b:90:3e
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

# ovs-ofctl dump-flows br-ex
NXST_FLOW reply (xid=0x4):
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=6527.527s, table=0, n_packets=29211, n_bytes=2725576, idle_age=0, priority=1 actions=NORMAL
 cookie=0x0, duration=2939.172s, table=0, n_packets=117, n_bytes=8296, idle_age=58, priority=4,in_port=4,dl_vlan=3 actions=mod_vlan_vid:172,NORMAL
 cookie=0x0, duration=6111.389s, table=0, n_packets=145, n_bytes=9368, idle_age=98, priority=4,in_port=4,dl_vlan=2 actions=mod_vlan_vid:171,NORMAL
 cookie=0x0, duration=6526.675s, table=0, n_packets=82, n_bytes=6700, idle_age=2462, priority=2,in_port=4 actions=drop

# ovs-ofctl show br-int
 18(int-br-ex): addr:fe:b7:cb:03:c5:c1
     config:     0
     state:      0
     speed: 0 Mbps now, 0 Mbps max

# ovs-ofctl dump-flows br-int
NXST_FLOW reply (xid=0x4):
 cookie=0x0, duration=6770.572s, table=0, n_packets=1239, n_bytes=127795, idle_age=106, priority=1 actions=NORMAL
 cookie=0x0, duration=3181.679s, table=0, n_packets=2605, n_bytes=246456, idle_age=0, priority=3,in_port=18,dl_vlan=172 actions=mod_vlan_vid:3,NORMAL
 cookie=0x0, duration=6353.898s, table=0, n_packets=5077, n_bytes=482582, idle_age=0, priority=3,in_port=18,dl_vlan=171 actions=mod_vlan_vid:2,NORMAL
 cookie=0x0, duration=6769.391s, table=0, n_packets=22301, n_bytes=2013101, idle_age=0, priority=2,in_port=18 actions=drop
 cookie=0x0, duration=6770.463s, table=23, n_packets=0, n_bytes=0, idle_age=6770, priority=0 actions=drop

# grep bridge_mapping /etc/neutron/plugins/ml2/openvswitch_agent.ini
bridge_mappings = physnet1:br-ex

# neutron net-show provider-vlan171
...
| provider:physical_network | physnet1
...

# neutron net-show provider-vlan171
...
| provider:network_type     | vlan                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  | 171                                  |
...

    Bridge br-ex
        Port phy-br-ex
            Interface phy-br-ex
                type: patch
                options: {peer=int-br-ex}
        Port "eth1"
            Interface "eth1"

# ip a
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP qlen 1000

Instances connected in this way are directly attached to the provider external networks, and have external routers configured as their default gateway; no OpenStack Networking (neutron) routers are used. This means that neutron routers cannot be used to proxy metadata requests from instances to the nova-metadata server, which may result in failures while running cloud-init. However, this issue can be resolved by configuring the dhcp agent to proxy metadata requests. You can enable this functionality in /etc/neutron/dhcp_agent.ini. For example:

enable_isolated_metadata = True

Note that the same network can be used to allocate floating IP addresses to instances, even if they have been added to private networks at the same time. The addresses allocated as floating IPs from this network will be bound to the qrouter-xxx namespace on the Network node, and will perform DNAT-SNAT to the associated private IP address. In contrast, the IP addresses allocated for direct external network access will be bound directly inside the instance, and allow the instance to communicate directly with external network.

The physical network adapters in your OpenStack nodes can be expected to carry different types of network traffic, such as instance traffic, storage data, or authentication requests. The type of traffic these NICs will carry affects how their ports are configured on the physical switch.

As a first step, you will need to decide which physical NICs on your Compute node will carry which types of traffic. Then, when the NIC is cabled to a physical switch port, that switch port will need to be specially configured to allow trunked or general traffic.

For example, this diagram depicts a Compute node with two NICs, eth0 and eth1. Each NIC is cabled to a Gigabit Ethernet port on a physical switch, with eth0 carrying instance traffic, and eth1 providing connectivity for OpenStack services:

OpenStack Networking has the ability to calculate the largest possible MTU size that can safely be applied to instances. The MTU value specifies the maximum amount of data a single network packet is able to transfer; this number is variable depending on the most appropriate size for the application. For example, NFS shares might require a different MTU size to that of a VoIP application.

MTU settings need to be set consistently from end-to-end in order to work properly. This means that the MTU setting needs to be the same size at every point the packet passes through, including the VM itself, the virtual network infrastructure, the physical network, and the destination server itself.

For example, the circles in the following diagram indicate the various points where an MTU value would need to be adjusted for traffic between an instance and a physical server. Every interface that handles network traffic will need to have its MTU value changed to accommodate packets of a particular MTU size. This will be required if traffic is expected to travel from the instance 192.168.200.15 through to the physical server 10.20.15.25:

Inconsistent MTU values can result in several network issues, the most common being random packet loss that results in connection drops and slow network performance. Such issues are problematic to troubleshoot, since every possible network point needs to be identified and examined to ensure it has the correct MTU value set.

advertise_mtu = True

            -
              type: ovs_bridge
              name: br-isolated
              use_dhcp: false
              mtu: 9000    # <--- Set MTU
              members:
                -
                  type: ovs_bond
                  name: bond1
                  mtu: 9000    # <--- Set MTU
                  ovs_options: {get_param: BondInterfaceOvsOptions}
                  members:
                    -
                      type: interface
                      name: ens15f0
                      mtu: 9000    # <--- Set MTU
                      primary: true
                    -
                      type: interface
                      name: enp131s0f0
                      mtu: 9000    # <--- Set MTU
                -
                  type: vlan
                  device: bond1
                  vlan_id: {get_param: InternalApiNetworkVlanID}
                  mtu: 9000    # <--- Set MTU
                  addresses:
                  -
                    ip_netmask: {get_param: InternalApiIpSubnet}
                -
                  type: vlan
                  device: bond1
                  mtu: 9000    # <--- Set MTU
                  vlan_id: {get_param: TenantNetworkVlanID}
                  addresses:
                  -
                    ip_netmask: {get_param: TenantIpSubnet}

# neutron net-show <network>

To deploy the base profile, pass the environments/neutron-ml2-ovn.yaml file to openstack overcloud deploy. For example:

$ openstack overcloud deploy \
    --templates /usr/share/openstack-tripleo-heat-templates \
    ...
    -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ml2-ovn.yaml
    ....

To deploy the Pacemaker HA profile, you need to pass the environments/neutron-ml2-ovn-ha.yaml file to the openstack overcloud deploy command. For example:

$ openstack overcloud deploy \
    --templates /usr/share/openstack-tripleo-heat-templates \
    ...
    -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ml2-ovn-ha.yaml
    ....

OVN requires the following components and services:

The following packages are required for OVN:

These are subpackages of the main openvswitch package; the minimum version required is OVS 2.7.2. These packages should already be included with the overcloud-full.qcow2 image.

When director is used to deploy OVN, it performs the following steps:

The following director components are used:

Director has a composable service for OVN named ovn-dbs with two profiles: the base profile, and the pacemaker HA profile. The OVN northbound and southbound databases are hosted by the ovsdb-server service. Similarly, the ovsdb-server process runs alongside ovs-vswitchd to host the OVS database (conf.db).

The ovsdb-server service does not currently support active-active mode, however it does support HA with the master-slave mode, which is managed by pacemaker using the resource agent OCF script. Having ovsdb-server run in master mode allows write access to the database, while all the other slave ovsdb-server services will replicate the database locally from the master, and do not allow write access.

For this reason, the base profile and HA profile are supplied to support both scenarios. When using the base profile, OVN database servers are started only in the bootstrap controller (if the deployment has multiple controllers). If HA profile is enabled, the OVN database servers are started on all the controllers, with pacemaker then selecting one to serve as the master role.

The YAML file for the base profile is available in tripleo-heat-templates/puppet/services/ovn-dbs.yaml. When this service is enabled, the OVN database servers are started only in the bootstrap controller.

For example, if a deployment has 3 controllers (controller-0, controller-1, and controller-2), the OVN database servers will be started on controller-0. If controller-0 goes down, then the OVN database servers are also unavailable, and they are not started on the other controllers. This is a single point of failure.

Director creates a virtual IP address for its internal network, being active on one of the controller nodes. This virtual IP is mapped to OVN_DBS_VIP. To enable the OVN ML2 driver and ovn-controller services to connect to the OVN database servers, puppet-tripleo generates the following HAproxy configuration in haproxy.cfg on each controller node:

Since the OVN database servers are not started in controller-1 and controller-2, HAproxy always redirects the traffic to the OVN database servers running on controller-0.

The YAML file for this profile is located in tripleo-heat-templates/puppet/services/pacemaker/ovn-dbs.yaml. When enabled, the OVN database servers are managed by Pacemaker, and puppet-tripleo creates a pacemaker OCF resource named ovn:ovndb-servers.

The OVN database servers are started on each controller node, and the controller owning the virtual IP address (OVN_DBS_VIP) will be running the OVN DB servers in master mode. The OVN ML2 mechanism driver and ovn-controller then connect to the database servers using the OVN_DBS_VIP value. In event of a failover, Pacemaker moves the virtual IP address (OVN_DBS_VIP) to another controller, and also promotes the OVN database server running on that node to master.

The ovn-controller service runs on each Compute node and connects to the OVN SB database server to retrieve the logical flows. It then translates these logical flows into physical OF flows and adds them to the OVS bridge (br-int). In order to communicate with ovs-vswitchd and install the OF flows, ovn-controller connects to the local ovsdb-server (that hosts conf.db) using the UNIX socket path which was passed when ovn-controller was started (for example unix:/var/run/openvswitch/db.sock).

The ovn-controller service expects certain key-value pairs in the external_ids column of “Open_vSwitch” table; puppet-ovn uses puppet-vswitch to populate these fields. Below are the key-value pairs which puppet-vswitch configures in the external_ids column:

hostname=<HOST NAME>
ovn-encap-ip=<IP OF THE NODE>
ovn-encap-type=geneve
ovn-remote=tcp:OVN_DBS_VIP:6642

This section describes known issues or other limitations known to exist with the Technology Preview of OVN:

QoS policies are applied to individual ports, or to a particular tenant network, where ports with no specific policy attached will inherit the policy.

QoS policies can be dynamically applied, modified, or removed. This example manually creates a bandwidth limiting rule and applies it to a port.

These values allow you to configure the policing algorithm accordingly:

Differentiated Services Code Point (DSCP) allows you to to implement QoS on your network by embedding relevant values in the IP headers. OpenStack Networking (neutron) QoS policies can now use DSCP marking to manage egress traffic on neutron ports and networks. At present, DSCP is only available for VLAN and flat provider networks using Open vSwitch (OVS); support for VXLAN is expected to follow.

In this implementation, a policy is first created, then DSCP rules are defined and applied to the policy. These rules use the --dscp-mark parameter, which specifies the decimal value of a DSCP mark. For example:

1. Create a new QoS policy:

neutron qos-policy-create qos-web-servers --tenant-id 98a2f53c20ce4d50a40dac4a38016c69

2. Create a DSCP rule and apply it to the qos-web-servers policy, using DSCP mark 18:

neutron qos-dscp-marking-rule-create qos-web-servers --dscp-mark 18
Created a new dscp_marking_rule:
+-----------+--------------------------------------+
| Field     | Value                                |
+-----------+--------------------------------------+
| dscp_mark | 18                                   |
| id        | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

3. View the DSCP rules for QoS policy qos-web-servers:

neutron qos-dscp-marking-rule-list qos-web-servers
+-----------+--------------------------------------+
| dscp_mark | id                                   |
+-----------+--------------------------------------+
|        18 | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

4. View the details of the DSCP rule assigned to policy qos-web-servers:

neutron qos-dscp-marking-rule-show d7f976ec-7fab-4e60-af70-f59bf88198e6 qos-web-servers
+-----------+--------------------------------------+
| Field     | Value                                |
+-----------+--------------------------------------+
| dscp_mark | 18                                   |
| id        | d7f976ec-7fab-4e60-af70-f59bf88198e6 |
+-----------+--------------------------------------+

5. Change the DSCP value assigned to a rule:

neutron qos-dscp-marking-rule-update d7f976ec-7fab-4e60-af70-f59bf88198e6 qos-web-servers --dscp-mark 22
Updated dscp_marking_rule: d7f976ec-7fab-4e60-af70-f59bf88198e6

6. Delete a DSCP rule:

neutron qos-dscp-marking-rule-delete d7f976ec-7fab-4e60-af70-f59bf88198e6 qos-web-servers
Deleted dscp_marking_rule(s): d7f976ec-7fab-4e60-af70-f59bf88198e6

Red Hat OpenStack Platform 11 adds Role-based Access Control (RBAC) for QoS policies. As a result, you can now make QoS policies available to certain projects.

For example, you can now create a QoS policy that allows for lower-priority network traffic, and have it only apply to certain projects. In the following command below, the bw-limiter policy created previously is assigned to the demo tenant:

# neutron rbac-create 'bw-limiter' --type qos-policy --target-tenant 80bf5732752a41128e612fe615c886c6 --action access_as_shared

Bridge mappings allow provider network traffic to reach the physical network. Traffic leaves the provider network from the router’s qg-xxx interface and arrives at br-int. A patch port between br-int and br-ex then allows the traffic to pass through the bridge of the provider network and out to the physical network.

int-br-ex <-> phy-br-ex

bridge_mappings = physnet1:br-ex,physnet2:br-ex2

network_vlan_ranges = physnet1:10:20,physnet2:21:25

After removing any mappings, a subsequent patch-port cleanup is required. This action ensures that the bridge configuration is cleared of erroneous entries, with two options available for performing this task:

Examples are given below for each of these two options:

bridge_mappings = physnet1:br-ex,physnet2:br-ex2

bridge_mappings = physnet1:br-ex

# ovs-vsctl del-port br-ex2 phy-br-ex2
# ovs-vsctl del-port br-int int-br-ex2

# service neutron-openvswitch-agent restart

# ovs-vsctl add-port br-ex eth1

# /usr/bin/neutron-ovs-cleanup
--config-file /etc/neutron/plugins/ml2/openvswitch_agent.ini
--log-file /var/log/neutron/ovs-cleanup.log --ovs_all_ports

# systemctl restart neutron-openvswitch-agent
# systemctl restart neutron-l3-agent.service
# systemctl restart neutron-dhcp-agent.service

# systemctl restart neutron-openvswitch-agent

Instances can now send and receive VLAN-tagged traffic over a single vNIC. This ability is particularly useful for NFV applications (VNFs) that expect VLAN-tagged traffic, allowing multiple customers/services to be served by a single vNIC.

For example, the tenant data network can use VLANs, or tunneling (VXLAN/GRE) segmentation, while the instances will see the traffic tagged with VLAN IDs. As a result, network packets are tagged just before they are injected to the instance; they don’t need to be tagged throughout the entire network.

To implement this, start by creating a parent port and attaching it to an existing neutron network. Doing so will add a trunk connection to the parent port you created. Next, create subports. These subports are the ports that connect VLANs to instances, thereby allowing connectivity to the trunk. Within the instance operating system, you need to create a sub-interface that tags traffic for the VLAN associated with the subport.

In a director-based deployment, the trunk plugin is turned on by default. You can review the configuration on the controller nodes:

1.On the controller node, confirm that the trunk plugin is enabled in /etc/neutron/neutron.conf. For example:

service_plugins=router,metering,qos,trunk

1. Identify the network that requires the trunk port connection. This would be the network that will contain the instance that requires access to the trunked VLANs. In this example, this is the public network:

openstack network list
+--------------------------------------+---------+--------------------------------------+
| ID                                   | Name    | Subnets                              |
+--------------------------------------+---------+--------------------------------------+
| 82845092-4701-4004-add7-838837837621 | private | 434c7982-cd96-4c41-a8c9-b93adbdcb197 |
| 8d8bc6d6-5b28-4e00-b99e-157516ff0050 | public  | 3fd811b4-c104-44b5-8ff8-7a86af5e332c |
+--------------------------------------+---------+--------------------------------------+

2. Create the parent trunk port, and attach it to the network that the instance will be connected to. In this example, a neutron port named parent-trunk-port is created on the public network. This trunk will be considered the parent port, as you can use it to create subports.

openstack port create --network public parent-trunk-port
+-----------------------+-----------------------------------------------------------------------------+
| Field                 | Value                                                                       |
+-----------------------+-----------------------------------------------------------------------------+
| admin_state_up        | UP                                                                          |
| allowed_address_pairs |                                                                             |
| binding_host_id       |                                                                             |
| binding_profile       |                                                                             |
| binding_vif_details   |                                                                             |
| binding_vif_type      | unbound                                                                     |
| binding_vnic_type     | normal                                                                      |
| created_at            | 2016-10-20T02:02:33Z                                                        |
| description           |                                                                             |
| device_id             |                                                                             |
| device_owner          |                                                                             |
| extra_dhcp_opts       |                                                                             |
| fixed_ips             | ip_address='172.24.4.230', subnet_id='dc608964-9af3-4fed-9f06-6d3844fb9b9b' |
| headers               |                                                                             |
| id                    | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39                                        |
| mac_address           | fa:16:3e:33:c4:75                                                           |
| name                  | parent-trunk-port                                                           |
| network_id            | 871a6bd8-4193-45d7-a300-dcb2420e7cc3                                        |
| project_id            | 745d33000ac74d30a77539f8920555e7                                            |
| project_id            | 745d33000ac74d30a77539f8920555e7                                            |
| revision_number       | 4                                                                           |
| security_groups       | 59e2af18-93c6-4201-861b-19a8a8b79b23                                        |
| status                | DOWN                                                                        |
| updated_at            | 2016-10-20T02:02:33Z                                                        |
+-----------------------+-----------------------------------------------------------------------------+

3. Create a trunk using the port you just created. In this example the trunk is named parent-trunk.

openstack network trunk create --parent-port parent-trunk-port parent-trunk
+-----------------+--------------------------------------+
| Field           | Value                                |
+-----------------+--------------------------------------+
| admin_state_up  | UP                                   |
| created_at      | 2016-10-20T02:05:17Z                 |
| description     |                                      |
| id              | 0e4263e2-5761-4cf6-ab6d-b22884a0fa88 |
| name            | parent-trunk                         |
| port_id         | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39 |
| revision_number | 1                                    |
| status          | DOWN                                 |
| sub_ports       |                                      |
| tenant_id       | 745d33000ac74d30a77539f8920555e7     |
| updated_at      | 2016-10-20T02:05:17Z                 |
+-----------------+--------------------------------------+

4. View the trunk connection:

openstack network trunk list
+--------------------------------------+--------------+--------------------------------------+-------------+
| ID                                   | Name         | Parent Port                          | Description |
+--------------------------------------+--------------+--------------------------------------+-------------+
| 0e4263e2-5761-4cf6-ab6d-b22884a0fa88 | parent-trunk | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39 |             |
+--------------------------------------+--------------+--------------------------------------+-------------+

openstack network trunk show parent-trunk
+-----------------+--------------------------------------+
| Field           | Value                                |
+-----------------+--------------------------------------+
| admin_state_up  | UP                                   |
| created_at      | 2016-10-20T02:05:17Z                 |
| description     |                                      |
| id              | 0e4263e2-5761-4cf6-ab6d-b22884a0fa88 |
| name            | parent-trunk                         |
| port_id         | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39 |
| revision_number | 1                                    |
| status          | DOWN                                 |
| sub_ports       |                                      |
| tenant_id       | 745d33000ac74d30a77539f8920555e7     |
| updated_at      | 2016-10-20T02:05:17Z                 |
+-----------------+--------------------------------------+

1. Create a neutron port. This port will be used as a subport connection to the trunk. You must also specify the MAC address that was assigned to the parent port:

openstack port create --network private --mac-address fa:16:3e:33:c4:75 subport-trunk-port
+-----------------------+--------------------------------------------------------------------------+
| Field                 | Value                                                                    |
+-----------------------+--------------------------------------------------------------------------+
| admin_state_up        | UP                                                                       |
| allowed_address_pairs |                                                                          |
| binding_host_id       |                                                                          |
| binding_profile       |                                                                          |
| binding_vif_details   |                                                                          |
| binding_vif_type      | unbound                                                                  |
| binding_vnic_type     | normal                                                                   |
| created_at            | 2016-10-20T02:08:14Z                                                     |
| description           |                                                                          |
| device_id             |                                                                          |
| device_owner          |                                                                          |
| extra_dhcp_opts       |                                                                          |
| fixed_ips             | ip_address='10.0.0.11', subnet_id='1a299780-56df-4c0b-a4c0-c5a612cef2e8' |
| headers               |                                                                          |
| id                    | 479d742e-dd00-4c24-8dd6-b7297fab3ee9                                     |
| mac_address           | fa:16:3e:33:c4:75                                                        |
| name                  | subport-trunk-port                                                       |
| network_id            | 3fe6b758-8613-4b17-901e-9ba30a7c4b51                                     |
| project_id            | 745d33000ac74d30a77539f8920555e7                                         |
| project_id            | 745d33000ac74d30a77539f8920555e7                                         |
| revision_number       | 4                                                                        |
| security_groups       | 59e2af18-93c6-4201-861b-19a8a8b79b23                                     |
| status                | DOWN                                                                     |
| updated_at            | 2016-10-20T02:08:15Z                                                     |
+-----------------------+--------------------------------------------------------------------------+

2. Associate the port with the trunk (parent-trunk), and specify the VLAN ID (55):

openstack network trunk set --subport port=subport-trunk-port,segmentation-type=vlan,segmentation-id=55 parent-trunk

The instance operating system must be configured to use the MAC address that neutron assigned to the subport. You can also configure the subport to use a specific MAC address during the subport creation step.

1. Review the configuration of your network trunk:

$ openstack network trunk list
+--------------------------------------+--------------+--------------------------------------+-------------+
| ID                                   | Name         | Parent Port                          | Description |
+--------------------------------------+--------------+--------------------------------------+-------------+
| 0e4263e2-5761-4cf6-ab6d-b22884a0fa88 | parent-trunk | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39 |             |
+--------------------------------------+--------------+--------------------------------------+-------------+


$ openstack network trunk show parent-trunk
+-----------------+------------------------------------------------------------------------------------------------+
| Field           | Value                                                                                          |
+-----------------+------------------------------------------------------------------------------------------------+
| admin_state_up  | UP                                                                                             |
| created_at      | 2016-10-20T02:05:17Z                                                                           |
| description     |                                                                                                |
| id              | 0e4263e2-5761-4cf6-ab6d-b22884a0fa88                                                           |
| name            | parent-trunk                                                                                   |
| port_id         | 20b6fdf8-0d43-475a-a0f1-ec8f757a4a39                                                           |
| revision_number | 2                                                                                              |
| status          | DOWN                                                                                           |
| sub_ports       | port_id='479d742e-dd00-4c24-8dd6-b7297fab3ee9', segmentation_id='55', segmentation_type='vlan' |
| tenant_id       | 745d33000ac74d30a77539f8920555e7                                                               |
| updated_at      | 2016-10-20T02:10:06Z                                                                           |
+-----------------+------------------------------------------------------------------------------------------------+

2. Create an instance that uses the parent port id as its vNIC:

nova boot --image cirros --flavor m1.tiny testInstance --security-groups default --key-name sshaccess --nic port-id=20b6fdf8-0d43-475a-a0f1-ec8f757a4a39


+--------------------------------------+-----------------------------------------------+
| Property                             | Value                                         |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                        |
| OS-EXT-AZ:availability_zone          |                                               |
| OS-EXT-SRV-ATTR:host                 | -                                             |
| OS-EXT-SRV-ATTR:hostname             | testinstance                                  |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                             |
| OS-EXT-SRV-ATTR:instance_name        |                                               |
| OS-EXT-SRV-ATTR:kernel_id            |                                               |
| OS-EXT-SRV-ATTR:launch_index         | 0                                             |
| OS-EXT-SRV-ATTR:ramdisk_id           |                                               |
| OS-EXT-SRV-ATTR:reservation_id       | r-juqco0el                                    |
| OS-EXT-SRV-ATTR:root_device_name     | -                                             |
| OS-EXT-SRV-ATTR:user_data            | -                                             |
| OS-EXT-STS:power_state               | 0                                             |
| OS-EXT-STS:task_state                | scheduling                                    |
| OS-EXT-STS:vm_state                  | building                                      |
| OS-SRV-USG:launched_at               | -                                             |
| OS-SRV-USG:terminated_at             | -                                             |
| accessIPv4                           |                                               |
| accessIPv6                           |                                               |
| adminPass                            | uMyL8PnZRBwQ                                  |
| config_drive                         |                                               |
| created                              | 2016-10-20T03:02:51Z                          |
| description                          | -                                             |
| flavor                               | m1.tiny (1)                                   |
| hostId                               |                                               |
| host_status                          |                                               |
| id                                   | 88b7aede-1305-4d91-a180-67e7eac8b70d          |
| image                                | cirros (568372f7-15df-4e61-a05f-10954f79a3c4) |
| key_name                             | sshaccess                                     |
| locked                               | False                                         |
| metadata                             | {}                                            |
| name                                 | testInstance                                  |
| os-extended-volumes:volumes_attached | []                                            |
| progress                             | 0                                             |
| security_groups                      | default                                       |
| status                               | BUILD                                         |
| tags                                 | []                                            |
| tenant_id                            | 745d33000ac74d30a77539f8920555e7              |
| updated                              | 2016-10-20T03:02:51Z                          |
| user_id                              | 8c4aea738d774967b4ef388eb41fef5e              |
+--------------------------------------+-----------------------------------------------+

This example procedure demonstrates how to use a RBAC policy to grant a tenant access to a shared network.

As a result, users in the auditors project are able to connect instances to the web-servers network.

You can grant RBAC access to external networks (networks with gateway interfaces attached) using the --action access_as_external parameter

For example, this procedure creates a RBAC for the web-servers network and grants access to the engineering tenant (c717f263785d4679b16a122516247deb):

1. Create a new RBAC policy using --action access_as_external:

# neutron rbac-create 6e437ff0-d20f-4483-b627-c3749399bdca --type network --target-tenant c717f263785d4679b16a122516247deb --action access_as_external
 Created a new rbac_policy:
+---------------+--------------------------------------+
| Field         | Value                                |
+---------------+--------------------------------------+
| action        | access_as_external                   |
| id            | ddef112a-c092-4ac1-8914-c714a3d3ba08 |
| object_id     | 6e437ff0-d20f-4483-b627-c3749399bdca |
| object_type   | network                              |
| target_tenant | c717f263785d4679b16a122516247deb     |
| tenant_id     | c717f263785d4679b16a122516247deb     |
+---------------+--------------------------------------+

2. As a result, users in the Engineering tenant are able to view the network or connect instances to it:

$ neutron net-list
+--------------------------------------+-------------+------------------------------------------------------+
| id                                   | name        | subnets                                              |
+--------------------------------------+-------------+------------------------------------------------------+
| 6e437ff0-d20f-4483-b627-c3749399bdca | web-servers | fa273245-1eff-4830-b40c-57eaeac9b904 192.168.10.0/24 |
+--------------------------------------+-------------+------------------------------------------------------+

OpenStack Networking (neutron) provides routing services for project networks. Without a router, instances in a project network are only able to communicate with one another over a shared L2 broadcast domain. Creating a router and assigning it to a tenant network allows the instances in that network to communicate with other project networks or upstream (if an external gateway is defined for the router).

Distributed Virtual Routing (DVR) offers an alternative routing design, which is now fully supported with Red Hat OpenStack Platform 11. It intends to isolate the failure domain of the Controller node and optimize network traffic by deploying the L3 agent and schedule routers on every Compute node. When using DVR:

The neutron-ovs-dvr.yaml environment file configures the required DVR-specific parameters. Configuring DVR for arbitrary deployment configuration requires additional consideration. The requirements are:

(a) The interface connected to the physical network for external network traffic must be configured on both the Compute and Controller nodes.

(b) A bridge must be created on Compute and Controller nodes, with an interface for external network traffic.

(c) Neutron must be configured to allow this bridge to be used.

The host networking configuration (a and b) are controlled by Heat templates that pass configuration to the Heat-managed nodes for consumption by the os-net-config process. This is essentially automation of provisioning host networking. Neutron must also be configured (c) to match the provisioned networking environment. The defaults are not expected to work in production environments. For example, a proof-of-concept environment using the typical defaults might be similar to the following:

1. Verify that the value for OS::TripleO::Compute::Net::SoftwareConfig in environments/neutron-ovs-dvr.yaml is the same as the OS::TripleO::Controller::Net::SoftwareConfig value in use. This can normally be found in the network environment file in use when deploying the overcloud, for example, environments/net-multiple-nics.yaml. This will create the appropriate external network bridge for the Compute node’s L3 agent.

2. Configure a neutron port for the Compute node on the external network by modifying OS::TripleO::Compute::Ports::ExternalPort to an appropriate value, such as OS::TripleO::Compute::Ports::ExternalPort: ../network/ports/external.yaml

3. Include environments/neutron-ovs-dvr.yaml as an environment file when deploying the overcloud. For example:

$ openstack overcloud deploy --templates -e /usr/share/openstack-tripleo-heat-templates/environments/neutron-ovs-dvr.yaml

4. Verify that L3 HA is disabled.

For production environments (or test environments that require special customization, for example, involving network isolation, dedicated NICs, among others) the example environments can be used as a guide. Particular attention should be give to the bridge mapping type parameters used by the L2 agents (for example, OVS) and any reference to external facing bridges for other agents (such as the L3 agent).

This section describes the upgrade path to distributed routing for Red Hat OpenStack Platform deployments that use L3 HA centralized routing.

1. Upgrade your deployment and validate that it is working correctly.

2. Run the director stack update to configure DVR, as described in Deploying DVR.

3. Confirm that routing still works through the existing routers.

4. You will not be able to directly transition a L3 HA router to distributed. Instead, for each router, turn off the L3 HA option, and then turn on the distributed option:

4a. Move the router out of the admin state:

$ neutron router-update --admin-state-up=False

4b. Convert the router to the legacy type:

$ neutron router-update --ha=False

4c. Configure the router to use DVR:

$ neutron router-update --distributed=True

4d. Move the router into the admin state:

$ neutron router-update --admin-state-up=True

4e. Confirm that routing still works, and is now distributed.

OpenStack Networking (neutron) services can be broadly classified into two categories.

1. - Neutron API server - This service runs the OpenStack Networking API server, which has the main responsibility of providing an API for end users and services to interact with OpenStack Networking. This server also has the responsibility of interacting with the underlying database to store and retrieve tenant network, router, and loadbalancer details, among others.

2. - Neutron Agents - These are the services that deliver various network functionality for OpenStack Networking.

The following diagram describes the flow of HTTPS traffic through to a pool member:

This procedure configures OpenStack Networking (neutron) to use LBaaS with the Open vSwitch (OVS) plug-in. Perform these steps on nodes running the neutron-server service.

On the Controller node (API Server):

You can configure neutron to automatically reschedule load balancers from failed LBaaS agents. Previously, load balancers could be scheduled across multiple LBaaS agents, however if a hypervisor died, the load balancers scheduled to that node would cease operation. Now these load balancers can be automatically rescheduled to a different agent. This feature is turned off by default and controlled using allow_automatic_lbaas_agent_failover.

IPv6 subnets are created using the neutron subnet-create command. In addition, you can optionally specify the address mode and the Router Advertisement mode. The possible combinations of these options are explained below:

# openstack project list
+----------------------------------+----------+
| ID                               | Name     |
+----------------------------------+----------+
| 25837c567ed5458fbb441d39862e1399 |    QA    |
| f59f631a77264a8eb0defc898cb836af |  admin   |
| 4e2e1951e70643b5af7ed52f3ff36539 |   demo   |
| 8561dff8310e4cd8be4b6fd03dc8acf5 | services |
+----------------------------------+----------+

# neutron net-list
+--------------------------------------+------------------+-------------------------------------------------------------+
| id                                   | name             | subnets                                                     |
+--------------------------------------+------------------+-------------------------------------------------------------+
| 8357062a-0dc2-4146-8a7f-d2575165e363 | private          | c17f74c4-db41-4538-af40-48670069af70 10.0.0.0/24            |
| 31d61f7d-287e-4ada-ac29-ed7017a54542 | public           | 303ced03-6019-4e79-a21c-1942a460b920 172.24.4.224/28        |
| 6aff6826-4278-4a35-b74d-b0ca0cbba340 | database-servers |                                                             |
+--------------------------------------+------------------+-------------------------------------------------------------+

# neutron subnet-create --ip-version 6 --ipv6_address_mode=dhcpv6-stateful --tenant-id 25837c567ed5458fbb441d39862e1399 database-servers fdf8:f53b:82e4::53/125

Created a new subnet:
+-------------------+--------------------------------------------------------------+
| Field             | Value                                                        |
+-------------------+--------------------------------------------------------------+
| allocation_pools  | {"start": "fdf8:f53b:82e4::52", "end": "fdf8:f53b:82e4::56"} |
| cidr              | fdf8:f53b:82e4::53/125                                       |
| dns_nameservers   |                                                              |
| enable_dhcp       | True                                                         |
| gateway_ip        | fdf8:f53b:82e4::51                                           |
| host_routes       |                                                              |
| id                | cdfc3398-997b-46eb-9db1-ebbd88f7de05                         |
| ip_version        | 6                                                            |
| ipv6_address_mode | dhcpv6-stateful                                              |
| ipv6_ra_mode      |                                                              |
| name              |                                                              |
| network_id        | 6aff6826-4278-4a35-b74d-b0ca0cbba340                         |
| tenant_id         | 25837c567ed5458fbb441d39862e1399                             |
+-------------------+--------------------------------------------------------------+

# neutron net-list
+--------------------------------------+------------------+-------------------------------------------------------------+
| id                                   | name             | subnets                                                     |
+--------------------------------------+------------------+-------------------------------------------------------------+
| 6aff6826-4278-4a35-b74d-b0ca0cbba340 | database-servers | cdfc3398-997b-46eb-9db1-ebbd88f7de05 fdf8:f53b:82e4::50/125 |
| 8357062a-0dc2-4146-8a7f-d2575165e363 | private          | c17f74c4-db41-4538-af40-48670069af70 10.0.0.0/24            |
| 31d61f7d-287e-4ada-ac29-ed7017a54542 | public           | 303ced03-6019-4e79-a21c-1942a460b920 172.24.4.224/28        |
+--------------------------------------+------------------+-------------------------------------------------------------+

# nova list
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+
| ID                                   | Name       | Status | Task State | Power State | Networks                            |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+
| fad04b7a-75b5-4f96-aed9-b40654b56e03 | corp-vm-01 | ACTIVE | -          | Running     | database-servers=fdf8:f53b:82e4::52 |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+

Quota options available for L3 networking: quota_floatingip - Number of floating IPs allowed per tenant. quota_network - Number of networks allowed per tenant. quota_port - Number of ports allowed per tenant. quota_router - Number of routers allowed per tenant. quota_subnet - Number of subnets allowed per tenant. quota_vip - Number of vips allowed per tenant.

Quota options governing firewall management: quota_firewall - Number of firewalls allowed per tenant. quota_firewall_policy - Number of firewall policies allowed per tenant. quota_firewall_rule - Number of firewall rules allowed per tenant.

Quota options for managing the permitted number of security groups: quota_security_group - Number of security groups allowed per tenant. quota_security_group_rule - Number of security group rules allowed per tenant.

Quota options for administrators to consider: default_quota - Default number of resource allowed per tenant. quota_health_monitor - Number of health monitors allowed per tenant. Health monitors do not consume resources, however the quota option is available due to the OpenStack Networking back end handling members as resource consumers. quota_member - Number of pool members allowed per tenant. Members do not consume resources, however the quota option is available due to the OpenStack Networking back end handling members as resource consumers. quota_pool - Number of pools allowed per tenant.

1. Install the FWaaS packages:

# yum install openstack-neutron-fwaas python-neutron-fwaas

2. Enable the FWaaS plugin in the neutron.conf file:

service_plugins = neutron.services.firewall.fwaas_plugin.FirewallPlugin

3. Configure FWaaS in the fwaas_driver.ini file:

[fwaas]
driver = neutron.services.firewall.drivers.linux.iptables_fwaas.IptablesFwaasDriver
enabled = True

[service_providers]
service_provider = LOADBALANCER:Haproxy:neutron_lbaas.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default

4. FWaaS management options are available in OpenStack dashboard. Enable this option in the local_settings.py file, usually located on the Controller node:

/usr/share/openstack-dashboard/openstack_dashboard/local/local_settings.py
'enable_firewall' = True

5. Restart neutron-server to apply the changes.

# systemctl restart neutron-server

First create the firewall rules and create a policy to contain them, then create a firewall and apply the policy:

1. Create a firewall rule:

$ neutron firewall-rule-create --protocol <tcp|udp|icmp|any> --destination-port <port-range> --action <allow|deny>

The CLI requires a protocol value. If the rule is protocol agnostic, the any value can be used.

2. Create a firewall policy:

$ neutron firewall-policy-create --firewall-rules "<firewall-rule IDs or names separated by space>" myfirewallpolicy

The order of the rules specified above is important. You can create an empty firewall policy and add rules later, either with the update operation (when adding multiple rules) or with the insert-rule operations (when adding a single rule).

Note: FWaaS always adds a default deny all rule at the lowest precedence of each policy. Consequently, a firewall policy with no rules blocks all traffic by default.

$ neutron firewall-create <firewall-policy-uuid>

The firewall remains in PENDING_CREATE state until an OpenStack Networking router is created, and an interface is attached.

Create a port and allow one address pair:

# neutron port-create net1 --allowed-address-pairs type=dict list=true mac_address=<mac_address>,ip_address=<ip_cidr>

# neutron port-update <port-uuid> --allowed-address-pairs type=dict list=true mac_address=<mac_address>,ip_address=<ip_cidr>

An OpenStack Networking deployment without any high availability features is going to be vulnerable to physical node failures.

In a typical deployment, tenants create virtual routers, which are scheduled to run on physical L3 agent nodes. This becomes an issue when you lose a L3 agent node and the dependent virtual machines subsequently lose connectivity to external networks. Any floating IP addresses will also be unavailable. In addition, connectivity is also lost between any networks hosted by the router.

This active/passive high availability configuration uses the industry standard VRRP (as defined in RFC 3768) to protect tenant routers and floating IP addresses. A virtual router is randomly scheduled across multiple OpenStack Networking nodes, with one designated as the active, and the remainder serving in a standby role.

In the diagram below, the active Router1 and Router2 are running on separate physical L3 agent nodes. Layer 3 High Availability has scheduled backup virtual routers on the corresponding nodes, ready to resume service in the case of a physical node failure. When the L3 agent node fails, Layer 3 High Availability reschedules the affected virtual router and floating IP addresses to a working node:

During a failover event, instance TCP sessions through floating IPs remain unaffected, and will migrate to the new L3 node without disruption. Only SNAT traffic is affected by failover events.

The L3 agent itself is further protected when in an active/active HA mode.

Layer 3 High Availability configuration occurs in the back end and is invisible to the tenant. They can continue to create and manage their virtual routers as usual, however there are some limitations to be aware of when designing your Layer 3 High Availability implementation:

The Neutron API has been updated to allow administrators to set the --ha=True/False flag when creating a router, which overrides the (Default) configuration of l3_ha in neutron.conf. See the next section for the necessary configuration steps.

This procedure enables Layer 3 High Availability on the OpenStack Networking and L3 agent nodes.

l3_ha = True
max_l3_agents_per_router = 2
min_l3_agents_per_router = 2

# neutron router-create --ha=<True | False> routerName

# systemctl restart neutron-server.service

# ip netns exec qrouter-b30064f9-414e-4c98-ab42-646197c74020 ip address
<snip>
2794: ha-45249562-ec: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 12:34:56:78:2b:5d brd ff:ff:ff:ff:ff:ff
inet 169.254.0.2/24 brd 169.254.0.255 scope global ha-54b92d86-4f

This chapter describes how to use Red Hat OpenStack Platform with OpenDaylight to create an SDN overlay solution. As a result, your deployment can have seamless connectivity between bare metal workloads, legacy VLAN domains, SR-IOV-enabled workloads, and virtualized workloads.

This connectivity is provided by deploying a VXLAN gateway on your physical ToR (top-of-rack) switches (also known as access switches, rather than core switches). The ToR switches must support VXLAN encapsulation and the HWVTEP schema. Network switching is performed using neutron’s multi-segment networks and the L2GW service plugin. The southbound configuration is implemented using a HWVTEP plugin, which implements the OVSDB protocol and hardware_vtep schema.

This guide describes a series of use cases for L2GW and require that SR-IOV be configured on your Compute nodes.

For information on configuring SR-IOV, see https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/html/network_functions_virtualization_configuration_guide/part-sriov-nfv-configuration.

This use case has two instances on the same SR-IOV-enabled compute node. In this scenario, you have a VXLAN tunnel running from ToR to the Controller node:

Now that each instance has an allocated IP address, IP packets between VM1 and VM2 on the compute node are switched locally by the Intel 82599ES NIC. In this example, the NIC used is an Intel 82599ES, which supports local switching, so the IP packets between VM1 and VM2 do not egress the NIC. For vendor NICs that do not support local switching, the packets from VM1 to VM2 would be switched by the ToR.

This use case is similar to Use Case 1, and DHCP works the same way. The difference is that IP forwarding between VM1 (on Compute Node 1) and VM2 (on Compute Node 2) is performed by the ToR switch, as indicated by the dashed red line:

In this use case, connectivity is established between instances attached to a software VEB (Virtual Ethernet Bridge) and a hardware VEB:

In this use case, ingress and egress traffic for VM2 (on Compute node 2) passes through the ToR switch and traverses the SR-IOV PF attached to VM2:

In this use case, the three Compute nodes are attached to two separate ToR switches. A single neutron network named multinet spans all three nodes, allowing the instances to connect with each other on the same logical Layer 2 network.

In this use case, VM1 and VM2 are on separate Compute nodes and share the same neutron network. VM3 and VM4 are also on separate nodes, and share the same neutron network. Traffic for both neutron networks passes through the same physical NIC on each node:

Each ToR vendor switch will likely have specific configuration steps to setup and prepare the switch for L2GW VXLAN tunnels. Please refer to your switch vendor documentation to determine the configuration commands for the steps described below.

Each ToR switch will need:

Once OpenDaylight (ODL) has been deployed, you will need to create the required transport zone:

As the result of following these steps, you should be able to SSH into one of the instances and ping an instance hosted on the other Compute node. The resulting ICMP traffic will traverse the OVS and ToR switches as an encapsulated layer 2 tunnel.