Chapter 10. Using a VXLAN to create a virtual layer-2 domain for VMs

Procedure

1. Add a new Ethernet connection profile to NetworkManager:

   # nmcli connection add con-name Example ifname enp1s0 type ethernet

2. Configure the IPv4 settings:

   # nmcli connection modify Example ipv4.addresses 198.51.100.2/24 ipv4.method manual ipv4.gateway 198.51.100.254 ipv4.dns 198.51.100.200 ipv4.dns-search example.com

   Skip this step if the network uses DHCP.

3. Activate the Example connection:

   # nmcli connection up Example

Verification

1. Display the status of the devices and connections:

   # nmcli device status
   DEVICE      TYPE      STATE      CONNECTION
   enp1s0      ethernet  connected  Example

2. Ping a host in a remote network to verify the IP settings:

   # ping RHEL-host-B.example.com

   Note that you cannot ping the other VM host before you have configured the network on that host as well.

Procedure

1. Create the bridge br0:

   # nmcli connection add type bridge con-name br0 ifname br0 ipv4.method disabled ipv6.method disabled

   This command sets no IPv4 and IPv6 addresses on the bridge device, because this bridge works on layer 2.

2. Create the VXLAN interface and attach it to br0:

   # nmcli connection add type vxlan port-type bridge con-name br0-vxlan10 ifname vxlan10 id 10 local 198.51.100.2 remote 203.0.113.1 controller br0

   This command uses the following settings:

   • id 10: Sets the VXLAN identifier.
   • local 198.51.100.2: Sets the source IP address of outgoing packets.
   • remote 203.0.113.1: Sets the unicast or multicast IP address to use in outgoing packets when the destination link layer address is not known in the VXLAN device forwarding database.
   • controller br0: Sets this VXLAN connection to be created as a port in the br0 connection.
   • ipv4.method disabled and ipv6.method disabled: Disables IPv4 and IPv6 on the bridge.

   By default, NetworkManager uses 8472 as the destination port. If the destination port is different, additionally, pass the destination-port <port_number> option to the command.

3. Activate the br0 connection profile:

   # nmcli connection up br0

4. Open port 8472 for incoming UDP connections in the local firewall:

   # firewall-cmd --permanent --add-port=8472/udp
   # firewall-cmd --reload

Procedure

1. Create the ~/vxlan10-bridge.xml file with the following content:

   <network>
    <name>vxlan10-bridge</name>
    <forward mode="bridge" />
    <bridge name="br0" />
   </network>

2. Use the ~/vxlan10-bridge.xml file to create a new virtual network in libvirt:

   # virsh net-define ~/vxlan10-bridge.xml

3. Remove the ~/vxlan10-bridge.xml file:

   # rm ~/vxlan10-bridge.xml

4. Start the vxlan10-bridge virtual network:

   # virsh net-start vxlan10-bridge

5. Configure the vxlan10-bridge virtual network to start automatically when the libvirtd service starts:

   # virsh net-autostart vxlan10-bridge

Verification

1. Display the virtual network interfaces of the VM on the host:

   # virsh domiflist VM_name
    Interface   Type     Source           Model    MAC
   -------------------------------------------------------------------
    vnet1       bridge   vxlan10-bridge   virtio   52:54:00:c5:98:1c

2. Display the interfaces attached to the vxlan10-bridge bridge:

   # ip link show master vxlan10-bridge
   18: vxlan10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default qlen 1000
       link/ether 2a:53:bd:d5:b3:0a brd ff:ff:ff:ff:ff:ff
   19: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UNKNOWN mode DEFAULT group default qlen 1000
       link/ether 52:54:00:c5:98:1c brd ff:ff:ff:ff:ff:ff

   Note that the libvirtd service dynamically updates the bridge’s configuration. When you start a VM which uses the vxlan10-bridge network, the corresponding vnet* device on the host appears as a port of the bridge.

3. Use address resolution protocol (ARP) requests to verify whether VMs are in the same VXLAN:

   1. Start two or more VMs in the same VXLAN.

   2. Send an ARP request from one VM to the other one:

      # arping -c 1 192.0.2.2
      ARPING 192.0.2.2 from 192.0.2.1 enp1s0
      Unicast reply from 192.0.2.2 [52:54:00:c5:98:1c] 1.450ms
      Sent 1 probe(s) (0 broadcast(s))
      Received 1 response(s) (0 request(s), 0 broadcast(s))

      If the command shows a reply, the VM is in the same layer-2 domain and, in this case in the same VXLAN.

      Install the iputils package to use the arping utility.