Chapter 10. Custom network interface templates
After you configure Chapter 8, Basic network isolation, you can create a set of custom network interface templates to suit the nodes in your environment. For example, you can include the following files:
-
The environment file to enable network isolation (
/usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml). -
The environment file to configure network defaults (
/usr/share/openstack-tripleo-heat-templates/environments/network-environment.yaml). - Templates to define your NIC layout for each node. The overcloud core template collection contains a set of defaults for different use cases. To create a custom NIC template, render a default Jinja2 template as the basis for your custom templates.
-
A custom environment file to enable NICs. This example uses a custom environment file (
/home/stack/templates/custom-network-configuration.yaml) that references your custom interface templates. - Any additional environment files to customize your networking parameters.
-
If you customize your networks, a custom
network_data.yamlfile. -
If you create additional or custom composable networks, a custom
network_data.yamlfile and a customroles_data.yamlfile.
Some of the files in the previous list are Jinja2 format files and have a .j2.yaml extension. Director renders these files to .yaml versions during deployment.
10.1. Custom network architecture Copy linkLink copied to clipboard!
The default NIC templates might not suit a specific network configuration. For example, you might want to create your own custom NIC template that suits a specific network layout. You might want to separate the control services and data services on to separate NICs. In this situation, you can map the service to NIC assignments in the following way:
NIC1 (Provisioning)
- Provisioning / Control Plane
NIC2 (Control Group)
- Internal API
- Storage Management
- External (Public API)
NIC3 (Data Group)
- Tenant Network (VXLAN tunneling)
- Tenant VLANs / Provider VLANs
- Storage
- External VLANs (Floating IP/SNAT)
NIC4 (Management)
- Management
10.2. Rendering default network interface templates for customization Copy linkLink copied to clipboard!
To simplify the configuration of custom interface templates, render the Jinja2 syntax of a default NIC template and use the rendered templates as the basis for your custom configuration.
Procedure
Render a copy of the
openstack-tripleo-heat-templatescollection with theprocess-templates.pyscript:cd /usr/share/openstack-tripleo-heat-templates ./tools/process-templates.py -o ~/openstack-tripleo-heat-templates-rendered
$ cd /usr/share/openstack-tripleo-heat-templates $ ./tools/process-templates.py -o ~/openstack-tripleo-heat-templates-renderedCopy to Clipboard Copied! Toggle word wrap Toggle overflow This converts all Jinja2 templates to their rendered YAML versions and saves the results to
~/openstack-tripleo-heat-templates-rendered.If you use a custom network file or custom roles file, you can include these files using the
-nand-roptions respectively:./tools/process-templates.py -o ~/openstack-tripleo-heat-templates-rendered -n /home/stack/network_data.yaml -r /home/stack/roles_data.yaml
$ ./tools/process-templates.py -o ~/openstack-tripleo-heat-templates-rendered -n /home/stack/network_data.yaml -r /home/stack/roles_data.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow Copy the multiple NIC example:
cp -r ~/openstack-tripleo-heat-templates-rendered/network/config/multiple-nics/ ~/templates/custom-nics/
$ cp -r ~/openstack-tripleo-heat-templates-rendered/network/config/multiple-nics/ ~/templates/custom-nics/Copy to Clipboard Copied! Toggle word wrap Toggle overflow -
Edit the template set in
custom-nicsto suit your own network configuration.
10.3. Network interface architecture Copy linkLink copied to clipboard!
The custom NIC templates that you render in Section 10.2, “Rendering default network interface templates for customization” contain the parameters and resources sections.
Parameters
The parameters section contains all network configuration parameters for network interfaces. This includes information such as subnet ranges and VLAN IDs. This section should remain unchanged as the heat template inherits values from its parent template. However, you can use a network environment file to modify the values for some parameters.
Resources
The resources section is where the main network interface configuration occurs. In most cases, the resources section is the only one that requires modification. Each resources section begins with the following header:
This snippet runs a script (run-os-net-config.sh) that creates a configuration file for os-net-config to use to configure network properties on a node. The network_config section contains the custom network interface data sent to the run-os-net-config.sh script. You arrange this custom interface data in a sequence based on the type of device.
If you create custom NIC templates, you must set the run-os-net-config.sh script location to an absolute path for each NIC template. The script is located at /usr/share/openstack-tripleo-heat-templates/network/scripts/run-os-net-config.sh on the undercloud.
10.4. Network interface reference Copy linkLink copied to clipboard!
Network interface configuration contains the following parameters:
interface
Defines a single network interface. The configuration defines each interface using either the actual interface name ("eth0", "eth1", "enp0s25") or a set of numbered interfaces ("nic1", "nic2", "nic3"):
- type: interface
name: nic2
- type: interface
name: nic2
| Option | Default | Description |
|---|---|---|
| name | Name of the interface. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the interface. | |
| routes | A list of routes assigned to the interface. For more information, see routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| primary | False | Defines the interface as the primary interface. |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the interface. |
| ethtool_opts |
Set this option to |
vlan
Defines a VLAN. Use the VLAN ID and subnet passed from the parameters section.
For example:
- type: vlan
vlan_id:{get_param: ExternalNetworkVlanID}
addresses:
- ip_netmask: {get_param: ExternalIpSubnet}
- type: vlan
vlan_id:{get_param: ExternalNetworkVlanID}
addresses:
- ip_netmask: {get_param: ExternalIpSubnet}
| Option | Default | Description |
|---|---|---|
| vlan_id | The VLAN ID. | |
| device | The parent device to attach the VLAN. Use this parameter when the VLAN is not a member of an OVS bridge. For example, use this parameter to attach the VLAN to a bonded interface device. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the VLAN. | |
| routes | A list of routes assigned to the VLAN. For more information, see routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| primary | False | Defines the VLAN as the primary interface. |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the VLAN. |
ovs_bond
Defines a bond in Open vSwitch to join two or more interfaces together. This helps with redundancy and increases bandwidth.
For example:
| Option | Default | Description |
|---|---|---|
| name | Name of the bond. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the bond. | |
| routes | A list of routes assigned to the bond. For more information, see routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| primary | False | Defines the interface as the primary interface. |
| members | A sequence of interface objects that you want to use in the bond. | |
| ovs_options | A set of options to pass to OVS when creating the bond. | |
| ovs_extra | A set of options to set as the OVS_EXTRA parameter in the network configuration file of the bond. | |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the bond. |
ovs_bridge
Defines a bridge in Open vSwitch, which connects multiple interface, ovs_bond, and vlan objects together.
The network interface type, ovs_bridge, takes a parameter name.
If you have multiple bridges, you must use distinct bridge names other than accepting the default name of bridge_name. If you do not use distinct names, then during the converge phase, two network bonds are placed on the same bridge.
If you are defining an OVS bridge for the external tripleo network, then retain the values bridge_name and interface_name as your deployment framework automatically replaces these values with an external bridge name and an external interface name, respectively.
For example:
The OVS bridge connects to the Networking service (neutron) server to obtain configuration data. If the OpenStack control traffic, typically the Control Plane and Internal API networks, is placed on an OVS bridge, then connectivity to the neutron server is lost whenever you upgrade OVS, or the OVS bridge is restarted by the admin user or process. This causes some downtime. If downtime is not acceptable in these circumstances, then you must place the Control group networks on a separate interface or bond rather than on an OVS bridge:
- You can achieve a minimal setting when you put the Internal API network on a VLAN on the provisioning interface and the OVS bridge on a second interface.
- To implement bonding, you need at least two bonds (four network interfaces). Place the control group on a Linux bond (Linux bridge). If the switch does not support LACP fallback to a single interface for PXE boot, then this solution requires at least five NICs.
| Option | Default | Description |
|---|---|---|
| name | Name of the bridge. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the bridge. | |
| routes | A list of routes assigned to the bridge. For more information, see routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| members | A sequence of interface, VLAN, and bond objects that you want to use in the bridge. | |
| ovs_options | A set of options to pass to OVS when creating the bridge. | |
| ovs_extra | A set of options to to set as the OVS_EXTRA parameter in the network configuration file of the bridge. | |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the bridge. |
linux_bond
Defines a Linux bond that joins two or more interfaces together. This helps with redundancy and increases bandwidth. Ensure that you include the kernel-based bonding options in the bonding_options parameter.
For example:
Note that nic2 uses primary: true to ensure that the bond uses the MAC address for nic2.
| Option | Default | Description |
|---|---|---|
| name | Name of the bond. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the bond. | |
| routes | A list of routes assigned to the bond. See routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| primary | False | Defines the interface as the primary interface. |
| members | A sequence of interface objects that you want to use in the bond. | |
| bonding_options | A set of options when creating the bond. | |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the bond. |
linux_bridge
Defines a Linux bridge, which connects multiple interface, linux_bond, and vlan objects together. The external bridge also uses two special values for parameters:
-
bridge_name, which is replaced with the external bridge name. -
interface_name, which is replaced with the external interface.
For example:
| Option | Default | Description |
|---|---|---|
| name | Name of the bridge. | |
| use_dhcp | False | Use DHCP to get an IP address. |
| use_dhcpv6 | False | Use DHCP to get a v6 IP address. |
| addresses | A list of IP addresses assigned to the bridge. | |
| routes | A list of routes assigned to the bridge. For more information, see routes. | |
| mtu | 1500 | The maximum transmission unit (MTU) of the connection. |
| members | A sequence of interface, VLAN, and bond objects that you want to use in the bridge. | |
| defroute | True |
Use a default route provided by the DHCP service. Only applies when you enable |
| persist_mapping | False | Write the device alias configuration instead of the system names. |
| dhclient_args | None | Arguments that you want to pass to the DHCP client. |
| dns_servers | None | List of DNS servers that you want to use for the bridge. |
routes
Defines a list of routes to apply to a network interface, VLAN, bridge, or bond.
For example:
| Option | Default | Description |
|---|---|---|
| ip_netmask | None | IP and netmask of the destination network. |
| default | False |
Sets this route to a default route. Equivalent to setting |
| next_hop | None | The IP address of the router used to reach the destination network. |
10.5. Example network interface layout Copy linkLink copied to clipboard!
The following snippet for an example Controller node NIC template demonstrates how to configure the custom network scenario to keep the control group separate from the OVS bridge:
This template uses three network interfaces and assigns a number of tagged VLAN devices to the numbered interfaces, nic1 to nic3. On nic2 and nic3 this template creates the OVS bridge that hosts the Storage, Tenant, and External networks. As a result, it creates the following layout:
NIC1 (Provisioning)
- Provisioning / Control Plane
NIC2 and NIC3 (Management)
- Internal API
- Storage
- Storage Management
- Tenant Network (VXLAN tunneling)
- Tenant VLANs / Provider VLANs
- External (Public API)
- External VLANs (Floating IP/SNAT)
10.6. Network interface template considerations for custom networks Copy linkLink copied to clipboard!
When you use composable networks, the process-templates.py script renders the static templates to include networks and roles that you define in your network_data.yaml and roles_data.yaml files. Ensure that your rendered NIC templates contain the following items:
- A static file for each role, including custom composable networks.
- The correct network definitions in the static file for each role.
Each static file requires all of the parameter definitions for any custom networks, even if the network is not used on the role. Ensure that the rendered templates contain these parameters. For example, if you add a StorageBackup network only to the Ceph nodes, you must also include this definition in the parameters section in the NIC configuration templates for all roles:
You can also include the parameters definitions for VLAN IDs and/or gateway IP, if necessary:
The IpSubnet parameter for the custom network appears in the parameter definitions for each role. However, since the Ceph role might be the only role that uses the StorageBackup network, only the NIC configuration template for the Ceph role uses the StorageBackup parameters in the network_config section of the template.
10.7. Custom network environment file Copy linkLink copied to clipboard!
The custom network environment file (in this case, /home/stack/templates/custom-network-configuration.yaml) is a heat environment file that describes the overcloud network environment and points to the custom network interface configuration templates. You can define the subnets and VLANs for your network along with IP address ranges. You can then customize these values for the local environment.
The resource_registry section contains references to the custom network interface templates for each node role. Each resource registered uses the following format:
-
OS::TripleO::[ROLE]::Net::SoftwareConfig: [FILE]
[ROLE] is the role name and [FILE] is the respective network interface template for that particular role. For example:
resource_registry: OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/custom-nics/controller.yaml
resource_registry:
OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/custom-nics/controller.yaml
The parameter_defaults section contains a list of parameters that define the network options for each network type.
10.8. Network environment parameters Copy linkLink copied to clipboard!
The following table is a list of parameters that you can use in the parameter_defaults section of a network environment file to override the default parameter values in your NIC templates.
| Parameter | Description | Type |
|---|---|---|
|
| The IP address of the router on the Control Plane, which is used as a default route for roles other than the Controller nodes. Set this value to the undercloud IP if you use IP masquerade instead of a router. | string |
|
|
The CIDR netmask of the IP network used on the Control Plane. If the Control Plane network uses 192.168.24.0/24, the CIDR is | string (though is always a number) |
|
|
The full network and CIDR netmask for a particular network. The default is automatically set to the network | string |
|
|
The IP allocation range for a particular network. The default is automatically set to the network | hash |
|
|
The VLAN ID for a node on a particular network. The default is set automatically to the network | number |
|
|
The router address for a particular network, which you can use as a default route for roles or for routes to other networks. The default is automatically set to the network | string |
|
| A list of DNS servers added to resolv.conf. Usually allows a maximum of 2 servers. | comma delimited list |
|
|
The options for bonding interfaces. For example, | string |
|
|
Legacy value for the name of the external bridge that you want to use for OpenStack Networking (neutron). This value is empty by default, which means that you can define multiple physical bridges in the | string |
|
|
Defines the flat networks that you want to configure in neutron plugins. The default value is | string |
|
|
The logical to physical bridge mappings that you want to use. The default value maps the external bridge on hosts ( | string |
|
|
Defines the interface that you want to bridge onto | string |
|
|
The tenant network type for OpenStack Networking (neutron). To specify multiple values, use a comma separated list. The first type that you specify is used until all available networks are exhausted, then the next type is used. For example, | string |
|
|
The tunnel types for the neutron tenant network. To specify multiple values, use a comma separated string. For example, | string / comma separated list |
|
|
Ranges of GRE tunnel IDs that you want to make available for tenant network allocation. For example, | string |
|
|
Ranges of VXLAN VNI IDs that you want to make available for tenant network allocation. For example, | string |
|
|
Defines whether to enable or completely disable all tunnelled networks. Leave this enabled unless you are sure that you do not want to create tunnelled networks in future. The default value is | Boolean |
|
|
The ML2 and Open vSwitch VLAN mapping range that you want to support. Defaults to permitting any VLAN on the | string |
|
|
The mechanism drivers for the neutron tenant network. The default value is | string / comma separated list |
10.9. Example custom network environment file Copy linkLink copied to clipboard!
The following snippet is an example of an environment file that you can use to enable your NIC templates and set custom parameters.
10.10. Enabling network isolation with custom NICs Copy linkLink copied to clipboard!
To deploy the overcloud with network isolation and custom NIC templates, include all of the relevant networking environment files in the overcloud deployment command.
Procedure
When you run the
openstack overcloud deploycommand, include the following files:-
The custom
network_data.yamlfile. - The rendered file name of the default network isolation.
- The rendered file name of the default network environment file.
- The custom environment network configuration that includes resource references to your custom NIC templates.
- Any additional environment files relevant to your configuration.
-
The custom
For example:
-
Include the
network-isolation.yamlfile first, then thenetwork-environment.yamlfile. The subsequentcustom-network-configuration.yamloverrides theOS::TripleO::[ROLE]::Net::SoftwareConfigresources from the previous two files. -
If you use composable networks, include the
network_data.yamlandroles_data.yamlfiles with this command.