第5章 Multiple networks
5.1. About using multiple networks
In addition to the default OVN-Kubernetes Container Network Interface (CNI) plugin, the MicroShift Multus CNI is available to chain other CNI plugins. Installing and using MicroShift Multus is optional.
5.1.1. Secondary networks in MicroShift
Using the MicroShift Multus CNI plugin, you can add additional interfaces to pods from other networks. With this configuration, you have flexibility when you configure pods that deliver network functionality, such as switching or routing.
During node installation, the default pod network is configured with default values unless you customize the configuration. The default network handles all ordinary network traffic for the node.
5.1.1.1. Supported secondary networks for network isolation
The following secondary networks are supported in MicroShift 4.17:
- Bridge: Allows pods on the same host to communicate with each other and the host.
IPVLAN: Allows pods on a host to communicate with other hosts.
- This is similar to a MACVLAN-based secondary network.
- Each pod shares the same MAC address as the parent physical network interface, unlike a MACVLAN-based secondary network.
- MACVLAN: Allows pods on a host to communicate with other hosts and the pods on those other hosts by using a physical network interface. Each pod that is attached to a MACVLAN-based secondary network is provided with a unique MAC address.
Setting network policies for secondary networks is not supported.
5.1.1.2. Use case: Secondary networks for network isolation
You can use an secondary network in situations where network isolation is needed, including control plane and data plane separation. For example, you can configure an secondary interface if you want pods to access a network on the host and also communicate with devices deployed to the edge. These edge devices might be on an isolated operator network or are periodically disconnected.
Isolating network traffic is useful for the following performance and security reasons:
- Performance
- You can send traffic on two different planes to manage the amount of traffic on each plane.
- Security
- You can send sensitive traffic onto a network plane that is managed specifically for security considerations, and you can separate private data that must not be shared between tenants or customers.
The Multus CNI plugin is deployed when the MicroShift service starts up. Therefore, a host restart is required if the microshift-multus RPM package is added after MicroShift has started. Restarting ensures that all containers are re-created with Multus annotations.
5.1.1.3. How secondary networks are implemented
All of the pods in the node still use the node-wide default network to maintain connectivity across the node. Every pod has an eth0 interface that is attached to the node-wide pod network.
-
You can view the interfaces for a pod by using the
oc get pod <pod_name> -o=jsonpath='{ .metadata.annotations.k8s\.v1\.cni\.cncf\.io/network-status }'command. -
If you add secondary network interfaces that use the MicroShift Multus CNI, they are named
net1,net2, …,netN. - The CNI configuration is created when the MicroShift Multus DaemonSet starts. This configuration is autogenerated and includes the primary CNI that is the default delegate. For MicroShift, the default CNI is OVN-Kubernetes.
5.1.1.4. How to attach secondary networks to pods
To attach secondary network interfaces to a pod, you must create and apply configurations that define how the interfaces are attached.
- You must configure any secondary networks you want to use. Because of individual differences in networks, no default configuration is provided.
-
You must apply YAML manifest to specify each interface by using a
NetworkAttachmentDefinitioncustom resource (CR). A configuration inside each of these CRs defines how that interface is created. CRI-O must be configured to use Multus. A default configuration is included in the
microshift-multusRPM.- If the Multus CNI is installed on an existing MicroShift instance, the host must be restarted.
- If the Multus CNI is installed alongside MicroShift, you can add CRs and pods and then start the MicroShift service. Restarting the host in this scenario is not needed.
5.1.1.5. Configurations for secondary network types
The specific configuration fields for secondary networks is described in the following sections.
5.1.2. Installing the Multus CNI plugin on a running node
If you want to attach additional networks to a pod for high-performance network configurations, you can install the MicroShift Multus RPM package. After installation, a host restart is required to re-create all the pods with the Multus annotation.
Uninstalling the Multus CNI plugin is not supported.
Prerequisites
- You have root access to the host.
Procedure
Install the Multus RPM package by running the following command:
$ sudo dnf install microshift-multusヒントIf you create your custom resources (CRs) for additional networks now, you can complete your installation and apply configurations with one restart.
To apply the package manifest to an active node, restart the host by running the following command:
$ sudo systemctl restart
Verification
After restarting, ensure that the Multus CNI plugin components are created by running the following command:
$ oc get pod -A | grep multusExample output
openshift-multus dhcp-daemon-ktzqf 1/1 Running 0 45h openshift-multus multus-4frf4 1/1 Running 0 45h
Next steps
- If you have not done so, configure and apply the additional networks you want to use.
- Deploy your applications that use the created CRs.
5.1.3. Configuration for a bridge secondary network
The bridge CNI plugin JSON configuration object describes the configuration parameters for the Bridge CNI plugin. The following table details these parameters:
| Field | Type | Description |
|---|---|---|
|
|
|
The CNI specification version. The |
|
|
|
The name of the CNI plugin to configure: |
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. |
|
|
|
Optional: Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, the bridge interface gets created. The default value is |
|
|
|
Optional: Set to |
|
|
|
Optional: Set to |
|
|
|
Optional: Set to |
|
|
|
Optional: Set to |
|
|
|
Optional: Set to |
|
|
|
Optional: Set to |
|
|
| Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
|
|
|
Optional: Enables duplicate address detection for the container side |
|
|
|
Optional: Enables mac spoof check, limiting the traffic originating from the container to the mac address of the interface. The default value is |
5.1.3.1. Bridge CNI plugin configuration example
The following example configures a secondary network named bridge-conf for use with the MicroShift Multus CNI:
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: bridge-conf
spec:
config: '{
"cniVersion": "0.4.0",
"type": "bridge",
"bridge": "test-bridge",
"mode": "bridge",
"ipam": {
"type": "host-local",
"ranges": [
[
{
"subnet": "10.10.0.0/16",
"rangeStart": "10.10.1.20",
"rangeEnd": "10.10.3.50",
"gateway": "10.10.0.254"
}
]
],
"dataDir": "/var/lib/cni/test-bridge"
}
}'5.1.4. Configuration for an IPVLAN secondary network
The IPVLAN CNI plugin JSON configuration object describes the configuration parameters for the IPVLAN, ipvlan, CNI plugin. The following table details these parameters:
| Field | Type | Description |
|---|---|---|
|
|
|
The CNI specification version. The |
|
|
|
The value for the |
|
|
|
The name of the CNI plugin to configure: |
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. This is required unless the plugin is chained. |
|
|
|
Optional: The operating mode for the virtual network. The value must be |
|
|
|
Optional: The Ethernet interface to associate with the network attachment. If a |
|
|
| Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
|
|
|
Optional: Specifies whether the |
-
The
ipvlanobject does not allow virtual interfaces to communicate with themasterinterface. Therefore the container is not able to reach the host by using theipvlaninterface. Be sure that the container joins a network that provides connectivity to the host, such as a network supporting the Precision Time Protocol (PTP). -
A single
masterinterface cannot simultaneously be configured to use bothmacvlanandipvlan. -
For IP allocation schemes that cannot be interface agnostic, the
ipvlanplugin can be chained with an earlier plugin that handles this logic. If themasteris omitted, then the previous result must contain a single interface name for theipvlanplugin to enslave. Ifipamis omitted, then the previous result is used to configure theipvlaninterface.
5.1.4.1. IPVLAN CNI plugin configuration example
The following example configures a secondary network named ipvlan-net:
{
"cniVersion": "0.3.1",
"name": "ipvlan-net",
"type": "ipvlan",
"master": "eth1",
"linkInContainer": false,
"mode": "l3",
"ipam": {
"type": "static",
"addresses": [
{
"address": "192.168.10.10/24"
}
]
}
}5.1.5. Configuration for a MACVLAN secondary network
The MACVLAN CNI plugin JSON configuration object describes the configuration parameters for the MAC Virtual LAN (MACVLAN) Container Network Interface (CNI) plugin. The following table describes these parameters:
| Field | Type | Description |
|---|---|---|
|
|
|
The CNI specification version. The |
|
|
|
The value for the |
|
|
|
The name of the CNI plugin to configure: |
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. |
|
|
|
Optional: Configures traffic visibility on the virtual network. Must be either |
|
|
| Optional: The host network interface to associate with the newly created macvlan interface. If a value is not specified, then the default route interface is used. |
|
|
| Optional: The maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
|
|
|
Optional: Specifies whether the |
If you specify the master key for the plugin configuration, use a different physical network interface than the one that is associated with your primary network plugin to avoid possible conflicts.
5.1.5.1. MACVLAN CNI plugin configuration example
The following example configures a secondary network named macvlan-net:
{
"cniVersion": "0.3.1",
"name": "macvlan-net",
"type": "macvlan",
"master": "eth1",
"linkInContainer": false,
"mode": "bridge",
"ipam": {
"type": "dhcp"
}
}
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}