Dieser Inhalt ist in der von Ihnen ausgewählten Sprache nicht verfügbar.
Chapter 21. Multiple networks
21.1. Understanding multiple networks Link kopierenLink in die Zwischenablage kopiert!
In Kubernetes, container networking is delegated to networking plugins that implement the Container Network Interface (CNI).
OpenShift Container Platform uses the Multus CNI plugin to allow chaining of CNI plugins. During cluster installation, you configure your default pod network. The default network handles all ordinary network traffic for the cluster. You can define an additional network based on the available CNI plugins and attach one or more of these networks to your pods. You can define more than one additional network for your cluster, depending on your needs. This gives you flexibility when you configure pods that deliver network functionality, such as switching or routing.
21.1.1. Usage scenarios for an additional network Link kopierenLink in die Zwischenablage kopiert!
You can use an additional network in situations where network isolation is needed, including data plane and control plane separation. Isolating network traffic is useful for the following performance and security reasons:
- Performance
- You can send traffic on two different planes to manage how much traffic is along each plane.
- Security
- You can send sensitive traffic onto a network plane that is managed specifically for security considerations, and you can separate private data that must not be shared between tenants or customers.
All of the pods in the cluster still use the cluster-wide default network to maintain connectivity across the cluster. Every pod has an
eth0
oc exec -it <pod_name> -- ip a
net1
net2
netN
To attach additional network interfaces to a pod, you must create configurations that define how the interfaces are attached. You specify each interface by using a
NetworkAttachmentDefinition
21.1.2. Additional networks in OpenShift Container Platform Link kopierenLink in die Zwischenablage kopiert!
OpenShift Container Platform provides the following CNI plugins for creating additional networks in your cluster:
- bridge: Configure a bridge-based additional network to allow pods on the same host to communicate with each other and the host.
- host-device: Configure a host-device additional network to allow pods access to a physical Ethernet network device on the host system.
- ipvlan: Configure an ipvlan-based additional network to allow pods on a host to communicate with other hosts and pods on those hosts, similar to a macvlan-based additional network. Unlike a macvlan-based additional network, each pod shares the same MAC address as the parent physical network interface.
- macvlan: Configure a macvlan-based additional network to allow pods on a host to communicate with other hosts and pods on those hosts by using a physical network interface. Each pod that is attached to a macvlan-based additional network is provided a unique MAC address.
- SR-IOV: Configure an SR-IOV based additional network to allow pods to attach to a virtual function (VF) interface on SR-IOV capable hardware on the host system.
21.2. Configuring an additional network Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can configure an additional network for your cluster. The following network types are supported:
21.2.1. Approaches to managing an additional network Link kopierenLink in die Zwischenablage kopiert!
You can manage the lifecycle of an additional network in OpenShift Container Platform by using one of two approaches: modifying the Cluster Network Operator (CNO) configuration or applying a YAML manifest. Each approach is mutually exclusive and you can only use one approach for managing an additional network at a time. For either approach, the additional network is managed by a Container Network Interface (CNI) plugin that you configure. The two different approaches are summarized here:
-
Modifying the Cluster Network Operator (CNO) configuration: Configuring additional networks through CNO is only possible for cluster administrators. The CNO automatically creates and manages the object. By using this approach, you can define
NetworkAttachmentDefinitionobjects at install time through configuration of theNetworkAttachmentDefinition.install-config -
Applying a YAML manifest: You can manage the additional network directly by creating an object. Compared to modifying the CNO configuration, this approach gives you more granular control and flexibility when it comes to configuration.
NetworkAttachmentDefinition
When deploying OpenShift Container Platform nodes with multiple network interfaces on Red Hat OpenStack Platform (RHOSP) with OVN Kubernetes, DNS configuration of the secondary interface might take precedence over the DNS configuration of the primary interface. In this case, remove the DNS nameservers for the subnet ID that is attached to the secondary interface:
$ openstack subnet set --dns-nameserver 0.0.0.0 <subnet_id>
21.2.2. IP address assignment for additional networks Link kopierenLink in die Zwischenablage kopiert!
For additional networks, IP addresses can be assigned using an IP Address Management (IPAM) CNI plugin, which supports various assignment methods, including Dynamic Host Configuration Protocol (DHCP) and static assignment.
The DHCP IPAM CNI plugin responsible for dynamic assignment of IP addresses operates with two distinct components:
- CNI Plugin: Responsible for integrating with the Kubernetes networking stack to request and release IP addresses.
- DHCP IPAM CNI Daemon: A listener for DHCP events that coordinates with existing DHCP servers in the environment to handle IP address assignment requests. This daemon is not a DHCP server itself.
For networks requiring
type: dhcp
- A DHCP server is available and running in the environment. The DHCP server is external to the cluster and is expected to be part of the customer’s existing network infrastructure.
- The DHCP server is appropriately configured to serve IP addresses to the nodes.
In cases where a DHCP server is unavailable in the environment, it is recommended to use the Whereabouts IPAM CNI plugin instead. The Whereabouts CNI provides similar IP address management capabilities without the need for an external DHCP server.
Use the Whereabouts CNI plugin when there is no external DHCP server or where static IP address management is preferred. The Whereabouts plugin includes a reconciler daemon to manage stale IP address allocations.
A DHCP lease must be periodically renewed throughout the container’s lifetime, so a separate daemon, the DHCP IPAM CNI Daemon, is required. To deploy the DHCP IPAM CNI daemon, modify the Cluster Network Operator (CNO) configuration to trigger the deployment of this daemon as part of the additional network setup.
21.2.3. Configuration for an additional network attachment Link kopierenLink in die Zwischenablage kopiert!
An additional network is configured by using the
NetworkAttachmentDefinition
k8s.cni.cncf.io
Do not store any sensitive information or a secret in the
NetworkAttachmentDefinition
The configuration for the API is described in the following table:
| Field | Type | Description |
|---|---|---|
|
|
| The name for the additional network. |
|
|
| The namespace that the object is associated with. |
|
|
| The CNI plugin configuration in JSON format. |
21.2.3.1. Configuration of an additional network through the Cluster Network Operator Link kopierenLink in die Zwischenablage kopiert!
The configuration for an additional network attachment is specified as part of the Cluster Network Operator (CNO) configuration.
The following YAML describes the configuration parameters for managing an additional network with the CNO:
Cluster Network Operator configuration
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
name: cluster
spec:
# ...
additionalNetworks:
- name: <name>
namespace: <namespace>
rawCNIConfig: |-
{
...
}
type: Raw
- 1
- An array of one or more additional network configurations.
- 2
- The name for the additional network attachment that you are creating. The name must be unique within the specified
namespace. - 3
- The namespace to create the network attachment in. If you do not specify a value, then the
defaultnamespace is used. - 4
- A CNI plugin configuration in JSON format.
21.2.3.2. Configuration of an additional network from a YAML manifest Link kopierenLink in die Zwischenablage kopiert!
The configuration for an additional network is specified from a YAML configuration file, such as in the following example:
apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: <name>
spec:
config: |-
{
...
}
21.2.4. Configurations for additional network types Link kopierenLink in die Zwischenablage kopiert!
The specific configuration fields for additional networks is described in the following sections.
21.2.4.1. Configuration for a bridge additional network Link kopierenLink in die Zwischenablage kopiert!
The following object describes the configuration parameters for the bridge CNI plugin:
| Field | Type | Description |
|---|---|---|
|
|
| The CNI specification version. The
|
|
|
| The value for the
|
|
|
| The name of the CNI plugin to configure:
|
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. |
|
|
| Optional: Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, it is created. The default value is
|
|
|
| Optional: Set to
|
|
|
| Optional: Set to
|
|
|
| Optional: Set to
|
|
|
| Optional: Set to
|
|
|
| Optional: Set to
|
|
|
| Optional: Set to
|
|
|
| Optional: Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned. |
|
|
| Optional: Indicates whether the default vlan must be preserved on the
|
|
|
| Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
|
|
| Optional: Enables duplicate address detection for the container side
|
|
|
| Optional: Enables mac spoof check, limiting the traffic originating from the container to the mac address of the interface. The default value is
|
The VLAN parameter configures the VLAN tag on the host end of the
veth
vlan_filtering
To configure uplink for a L2 network you need to allow the vlan on the uplink interface by using the following command:
$ bridge vlan add vid VLAN_ID dev DEV
21.2.4.1.1. bridge configuration example Link kopierenLink in die Zwischenablage kopiert!
The following example configures an additional network named
bridge-net
{
"cniVersion": "0.3.1",
"name": "bridge-net",
"type": "bridge",
"isGateway": true,
"vlan": 2,
"ipam": {
"type": "dhcp"
}
}
21.2.4.2. Configuration for a host device additional network Link kopierenLink in die Zwischenablage kopiert!
Specify your network device by setting only one of the following parameters:
device
hwaddr
kernelpath
pciBusID
The following object describes the configuration parameters for the host-device CNI plugin:
| Field | Type | Description |
|---|---|---|
|
|
| The CNI specification version. The
|
|
|
| The value for the
|
|
|
| The name of the CNI plugin to configure:
|
|
|
| Optional: The name of the device, such as
|
|
|
| Optional: The device hardware MAC address. |
|
|
| Optional: The Linux kernel device path, such as
|
|
|
| Optional: The PCI address of the network device, such as
|
21.2.4.2.1. host-device configuration example Link kopierenLink in die Zwischenablage kopiert!
The following example configures an additional network named
hostdev-net
{
"cniVersion": "0.3.1",
"name": "hostdev-net",
"type": "host-device",
"device": "eth1"
}
21.2.4.3. Configuration for an IPVLAN additional network Link kopierenLink in die Zwischenablage kopiert!
The following object describes the configuration parameters for the IPVLAN CNI plugin:
| Field | Type | Description |
|---|---|---|
|
|
| The CNI specification version. The
|
|
|
| The value for the
|
|
|
| The name of the CNI plugin to configure:
|
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. This is required unless the plugin is chained. |
|
|
| Optional: The operating mode for the virtual network. The value must be
|
|
|
| Optional: The Ethernet interface to associate with the network attachment. If a
|
|
|
| Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
-
The object does not allow virtual interfaces to communicate with the
ipvlaninterface. Therefore the container will not be able to reach the host by using themasterinterface. Be sure that the container joins a network that provides connectivity to the host, such as a network supporting the Precision Time Protocol (ipvlan).PTP -
A single interface cannot simultaneously be configured to use both
masterandmacvlan.ipvlan -
For IP allocation schemes that cannot be interface agnostic, the plugin can be chained with an earlier plugin that handles this logic. If the
ipvlanis omitted, then the previous result must contain a single interface name for themasterplugin to enslave. Ifipvlanis omitted, then the previous result is used to configure theipaminterface.ipvlan
21.2.4.3.1. ipvlan configuration example Link kopierenLink in die Zwischenablage kopiert!
The following example configures an additional network named
ipvlan-net
{
"cniVersion": "0.3.1",
"name": "ipvlan-net",
"type": "ipvlan",
"master": "eth1",
"mode": "l3",
"ipam": {
"type": "static",
"addresses": [
{
"address": "192.168.10.10/24"
}
]
}
}
21.2.4.4. Configuration for a MACVLAN additional network Link kopierenLink in die Zwischenablage kopiert!
The following object describes the configuration parameters for the MAC Virtual LAN (MACVLAN) Container Network Interface (CNI) plugin:
| Field | Type | Description |
|---|---|---|
|
|
| The CNI specification version. The
|
|
|
| The value for the
|
|
|
| The name of the CNI plugin to configure:
|
|
|
| The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. |
|
|
| Optional: Configures traffic visibility on the virtual network. Must be either
|
|
|
| Optional: The host network interface to associate with the newly created macvlan interface. If a value is not specified, then the default route interface is used. |
|
|
| Optional: The maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel. |
If you specify the
master
21.2.4.4.1. MACVLAN configuration example Link kopierenLink in die Zwischenablage kopiert!
The following example configures an additional network named
macvlan-net
{
"cniVersion": "0.3.1",
"name": "macvlan-net",
"type": "macvlan",
"master": "eth1",
"mode": "bridge",
"ipam": {
"type": "dhcp"
}
}
21.2.5. Configuration of IP address assignment for an additional network Link kopierenLink in die Zwischenablage kopiert!
The IP address management (IPAM) Container Network Interface (CNI) plugin provides IP addresses for other CNI plugins.
You can use the following IP address assignment types:
- Static assignment.
- Dynamic assignment through a DHCP server. The DHCP server you specify must be reachable from the additional network.
- Dynamic assignment through the Whereabouts IPAM CNI plugin.
21.2.5.1. Static IP address assignment configuration Link kopierenLink in die Zwischenablage kopiert!
The following table describes the configuration for static IP address assignment:
| Field | Type | Description |
|---|---|---|
|
|
| The IPAM address type. The value
|
|
|
| An array of objects specifying IP addresses to assign to the virtual interface. Both IPv4 and IPv6 IP addresses are supported. |
|
|
| An array of objects specifying routes to configure inside the pod. |
|
|
| Optional: An array of objects specifying the DNS configuration. |
The
addresses
| Field | Type | Description |
|---|---|---|
|
|
| An IP address and network prefix that you specify. For example, if you specify
|
|
|
| The default gateway to route egress network traffic to. |
| Field | Type | Description |
|---|---|---|
|
|
| The IP address range in CIDR format, such as
|
|
|
| The gateway where network traffic is routed. |
| Field | Type | Description |
|---|---|---|
|
|
| An array of one or more IP addresses for to send DNS queries to. |
|
|
| The default domain to append to a hostname. For example, if the domain is set to
|
|
|
| An array of domain names to append to an unqualified hostname, such as
|
Static IP address assignment configuration example
{
"ipam": {
"type": "static",
"addresses": [
{
"address": "191.168.1.7/24"
}
]
}
}
21.2.5.2. Dynamic IP address (DHCP) assignment configuration Link kopierenLink in die Zwischenablage kopiert!
The following JSON describes the configuration for dynamic IP address address assignment with DHCP.
A pod obtains its original DHCP lease when it is created. The lease must be periodically renewed by a minimal DHCP server deployment running on the cluster.
To trigger the deployment of the DHCP server, you must create a shim network attachment by editing the Cluster Network Operator configuration, as in the following example:
Example shim network attachment definition
apiVersion: operator.openshift.io/v1
kind: Network
metadata:
name: cluster
spec:
additionalNetworks:
- name: dhcp-shim
namespace: default
type: Raw
rawCNIConfig: |-
{
"name": "dhcp-shim",
"cniVersion": "0.3.1",
"type": "bridge",
"ipam": {
"type": "dhcp"
}
}
# ...
| Field | Type | Description |
|---|---|---|
|
|
| The IPAM address type. The value
|
Dynamic IP address (DHCP) assignment configuration example
{
"ipam": {
"type": "dhcp"
}
}
21.2.5.3. Dynamic IP address assignment configuration with Whereabouts Link kopierenLink in die Zwischenablage kopiert!
The Whereabouts CNI plugin allows the dynamic assignment of an IP address to an additional network without the use of a DHCP server.
The following table describes the configuration for dynamic IP address assignment with Whereabouts:
| Field | Type | Description |
|---|---|---|
|
|
| The IPAM address type. The value
|
|
|
| An IP address and range in CIDR notation. IP addresses are assigned from within this range of addresses. |
|
|
| Optional: A list of zero or more IP addresses and ranges in CIDR notation. IP addresses within an excluded address range are not assigned. |
Dynamic IP address assignment configuration example that uses Whereabouts
{
"ipam": {
"type": "whereabouts",
"range": "192.0.2.192/27",
"exclude": [
"192.0.2.192/30",
"192.0.2.196/32"
]
}
}
21.2.5.4. Creating a Whereabouts reconciler daemon set Link kopierenLink in die Zwischenablage kopiert!
The Whereabouts reconciler is responsible for managing dynamic IP address assignments for the pods within a cluster using the Whereabouts IP Address Management (IPAM) solution. It ensures that each pods gets a unique IP address from the specified IP address range. It also handles IP address releases when pods are deleted or scaled down.
You can also use a
NetworkAttachmentDefinition
The Whereabouts reconciler daemon set is automatically created when you configure an additional network through the Cluster Network Operator. It is not automatically created when you configure an additional network from a YAML manifest.
To trigger the deployment of the Whereabouts reconciler daemonset, you must manually create a
whereabouts-shim
Use the following procedure to deploy the Whereabouts reconciler daemonset.
Procedure
Edit the
custom resource (CR) by running the following command:Network.operator.openshift.io$ oc edit network.operator.openshift.io clusterModify the
parameter in the CR to add theadditionalNetworksnetwork attachment definition. For example:whereabouts-shimapiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: additionalNetworks: - name: whereabouts-shim namespace: default rawCNIConfig: |- { "name": "whereabouts-shim", "cniVersion": "0.3.1", "type": "bridge", "ipam": { "type": "whereabouts" } } type: Raw- Save the file and exit the text editor.
Verify that the
daemon set deployed successfully by running the following command:whereabouts-reconciler$ oc get all -n openshift-multus | grep whereabouts-reconcilerExample output
pod/whereabouts-reconciler-jnp6g 1/1 Running 0 6s pod/whereabouts-reconciler-k76gg 1/1 Running 0 6s pod/whereabouts-reconciler-k86t9 1/1 Running 0 6s pod/whereabouts-reconciler-p4sxw 1/1 Running 0 6s pod/whereabouts-reconciler-rvfdv 1/1 Running 0 6s pod/whereabouts-reconciler-svzw9 1/1 Running 0 6s daemonset.apps/whereabouts-reconciler 6 6 6 6 6 kubernetes.io/os=linux 6s
21.2.6. Creating an additional network attachment with the Cluster Network Operator Link kopierenLink in die Zwischenablage kopiert!
The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the
NetworkAttachmentDefinition
Do not edit the
NetworkAttachmentDefinition
Prerequisites
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
Procedure
Optional: Create the namespace for the additional networks:
$ oc create namespace <namespace_name>To edit the CNO configuration, enter the following command:
$ oc edit networks.operator.openshift.io clusterModify the CR that you are creating by adding the configuration for the additional network that you are creating, as in the following example CR.
apiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: # ... additionalNetworks: - name: tertiary-net namespace: namespace2 type: Raw rawCNIConfig: |- { "cniVersion": "0.3.1", "name": "tertiary-net", "type": "ipvlan", "master": "eth1", "mode": "l2", "ipam": { "type": "static", "addresses": [ { "address": "192.168.1.23/24" } ] } }- Save your changes and quit the text editor to commit your changes.
Verification
Confirm that the CNO created the
object by running the following command. There might be a delay before the CNO creates the object.NetworkAttachmentDefinition$ oc get network-attachment-definitions -n <namespace>where:
<namespace>- Specifies the namespace for the network attachment that you added to the CNO configuration.
Example output
NAME AGE test-network-1 14m
21.2.7. Creating an additional network attachment by applying a YAML manifest Link kopierenLink in die Zwischenablage kopiert!
Prerequisites
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
Procedure
Create a YAML file with your additional network configuration, such as in the following example:
apiVersion: k8s.cni.cncf.io/v1 kind: NetworkAttachmentDefinition metadata: name: next-net spec: config: |- { "cniVersion": "0.3.1", "name": "work-network", "type": "host-device", "device": "eth1", "ipam": { "type": "dhcp" } }To create the additional network, enter the following command:
$ oc apply -f <file>.yamlwhere:
<file>- Specifies the name of the file contained the YAML manifest.
21.3. About virtual routing and forwarding Link kopierenLink in die Zwischenablage kopiert!
21.3.1. About virtual routing and forwarding Link kopierenLink in die Zwischenablage kopiert!
Virtual routing and forwarding (VRF) devices combined with IP rules provide the ability to create virtual routing and forwarding domains. VRF reduces the number of permissions needed by CNF, and provides increased visibility of the network topology of secondary networks. VRF is used to provide multi-tenancy functionality, for example, where each tenant has its own unique routing tables and requires different default gateways.
Processes can bind a socket to the VRF device. Packets through the binded socket use the routing table associated with the VRF device. An important feature of VRF is that it impacts only OSI model layer 3 traffic and above so L2 tools, such as LLDP, are not affected. This allows higher priority IP rules such as policy based routing to take precedence over the VRF device rules directing specific traffic.
21.3.1.1. Benefits of secondary networks for pods for telecommunications operators Link kopierenLink in die Zwischenablage kopiert!
In telecommunications use cases, each CNF can potentially be connected to multiple different networks sharing the same address space. These secondary networks can potentially conflict with the cluster’s main network CIDR. Using the CNI VRF plugin, network functions can be connected to different customers' infrastructure using the same IP address, keeping different customers isolated. IP addresses are overlapped with OpenShift Container Platform IP space. The CNI VRF plugin also reduces the number of permissions needed by CNF and increases the visibility of network topologies of secondary networks.
21.4. Configuring multi-network policy Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can configure multi-network for additional networks. You can specify multi-network policy for SR-IOV and macvlan additional networks. Macvlan additional networks are fully supported. Other types of additional networks, such as ipvlan, are not supported.
Support for configuring multi-network policies for SR-IOV additional networks is a Technology Preview feature and is only supported with kernel network interface cards (NICs). SR-IOV is not supported for Data Plane Development Kit (DPDK) applications.
For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
Configured network policies are ignored in IPv6 networks.
21.4.1. Differences between multi-network policy and network policy Link kopierenLink in die Zwischenablage kopiert!
Although the
MultiNetworkPolicy
NetworkPolicy
You must use the
API:MultiNetworkPolicyapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy-
You must use the resource name when using the CLI to interact with multi-network policies. For example, you can view a multi-network policy object with the
multi-networkpolicycommand whereoc get multi-networkpolicy <name>is the name of a multi-network policy.<name> You can use the
annotation on ak8s.v1.cni.cncf.io/policy-forobject to point to aMultiNetworkPolicy(NAD) custom resource (CR). The NAD CR defines the network to which the policy applies.NetworkAttachmentDefinitionExample multi-network policy that includes the
k8s.v1.cni.cncf.io/policy-forannotationapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>where:
<namespace_name>- Specifies the namespace name.
<network_name>- Specifies the name of a network attachment definition.
21.4.2. Enabling multi-network policy for the cluster Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can enable multi-network policy support on your cluster.
Prerequisites
-
Install the OpenShift CLI ().
oc -
Log in to the cluster with a user with privileges.
cluster-admin
Procedure
Create the
file with the following YAML:multinetwork-enable-patch.yamlapiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: useMultiNetworkPolicy: trueConfigure the cluster to enable multi-network policy:
$ oc patch network.operator.openshift.io cluster --type=merge --patch-file=multinetwork-enable-patch.yamlExample output
network.operator.openshift.io/cluster patched
21.4.3. Working with multi-network policy Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can create, edit, view, and delete multi-network policies.
21.4.3.1. Prerequisites Link kopierenLink in die Zwischenablage kopiert!
- You have enabled multi-network policy support for your cluster.
21.4.3.2. Creating a multi-network policy using the CLI Link kopierenLink in die Zwischenablage kopiert!
To define granular rules describing ingress or egress network traffic allowed for namespaces in your cluster, you can create a multi-network policy.
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace that the multi-network policy applies to.
Procedure
Create a policy rule:
Create a
file:<policy_name>.yaml$ touch <policy_name>.yamlwhere:
<policy_name>- Specifies the multi-network policy file name.
Define a multi-network policy in the file that you just created, such as in the following examples:
Deny ingress from all pods in all namespaces
This is a fundamental policy, blocking all cross-pod networking other than cross-pod traffic allowed by the configuration of other Network Policies.
apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: deny-by-default annotations: k8s.v1.cni.cncf.io/policy-for: <network_name> spec: podSelector: ingress: []where:
<network_name>- Specifies the name of a network attachment definition.
Allow ingress from all pods in the same namespace
apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: allow-same-namespace annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: ingress: - from: - podSelector: {}where:
<network_name>- Specifies the name of a network attachment definition.
Allow ingress traffic to one pod from a particular namespace
This policy allows traffic to pods labelled
from pods running inpod-a.namespace-yapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: allow-traffic-pod annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: pod: pod-a policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: namespace-ywhere:
<network_name>- Specifies the name of a network attachment definition.
Restrict traffic to a service
This policy when applied ensures every pod with both labels
andapp=bookstorecan only be accessed by pods with labelrole=api. In this example the application could be a REST API server, marked with labelsapp=bookstoreandapp=bookstore.role=apiThis example addresses the following use cases:
- Restricting the traffic to a service to only the other microservices that need to use it.
Restricting the connections to a database to only permit the application using it.
apiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: api-allow annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: app: bookstore role: api ingress: - from: - podSelector: matchLabels: app: bookstorewhere:
<network_name>- Specifies the name of a network attachment definition.
To create the multi-network policy object, enter the following command:
$ oc apply -f <policy_name>.yaml -n <namespace>where:
<policy_name>- Specifies the multi-network policy file name.
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Example output
multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created
If you log in to the web console with
cluster-admin
21.4.3.3. Editing a multi-network policy Link kopierenLink in die Zwischenablage kopiert!
You can edit a multi-network policy in a namespace.
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace where the multi-network policy exists.
Procedure
Optional: To list the multi-network policy objects in a namespace, enter the following command:
$ oc get multi-networkpolicywhere:
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Edit the multi-network policy object.
If you saved the multi-network policy definition in a file, edit the file and make any necessary changes, and then enter the following command.
$ oc apply -n <namespace> -f <policy_file>.yamlwhere:
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
<policy_file>- Specifies the name of the file containing the network policy.
If you need to update the multi-network policy object directly, enter the following command:
$ oc edit multi-networkpolicy <policy_name> -n <namespace>where:
<policy_name>- Specifies the name of the network policy.
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Confirm that the multi-network policy object is updated.
$ oc describe multi-networkpolicy <policy_name> -n <namespace>where:
<policy_name>- Specifies the name of the multi-network policy.
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
If you log in to the web console with
cluster-admin
21.4.3.4. Viewing multi-network policies using the CLI Link kopierenLink in die Zwischenablage kopiert!
You can examine the multi-network policies in a namespace.
Prerequisites
-
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace where the multi-network policy exists.
Procedure
List multi-network policies in a namespace:
To view multi-network policy objects defined in a namespace, enter the following command:
$ oc get multi-networkpolicyOptional: To examine a specific multi-network policy, enter the following command:
$ oc describe multi-networkpolicy <policy_name> -n <namespace>where:
<policy_name>- Specifies the name of the multi-network policy to inspect.
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
If you log in to the web console with
cluster-admin
21.4.3.5. Deleting a multi-network policy using the CLI Link kopierenLink in die Zwischenablage kopiert!
You can delete a multi-network policy in a namespace.
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace where the multi-network policy exists.
Procedure
To delete a multi-network policy object, enter the following command:
$ oc delete multi-networkpolicy <policy_name> -n <namespace>where:
<policy_name>- Specifies the name of the multi-network policy.
<namespace>- Optional: Specifies the namespace if the object is defined in a different namespace than the current namespace.
Example output
multinetworkpolicy.k8s.cni.cncf.io/default-deny deleted
If you log in to the web console with
cluster-admin
21.4.3.6. Creating a default deny all multi-network policy Link kopierenLink in die Zwischenablage kopiert!
This is a fundamental policy, blocking all cross-pod networking other than network traffic allowed by the configuration of other deployed network policies. This procedure enforces a default
deny-by-default
If you log in with a user with the
cluster-admin
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace that the multi-network policy applies to.
Procedure
Create the following YAML that defines a
policy to deny ingress from all pods in all namespaces. Save the YAML in thedeny-by-defaultfile:deny-by-default.yamlapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: deny-by-default namespace: default1 annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name>2 spec: podSelector: {}3 ingress: []4 - 1
namespace: defaultdeploys this policy to thedefaultnamespace.- 2
network_name: specifies the name of a network attachment definition.- 3
podSelector:is empty, this means it matches all the pods. Therefore, the policy applies to all pods in the default namespace.- 4
- There are no
ingressrules specified. This causes incoming traffic to be dropped to all pods.
Apply the policy by entering the following command:
$ oc apply -f deny-by-default.yamlExample output
multinetworkpolicy.k8s.cni.cncf.io/deny-by-default created
21.4.3.7. Creating a multi-network policy to allow traffic from external clients Link kopierenLink in die Zwischenablage kopiert!
With the
deny-by-default
app=web
If you log in with a user with the
cluster-admin
Follow this procedure to configure a policy that allows external service from the public Internet directly or by using a Load Balancer to access the pod. Traffic is only allowed to a pod with the label
app=web
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace that the multi-network policy applies to.
Procedure
Create a policy that allows traffic from the public Internet directly or by using a load balancer to access the pod. Save the YAML in the
file:web-allow-external.yamlapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: web-allow-external namespace: default annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: policyTypes: - Ingress podSelector: matchLabels: app: web ingress: - {}Apply the policy by entering the following command:
$ oc apply -f web-allow-external.yamlExample output
multinetworkpolicy.k8s.cni.cncf.io/web-allow-external created
This policy allows traffic from all resources, including external traffic as illustrated in the following diagram:
21.4.3.8. Creating a multi-network policy allowing traffic to an application from all namespaces Link kopierenLink in die Zwischenablage kopiert!
If you log in with a user with the
cluster-admin
Follow this procedure to configure a policy that allows traffic from all pods in all namespaces to a particular application.
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace that the multi-network policy applies to.
Procedure
Create a policy that allows traffic from all pods in all namespaces to a particular application. Save the YAML in the
file:web-allow-all-namespaces.yamlapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: web-allow-all-namespaces namespace: default annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: app: web1 policyTypes: - Ingress ingress: - from: - namespaceSelector: {}2 NoteBy default, if you omit specifying a
it does not select any namespaces, which means the policy allows traffic only from the namespace the network policy is deployed to.namespaceSelectorApply the policy by entering the following command:
$ oc apply -f web-allow-all-namespaces.yamlExample output
multinetworkpolicy.k8s.cni.cncf.io/web-allow-all-namespaces created
Verification
Start a web service in the
namespace by entering the following command:default$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80Run the following command to deploy an
image in thealpinenamespace and to start a shell:secondary$ oc run test-$RANDOM --namespace=secondary --rm -i -t --image=alpine -- shRun the following command in the shell and observe that the request is allowed:
# wget -qO- --timeout=2 http://web.defaultExpected output
<!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
21.4.3.9. Creating a multi-network policy allowing traffic to an application from a namespace Link kopierenLink in die Zwischenablage kopiert!
If you log in with a user with the
cluster-admin
Follow this procedure to configure a policy that allows traffic to a pod with the label
app=web
- Restrict traffic to a production database only to namespaces where production workloads are deployed.
- Enable monitoring tools deployed to a particular namespace to scrape metrics from the current namespace.
Prerequisites
-
Your cluster uses a network plugin that supports objects, such as the OpenShift SDN network provider with
NetworkPolicyset. This mode is the default for OpenShift SDN.mode: NetworkPolicy -
You installed the OpenShift CLI ().
oc -
You are logged in to the cluster with a user with privileges.
cluster-admin - You are working in the namespace that the multi-network policy applies to.
Procedure
Create a policy that allows traffic from all pods in a particular namespaces with a label
. Save the YAML in thepurpose=productionfile:web-allow-prod.yamlapiVersion: k8s.cni.cncf.io/v1beta1 kind: MultiNetworkPolicy metadata: name: web-allow-prod namespace: default annotations: k8s.v1.cni.cncf.io/policy-for:<namespace_name>/<network_name> spec: podSelector: matchLabels: app: web1 policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: purpose: production2 Apply the policy by entering the following command:
$ oc apply -f web-allow-prod.yamlExample output
multinetworkpolicy.k8s.cni.cncf.io/web-allow-prod created
Verification
Start a web service in the
namespace by entering the following command:default$ oc run web --namespace=default --image=nginx --labels="app=web" --expose --port=80Run the following command to create the
namespace:prod$ oc create namespace prodRun the following command to label the
namespace:prod$ oc label namespace/prod purpose=productionRun the following command to create the
namespace:dev$ oc create namespace devRun the following command to label the
namespace:dev$ oc label namespace/dev purpose=testingRun the following command to deploy an
image in thealpinenamespace and to start a shell:dev$ oc run test-$RANDOM --namespace=dev --rm -i -t --image=alpine -- shRun the following command in the shell and observe that the request is blocked:
# wget -qO- --timeout=2 http://web.defaultExpected output
wget: download timed outRun the following command to deploy an
image in thealpinenamespace and start a shell:prod$ oc run test-$RANDOM --namespace=prod --rm -i -t --image=alpine -- shRun the following command in the shell and observe that the request is allowed:
# wget -qO- --timeout=2 http://web.defaultExpected output
<!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> html { color-scheme: light dark; } body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
21.5. Attaching a pod to an additional network Link kopierenLink in die Zwischenablage kopiert!
As a cluster user you can attach a pod to an additional network.
21.5.1. Adding a pod to an additional network Link kopierenLink in die Zwischenablage kopiert!
You can add a pod to an additional network. The pod continues to send normal cluster-related network traffic over the default network.
When a pod is created additional networks are attached to it. However, if a pod already exists, you cannot attach additional networks to it.
The pod must be in the same namespace as the additional network.
Prerequisites
-
Install the OpenShift CLI ().
oc - Log in to the cluster.
Procedure
Add an annotation to the
object. Only one of the following annotation formats can be used:PodTo attach an additional network without any customization, add an annotation with the following format. Replace
with the name of the additional network to associate with the pod:<network>metadata: annotations: k8s.v1.cni.cncf.io/networks: <network>[,<network>,...]1 - 1
- To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.
To attach an additional network with customizations, add an annotation with the following format:
metadata: annotations: k8s.v1.cni.cncf.io/networks: |- [ { "name": "<network>",1 "namespace": "<namespace>",2 "default-route": ["<default-route>"]3 } ]
To create the pod, enter the following command. Replace
with the name of the pod.<name>$ oc create -f <name>.yamlOptional: To Confirm that the annotation exists in the
CR, enter the following command, replacingPodwith the name of the pod.<name>$ oc get pod <name> -o yamlIn the following example, the
pod is attached to theexample-podadditional network:net1$ oc get pod example-pod -o yaml apiVersion: v1 kind: Pod metadata: annotations: k8s.v1.cni.cncf.io/networks: macvlan-bridge k8s.v1.cni.cncf.io/networks-status: |-1 [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.2.14" ], "default": true, "dns": {} },{ "name": "macvlan-bridge", "interface": "net1", "ips": [ "20.2.2.100" ], "mac": "22:2f:60:a5:f8:00", "dns": {} }] name: example-pod namespace: default spec: ... status: ...- 1
- The
k8s.v1.cni.cncf.io/networks-statusparameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.
21.5.1.1. Specifying pod-specific addressing and routing options Link kopierenLink in die Zwischenablage kopiert!
When attaching a pod to an additional network, you may want to specify further properties about that network in a particular pod. This allows you to change some aspects of routing, as well as specify static IP addresses and MAC addresses. To accomplish this, you can use the JSON formatted annotations.
Prerequisites
- The pod must be in the same namespace as the additional network.
-
Install the OpenShift CLI ().
oc - You must log in to the cluster.
Procedure
To add a pod to an additional network while specifying addressing and/or routing options, complete the following steps:
Edit the
resource definition. If you are editing an existingPodresource, run the following command to edit its definition in the default editor. ReplacePodwith the name of the<name>resource to edit.Pod$ oc edit pod <name>In the
resource definition, add thePodparameter to the podk8s.v1.cni.cncf.io/networksmapping. Themetadataaccepts a JSON string of a list of objects that reference the name ofk8s.v1.cni.cncf.io/networkscustom resource (CR) names in addition to specifying additional properties.NetworkAttachmentDefinitionmetadata: annotations: k8s.v1.cni.cncf.io/networks: '[<network>[,<network>,...]]'1 - 1
- Replace
<network>with a JSON object as shown in the following examples. The single quotes are required.
In the following example the annotation specifies which network attachment will have the default route, using the
parameter.default-routeapiVersion: v1 kind: Pod metadata: name: example-pod annotations: k8s.v1.cni.cncf.io/networks: '[ { "name": "net1" }, { "name": "net2",1 "default-route": ["192.0.2.1"]2 }]' spec: containers: - name: example-pod command: ["/bin/bash", "-c", "sleep 2000000000000"] image: centos/tools- 1
- The
namekey is the name of the additional network to associate with the pod. - 2
- The
default-routekey specifies a value of a gateway for traffic to be routed over if no other routing entry is present in the routing table. If more than onedefault-routekey is specified, this will cause the pod to fail to become active.
The default route will cause any traffic that is not specified in other routes to be routed to the gateway.
Setting the default route to an interface other than the default network interface for OpenShift Container Platform may cause traffic that is anticipated for pod-to-pod traffic to be routed over another interface.
To verify the routing properties of a pod, the
oc
ip
$ oc exec -it <pod_name> -- ip route
You may also reference the pod’s
k8s.v1.cni.cncf.io/networks-status
default-route
To set a static IP address or MAC address for a pod you can use the JSON formatted annotations. This requires you create networks that specifically allow for this functionality. This can be specified in a rawCNIConfig for the CNO.
Edit the CNO CR by running the following command:
$ oc edit networks.operator.openshift.io cluster
The following YAML describes the configuration parameters for the CNO:
Cluster Network Operator YAML configuration
name: <name>
namespace: <namespace>
rawCNIConfig: '{
...
}'
type: Raw
- 1
- Specify a name for the additional network attachment that you are creating. The name must be unique within the specified
namespace. - 2
- Specify the namespace to create the network attachment in. If you do not specify a value, then the
defaultnamespace is used. - 3
- Specify the CNI plugin configuration in JSON format, which is based on the following template.
The following object describes the configuration parameters for utilizing static MAC address and IP address using the macvlan CNI plugin:
macvlan CNI plugin JSON configuration object using static IP and MAC address
{
"cniVersion": "0.3.1",
"name": "<name>",
"plugins": [{
"type": "macvlan",
"capabilities": { "ips": true },
"master": "eth0",
"mode": "bridge",
"ipam": {
"type": "static"
}
}, {
"capabilities": { "mac": true },
"type": "tuning"
}]
}
- 1
- Specifies the name for the additional network attachment to create. The name must be unique within the specified
namespace. - 2
- Specifies an array of CNI plugin configurations. The first object specifies a macvlan plugin configuration and the second object specifies a tuning plugin configuration.
- 3
- Specifies that a request is made to enable the static IP address functionality of the CNI plugin runtime configuration capabilities.
- 4
- Specifies the interface that the macvlan plugin uses.
- 5
- Specifies that a request is made to enable the static MAC address functionality of a CNI plugin.
The above network attachment can be referenced in a JSON formatted annotation, along with keys to specify which static IP and MAC address will be assigned to a given pod.
Edit the pod with:
$ oc edit pod <name>
macvlan CNI plugin JSON configuration object using static IP and MAC address
apiVersion: v1
kind: Pod
metadata:
name: example-pod
annotations:
k8s.v1.cni.cncf.io/networks: '[
{
"name": "<name>",
"ips": [ "192.0.2.205/24" ],
"mac": "CA:FE:C0:FF:EE:00"
}
]'
Static IP addresses and MAC addresses do not have to be used at the same time, you may use them individually, or together.
To verify the IP address and MAC properties of a pod with additional networks, use the
oc
$ oc exec -it <pod_name> -- ip a
21.6. Removing a pod from an additional network Link kopierenLink in die Zwischenablage kopiert!
As a cluster user you can remove a pod from an additional network.
21.6.1. Removing a pod from an additional network Link kopierenLink in die Zwischenablage kopiert!
You can remove a pod from an additional network only by deleting the pod.
Prerequisites
- An additional network is attached to the pod.
-
Install the OpenShift CLI ().
oc - Log in to the cluster.
Procedure
To delete the pod, enter the following command:
$ oc delete pod <name> -n <namespace>-
is the name of the pod.
<name> -
is the namespace that contains the pod.
<namespace>
-
21.7. Editing an additional network Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator you can modify the configuration for an existing additional network.
21.7.1. Modifying an additional network attachment definition Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can make changes to an existing additional network. Any existing pods attached to the additional network will not be updated.
Prerequisites
- You have configured an additional network for your cluster.
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
Procedure
To edit an additional network for your cluster, complete the following steps:
Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:
$ oc edit networks.operator.openshift.io cluster-
In the collection, update the additional network with your changes.
additionalNetworks - Save your changes and quit the text editor to commit your changes.
Optional: Confirm that the CNO updated the
object by running the following command. ReplaceNetworkAttachmentDefinitionwith the name of the additional network to display. There might be a delay before the CNO updates the<network-name>object to reflect your changes.NetworkAttachmentDefinition$ oc get network-attachment-definitions <network-name> -o yamlFor example, the following console output displays a
object that is namedNetworkAttachmentDefinition:net1$ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}' { "cniVersion": "0.3.1", "type": "macvlan", "master": "ens5", "mode": "bridge", "ipam": {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }
21.8. Removing an additional network Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator you can remove an additional network attachment.
21.8.1. Removing an additional network attachment definition Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can remove an additional network from your OpenShift Container Platform cluster. The additional network is not removed from any pods it is attached to.
Prerequisites
-
Install the OpenShift CLI ().
oc -
Log in as a user with privileges.
cluster-admin
Procedure
To remove an additional network from your cluster, complete the following steps:
Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:
$ oc edit networks.operator.openshift.io clusterModify the CR by removing the configuration from the
collection for the network attachment definition you are removing.additionalNetworksapiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: additionalNetworks: []1 - 1
- If you are removing the configuration mapping for the only additional network attachment definition in the
additionalNetworkscollection, you must specify an empty collection.
- Save your changes and quit the text editor to commit your changes.
Optional: Confirm that the additional network CR was deleted by running the following command:
$ oc get network-attachment-definition --all-namespaces
21.9. Assigning a secondary network to a VRF Link kopierenLink in die Zwischenablage kopiert!
As a cluster administrator, you can configure an additional network for a virtual routing and forwarding (VRF) domain by using the CNI VRF plugin. The virtual network that this plugin creates is associated with the physical interface that you specify.
Using a secondary network with a VRF instance has the following advantages:
- Workload isolation
- Isolate workload traffic by configuring a VRF instance for the additional network.
- Improved security
- Enable improved security through isolated network paths in the VRF domain.
- Multi-tenancy support
- Support multi-tenancy through network segmentation with a unique routing table in the VRF domain for each tenant.
Applications that use VRFs must bind to a specific device. The common usage is to use the
SO_BINDTODEVICE
SO_BINDTODEVICE
eth1
SO_BINDTODEVICE
CAP_NET_RAW
Using a VRF through the
ip vrf exec
21.9.1. Creating an additional network attachment with the CNI VRF plugin Link kopierenLink in die Zwischenablage kopiert!
The Cluster Network Operator (CNO) manages additional network definitions. When you specify an additional network to create, the CNO creates the
NetworkAttachmentDefinition
Do not edit the
NetworkAttachmentDefinition
To create an additional network attachment with the CNI VRF plugin, perform the following procedure.
Prerequisites
- Install the OpenShift Container Platform CLI (oc).
- Log in to the OpenShift cluster as a user with cluster-admin privileges.
Procedure
Create the
custom resource (CR) for the additional network attachment and insert theNetworkconfiguration for the additional network, as in the following example CR. Save the YAML as the filerawCNIConfig.additional-network-attachment.yamlapiVersion: operator.openshift.io/v1 kind: Network metadata: name: cluster spec: additionalNetworks: - name: test-network-1 namespace: additional-network-1 type: Raw rawCNIConfig: '{ "cniVersion": "0.3.1", "name": "macvlan-vrf", "plugins": [1 { "type": "macvlan", "master": "eth1", "ipam": { "type": "static", "addresses": [ { "address": "191.168.1.23/24" } ] } }, { "type": "vrf",2 "vrfname": "vrf-1",3 "table": 10014 }] }'- 1
pluginsmust be a list. The first item in the list must be the secondary network underpinning the VRF network. The second item in the list is the VRF plugin configuration.- 2
typemust be set tovrf.- 3
vrfnameis the name of the VRF that the interface is assigned to. If it does not exist in the pod, it is created.- 4
- Optional.
tableis the routing table ID. By default, thetableidparameter is used. If it is not specified, the CNI assigns a free routing table ID to the VRF.
NoteVRF functions correctly only when the resource is of type
.netdeviceCreate the
resource:Network$ oc create -f additional-network-attachment.yamlConfirm that the CNO created the
CR by running the following command. ReplaceNetworkAttachmentDefinitionwith the namespace that you specified when configuring the network attachment, for example,<namespace>.additional-network-1$ oc get network-attachment-definitions -n <namespace>Example output
NAME AGE additional-network-1 14mNoteThere might be a delay before the CNO creates the CR.
Verification
Create a pod and assign it to the additional network with the VRF instance:
Create a YAML file that defines the
resource:PodExample
pod-additional-net.yamlfileapiVersion: v1 kind: Pod metadata: name: pod-additional-net annotations: k8s.v1.cni.cncf.io/networks: '[ { "name": "test-network-1"1 } ]' spec: containers: - name: example-pod-1 command: ["/bin/bash", "-c", "sleep 9000000"] image: centos:8- 1
- Specify the name of the additional network with the VRF instance.
Create the
resource by running the following command:Pod$ oc create -f pod-additional-net.yamlExample output
pod/test-pod created
Verify that the pod network attachment is connected to the VRF additional network. Start a remote session with the pod and run the following command:
$ ip vrf showExample output
Name Table ----------------------- vrf-1 1001Confirm that the VRF interface is the controller for the additional interface:
$ ip linkExample output
5: net1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master red state UP mode