Chapter 20. Multiple networks

Procedure

1. Optional: Create the namespace for the additional networks:

   $ oc create namespace <namespace_name>

2. To edit the CNO configuration, enter the following command:

   $ oc edit networks.operator.openshift.io cluster

3. Modify the CR that you are creating by adding the configuration for the additional network that you are creating, as in the following example CR.

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     # ...
     additionalNetworks:
     - name: tertiary-net
       namespace: namespace2
       type: Raw
       rawCNIConfig: |-
         {
           "cniVersion": "0.3.1",
           "name": "tertiary-net",
           "type": "ipvlan",
           "master": "eth1",
           "mode": "l2",
           "ipam": {
             "type": "static",
             "addresses": [
               {
                 "address": "192.168.1.23/24"
               }
             ]
           }
         }

4. Save your changes and quit the text editor to commit your changes.

Procedure

1. Create a YAML file with your additional network configuration, such as in the following example:

   apiVersion: k8s.cni.cncf.io/v1
   kind: NetworkAttachmentDefinition
   metadata:
     name: next-net
   spec:
     config: |-
       {
         "cniVersion": "0.3.1",
         "name": "work-network",
         "type": "host-device",
         "device": "eth1",
         "ipam": {
           "type": "dhcp"
         }
       }

2. To create the additional network, enter the following command:

   $ oc apply -f <file>.yaml

   where:

   <file>

   Specifies the name of the file contained the YAML manifest.

Procedure

1. Create the

   multinetwork-enable-patch.yaml

   file with the following YAML:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     useMultiNetworkPolicy: true

2. Configure the cluster to enable multi-network policy:

   $ oc patch network.operator.openshift.io cluster --type=merge --patch-file=multinetwork-enable-patch.yaml

   Example output

   network.operator.openshift.io/cluster patched

Procedure

1. Add an annotation to the

   Pod

   object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace

      <network>

      with the name of the additional network to associate with the pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 

      1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 

      1

      
                "namespace": "<namespace>", 

      2

      
                "default-route": ["<default-route>"] 

      3

      
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition object.

      2

      Specify the namespace where the NetworkAttachmentDefinition object is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the pod, enter the following command. Replace

   <name>

   with the name of the pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the

   Pod

   CR, enter the following command, replacing

   <name>

   with the name of the pod.

   $ oc get pod <name> -o yaml

   In the following example, the

   example-pod

   pod is attached to the

   net1

   additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 

   1

   
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.

1. Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:

   $ oc edit networks.operator.openshift.io cluster

2. In the

   additionalNetworks

   collection, update the additional network with your changes.
3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO updated the

   NetworkAttachmentDefinition

   object by running the following command. Replace

   <network-name>

   with the name of the additional network to display. There might be a delay before the CNO updates the

   NetworkAttachmentDefinition

   object to reflect your changes.

   $ oc get network-attachment-definitions <network-name> -o yaml

   For example, the following console output displays a

   NetworkAttachmentDefinition

   object that is named

   net1

   :

   $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
   { "cniVersion": "0.3.1", "type": "macvlan",
   "master": "ens5",
   "mode": "bridge",
   "ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }

1. Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR by removing the configuration from the

   additionalNetworks

   collection for the network attachment definition you are removing.

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: [] 

   1

   1

   If you are removing the configuration mapping for the only additional network attachment definition in the additionalNetworks collection, you must specify an empty collection.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the additional network CR was deleted by running the following command:

   $ oc get network-attachment-definition --all-namespaces