Chapter 10. Multiple networks

Procedure

1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition object.

      2

      Specify the namespace where the NetworkAttachmentDefinition object is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the pod, enter the following command. Replace <name> with the name of the pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the pod.

   $ oc get pod <name> -o yaml

   In the following example, the example-pod pod is attached to the net1 additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 1
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the bridge CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "bridge",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "192.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the host-device CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "host-device",
         "device": "eth1",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "192.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the ipvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "ipvlan",
         "master": "eth1",
         "mode": "l2",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "192.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the macvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: SimpleMacvlan
       simpleMacvlanConfig:
         ipamConfig:
           type: static
           staticIPAMConfig:
             addresses:
             - address: 10.1.1.7/24

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the macvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "macvlan",
         "master": "eth1",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "192.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. Replace <namespace> with the namespace that you specified when configuring the network attachment. There might be a delay before the CNO creates the object.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:

   $ oc edit networks.operator.openshift.io cluster

2. In the additionalNetworks collection, update the additional network with your changes.
3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO updated the NetworkAttachmentDefinition object by running the following command. Replace <network-name> with the name of the additional network to display. There might be a delay before the CNO updates the NetworkAttachmentDefinition object to reflect your changes.

   $ oc get network-attachment-definitions <network-name> -o yaml

   For example, the following console output displays a NetworkAttachmentDefinition object that is named net1:

   $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
   { "cniVersion": "0.3.1", "type": "macvlan",
   "master": "ens5",
   "mode": "bridge",
   "ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }

1. Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR by removing the configuration from the additionalNetworks collection for the network attachment definition you are removing.

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: [] 1

   1

   If you are removing the configuration mapping for the only additional network attachment definition in the additionalNetworks collection, you must specify an empty collection.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the additional network CR was deleted by running the following command:

   $ oc get network-attachment-definition --all-namespaces