Networking

Procedure

1. Create a policy rule:

   1. Create a <policy-name>.yaml file where <policy-name> describes the policy rule.

   2. In the file you just created define a policy object, such as in the following example:

      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: <policy-name> 1
      spec:
        podSelector:
        ingress: []

      1

      Specify a name for the policy object.

2. Run the following command to create the policy object:

   $ oc create -f <policy-name>.yaml -n <project>

   In the following example, a new NetworkPolicy object is created in a project named project1:

   $ oc create -f default-deny.yaml -n project1

   Example output

   networkpolicy "default-deny" created

Procedure

1. Optional: List the current NetworkPolicy objects.

   1. If you want to list the policy objects in a specific namespace, enter the following command. Replace <namespace> with the namespace for a project.

      $ oc get networkpolicy -n <namespace>

   2. If you want to list the policy objects for the entire cluster, enter the following command:

      $ oc get networkpolicy --all-namespaces

2. Edit the NetworkPolicy object.

   1. If you saved the network policy definition in a file, edit the file and make any necessary changes, and then enter the following command. Replace <policy-file> with the name of the file containing the object definition.

      $ oc apply -f <policy-file>.yaml

   2. If you need to update the NetworkPolicy object directly, you can enter the following command. Replace <policy-name> with the name of the NetworkPolicy object and <namespace> with the name of the project where the object exists.

      $ oc edit <policy-name> -n <namespace>

3. Confirm that the NetworkPolicy object is updated. Replace <namespace> with the name of the project where the object exists.

   $ oc get networkpolicy -n <namespace> -o yaml

Procedure

1. Log in as a user with cluster-admin privileges.

2. Generate the default project template:

   $ oc adm create-bootstrap-project-template -o yaml > template.yaml

3. Use a text editor to modify the generated template.yaml file by adding objects or modifying existing objects.

4. The project template must be created in the openshift-config namespace. Load your modified template:

   $ oc create -f template.yaml -n openshift-config

5. Edit the project configuration resource using the web console or CLI.

   • Using the web console:

     1. Navigate to the Administration → Cluster Settings page.
     2. Click Global Configuration to view all configuration resources.
     3. Find the entry for Project and click Edit YAML.

   • Using the CLI:

     1. Edit the project.config.openshift.io/cluster resource:

        $ oc edit project.config.openshift.io/cluster

6. Update the spec section to include the projectRequestTemplate and name parameters, and set the name of your uploaded project template. The default name is project-request.

   Project configuration resource with custom project template

   apiVersion: config.openshift.io/v1
   kind: Project
   metadata:
     ...
   spec:
     projectRequestTemplate:
       name: <template_name>

7. After you save your changes, create a new project to verify that your changes were successfully applied.

Procedure

1. Edit the default template for a new project by running the following command:

   $ oc edit template <project_template> -n openshift-config

   Replace <project_template> with the name of the default template that you configured for your cluster. The default template name is project-request.

2. In the template, add each NetworkPolicy object as an element to the objects parameter. The objects parameter accepts a collection of one or more objects.

   In the following example, the objects parameter collection includes several NetworkPolicy objects:

   objects:
   - apiVersion: networking.k8s.io/v1
     kind: NetworkPolicy
     metadata:
       name: allow-from-same-namespace
     spec:
       podSelector:
       ingress:
       - from:
         - podSelector: {}
   - apiVersion: networking.k8s.io/v1
     kind: NetworkPolicy
     metadata:
       name: allow-from-openshift-ingress
     spec:
       ingress:
       - from:
         - namespaceSelector:
             matchLabels:
               network.openshift.io/policy-group: ingress
       podSelector: {}
       policyTypes:
       - Ingress
   ...

3. Optional: Create a new project to confirm that your network policy objects are created successfully by running the following commands:

   1. Create a new project:

      $ oc new-project <project> 1

      1

      Replace <project> with the name for the project you are creating.

   2. Confirm that the network policy objects in the new project template exist in the new project:

      $ oc get networkpolicy
      NAME                           POD-SELECTOR   AGE
      allow-from-openshift-ingress   <none>         7s
      allow-from-same-namespace      <none>         7s

Procedure

1. Create the following NetworkPolicy objects:

   1. A policy named allow-from-openshift-ingress:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-ingress
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                network.openshift.io/policy-group: ingress
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

   2. A policy named allow-from-openshift-monitoring:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-monitoring
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                network.openshift.io/policy-group: monitoring
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

   3. A policy named allow-same-namespace:

      $ cat << EOF| oc create -f -
      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: allow-same-namespace
      spec:
        podSelector:
        ingress:
        - from:
          - podSelector: {}
      EOF

2. If the default Ingress Controller configuration has the spec.endpointPublishingStrategy: HostNetwork value set, you must apply a label to the default OpenShift Container Platform namespace to allow network traffic between the Ingress Controller and the project:

   1. Determine if your default Ingress Controller uses the HostNetwork endpoint publishing strategy:

      $ oc get --namespace openshift-ingress-operator ingresscontrollers/default \
        --output jsonpath='{.status.endpointPublishingStrategy.type}'

   2. If the previous command reports the endpoint publishing strategy as HostNetwork, set a label on the default namespace:

      $ oc label namespace default 'network.openshift.io/policy-group=ingress'

3. Confirm that the NetworkPolicy object exists in your current project by running the following command:

   $ oc get networkpolicy <policy-name> -o yaml

   In the following example, the allow-from-openshift-ingress NetworkPolicy object is displayed:

   $ oc get -n project1 networkpolicy allow-from-openshift-ingress -o yaml

   Example output

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: allow-from-openshift-ingress
     namespace: project1
   spec:
     ingress:
     - from:
       - namespaceSelector:
           matchLabels:
             network.openshift.io/policy-group: ingress
     podSelector: {}
     policyTypes:
     - Ingress

Procedure

1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition object.

      2

      Specify the namespace where the NetworkAttachmentDefinition object is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the pod, enter the following command. Replace <name> with the name of the pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the pod.

   $ oc get pod <name> -o yaml

   In the following example, the example-pod pod is attached to the net1 additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 1
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the bridge CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "bridge",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "191.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the macvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: SimpleMacvlan
       simpleMacvlanConfig:
         ipamConfig:
           type: static
           staticIPAMConfig:
             addresses:
             - address: 10.1.1.7/24

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the ipvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "ipvlan",
         "master": "eth1",
         "mode": "l2",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "191.168.1.23/24"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the host-device CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "host-device",
         "device": "eth1"
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition object by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:

   $ oc edit networks.operator.openshift.io cluster

2. In the additionalNetworks collection, update the additional network with your changes.
3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO updated the NetworkAttachmentDefinition object by running the following command. Replace <network-name> with the name of the additional network to display. There might be a delay before the CNO updates the NetworkAttachmentDefinition object to reflect your changes.

   $ oc get network-attachment-definitions <network-name> -o yaml

   For example, the following console output displays a NetworkAttachmentDefinition object that is named net1:

   $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
   { "cniVersion": "0.3.1", "type": "macvlan",
   "master": "ens5",
   "mode": "bridge",
   "ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }

1. Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR by removing the configuration from the additionalNetworks collection for the network attachment definition you are removing.

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: [] 1

   1

   If you are removing the configuration mapping for the only additional network attachment definition in the additionalNetworks collection, you must specify an empty collection.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the additional network CR was deleted by running the following command:

   $ oc get network-attachment-definition --all-namespaces

Procedure

1. Create the following PtpConfig CR, and then save the YAML in the <name>-ptp-config.yaml file. Replace <name> with the name for this configuration.

   apiVersion: ptp.openshift.io/v1
   kind: PtpConfig
   metadata:
     name: <name> 1
     namespace: openshift-ptp 2
   spec:
     profile: 3
     - name: "profile1" 4
       interface: "ens787f1" 5
       ptp4lOpts: "-s -2" 6
       phc2sysOpts: "-a -r" 7
     recommend: 8
     - profile: "profile1" 9
       priority: 10 10
       match: 11
       - nodeLabel: "node-role.kubernetes.io/worker" 12
         nodeName: "dev-worker-0" 13

   1

   Specify a name for the PtpConfig CR.

   2

   Specify the namespace where the PTP Operator is installed.

   3

   Specify an array of one or more profile objects.

   4

   Specify the name of a profile object which is used to uniquely identify a profile object.

   5

   Specify the network interface name to use by the ptp4l service, for example ens787f1.

   6

   Specify system config options for the ptp4l service, for example -s -2. This should not include the interface name -i <interface> and service config file -f /etc/ptp4l.conf because these will be automatically appended.

   7

   Specify system config options for the phc2sys service, for example -a -r.

   8

   Specify an array of one or more recommend objects which define rules on how the profile should be applied to nodes.

   9

   Specify the profile object name defined in the profile section.

   10

   Specify the priority with an integer value between 0 and 99. A larger number gets lower priority, so a priority of 99 is lower than a priority of 10. If a node can be matched with multiple profiles according to rules defined in the match field, the profile with the higher priority will be applied to that node.

   11

   Specify match rules with nodeLabel or nodeName.

   12

   Specify nodeLabel with the key of node.Labels from the node object.

   13

   Specify nodeName with node.Name from the node object.

2. Create the CR by running the following command:

   $ oc create -f <filename> 1

   1

   Replace <filename> with the name of the file you created in the previous step.

3. Optional: Check that the PtpConfig profile is applied to nodes that match with nodeLabel or nodeName.

   $ oc get pods -n openshift-ptp -o wide

   Example output

   NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE           NOMINATED NODE   READINESS GATES
   linuxptp-daemon-4xkbb           1/1     Running   0          43m   192.168.111.15   dev-worker-0   <none>           <none>
   linuxptp-daemon-tdspf           1/1     Running   0          43m   192.168.111.11   dev-master-0   <none>           <none>
   ptp-operator-657bbb64c8-2f8sj   1/1     Running   0          43m   10.128.0.116     dev-master-0   <none>           <none>
   
   $ oc logs linuxptp-daemon-4xkbb -n openshift-ptp
   I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
   I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
   I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
   I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: profile1 1
   I1115 09:41:17.117616 4143292 daemon.go:102] Interface: ens787f1    2
   I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -s -2       3
   I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r     4
   I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------
   I1115 09:41:18.117934 4143292 daemon.go:186] Starting phc2sys...
   I1115 09:41:18.117985 4143292 daemon.go:187] phc2sys cmd: &{Path:/usr/sbin/phc2sys Args:[/usr/sbin/phc2sys -a -r] Env:[] Dir: Stdin:<nil> Stdout:<nil> Stderr:<nil> ExtraFiles:[] SysProcAttr:<nil> Process:<nil> ProcessState:<nil> ctx:<nil> lookPathErr:<nil> finished:false childFiles:[] closeAfterStart:[] closeAfterWait:[] goroutine:[] errch:<nil> waitDone:<nil>}
   I1115 09:41:19.118175 4143292 daemon.go:186] Starting ptp4l...
   I1115 09:41:19.118209 4143292 daemon.go:187] ptp4l cmd: &{Path:/usr/sbin/ptp4l Args:[/usr/sbin/ptp4l -m -f /etc/ptp4l.conf -i ens787f1 -s -2] Env:[] Dir: Stdin:<nil> Stdout:<nil> Stderr:<nil> ExtraFiles:[] SysProcAttr:<nil> Process:<nil> ProcessState:<nil> ctx:<nil> lookPathErr:<nil> finished:false childFiles:[] closeAfterStart:[] closeAfterWait:[] goroutine:[] errch:<nil> waitDone:<nil>}
   ptp4l[102189.864]: selected /dev/ptp5 as PTP clock
   ptp4l[102189.886]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
   ptp4l[102189.886]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE

   1

   Profile Name is the name that is applied to node dev-worker-0.

   2

   Interface is the PTP device specified in the profile1 interface field. The ptp4l service runs on this interface.

   3

   Ptp4lOpts are the ptp4l sysconfig options specified in profile1 Ptp4lOpts field.

   4

   Phc2sysOpts are the phc2sys sysconfig options specified in profile1 Phc2sysOpts field.

Procedure

1. Create an SriovNetworkNodePolicy object, and then save the YAML in the <name>-sriov-node-network.yaml file. Replace <name> with the name for this configuration.

2. Create the SriovNetworkNodePolicy CR:

   $ oc create -f <name>-sriov-node-network.yaml

   where <name> specifies the name for this configuration.

   After applying the configuration update, all the pods in sriov-network-operator namespace transition to the Running status.

3. To verify that the SR-IOV network device is configured, enter the following command. Replace <node_name> with the name of a node with the SR-IOV network device that you just configured.

   $ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'

Procedure

1. Create a SriovNetwork object, and then save the YAML in the <name>.yaml file, where <name> is a name for this additional network. The object specification might resemble the following example:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: attach1
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: net1
     networkNamespace: project2
     ipam: |-
     {
       "type": "host-local",
       "subnet": "10.56.217.0/24",
       "rangeStart": "10.56.217.171",
       "rangeEnd": "10.56.217.181",
       "gateway": "10.56.217.1"
     }

2. To create the object, enter the following command:

   $ oc create -f <name>.yaml

   where <name> specifies the name of the additional network.

3. Optional: To confirm that the NetworkAttachmentDefinition object that is associated with the SriovNetwork object that you created in the previous step exists, enter the following command. Replace <namespace> with the networkNamespace you specified in the SriovNetwork object.

   $ oc get net-attach-def -n <namespace>

Procedure

1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition object.

      2

      Specify the namespace where the NetworkAttachmentDefinition object is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the pod, enter the following command. Replace <name> with the name of the pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the pod.

   $ oc get pod <name> -o yaml

   In the following example, the example-pod pod is attached to the net1 additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 1
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the pod. The annotation value is stored as a plain text value.

Procedure

1. Create the following SR-IOV pod spec, and then save the YAML in the <name>-sriov-pod.yaml file. Replace <name> with a name for this pod.

   The following example shows an SR-IOV pod spec:

   apiVersion: v1
   kind: Pod
   metadata:
     name: sample-pod
     annotations:
       k8s.v1.cni.cncf.io/networks: <name> 1
   spec:
     containers:
     - name: sample-container
       image: <image> 2
       command: ["sleep", "infinity"]
       resources:
         limits:
           memory: "1Gi" 3
           cpu: "2" 4
         requests:
           memory: "1Gi"
           cpu: "2"

   1

   Replace <name> with the name of the SR-IOV network attachment definition CR.

   2

   Replace <image> with the name of the sample-pod image.

   3

   To create the SR-IOV pod with guaranteed QoS, set memory limits equal to memory requests.

   4

   To create the SR-IOV pod with guaranteed QoS, set cpu limits equals to cpu requests.

2. Create the sample SR-IOV pod by running the following command:

   $ oc create -f <filename> 1

   1

   Replace <filename> with the name of the file you created in the previous step.

3. Confirm that the sample-pod is configured with guaranteed QoS.

   $ oc describe pod sample-pod

4. Confirm that the sample-pod is allocated with exclusive CPUs.

   $ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus

5. Confirm that the SR-IOV device and CPUs that are allocated for the sample-pod are on the same NUMA node.

   $ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus

Procedure

1. Create a SriovNetworkNodePolicy object:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: policy-example
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: example
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     numVfs: 4
     nicSelector:
       vendor: "8086"
       pfNames: ['ens803f0']
       rootDevices: ['0000:86:00.0']

2. Create a SriovNetwork object:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: net-example
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: default
     ipam: | 1
       {
         "type": "host-local", 2
         "subnet": "10.56.217.0/24",
         "rangeStart": "10.56.217.171",
         "rangeEnd": "10.56.217.181",
         "routes": [
           {"dst": "224.0.0.0/5"},
           {"dst": "232.0.0.0/5"}
         ],
         "gateway": "10.56.217.1"
       }
     resourceName: example

   1 2

   If you choose to configure DHCP as IPAM, ensure that you provision the following default routes through your DHCP server: 224.0.0.0/5 and 232.0.0.0/5. This is to override the static multicast route set by the default network provider.

3. Create a pod with multicast application:

   apiVersion: v1
   kind: Pod
   metadata:
     name: testpmd
     namespace: default
     annotations:
       k8s.v1.cni.cncf.io/networks: nic1
   spec:
     containers:
     - name: example
       image: rhel7:latest
       securityContext:
         capabilities:
           add: ["NET_ADMIN"] 1
       command: [ "sleep", "infinity"]

   1

   The NET_ADMIN capability is required only if your application needs to assign the multicast IP address to the SR-IOV interface. Otherwise, it can be omitted.

Procedure

1. Create the following SriovNetworkNodePolicy object, and then save the YAML in the intel-dpdk-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: intel-dpdk-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: intelnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "8086"
       deviceID: "158b"
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: vfio-pci 1

   1

   Specify the driver type for the virtual functions to vfio-pci.

   Note

   Please refer to the Configuring SR-IOV network devices section for a detailed explanation on each option in SriovNetworkNodePolicy.

   When applying the configuration specified in a SriovNetworkNodePolicy object, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   After the configuration update is applied, all the pods in openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f intel-dpdk-node-policy.yaml

3. Create the following SriovNetwork object, and then save the YAML in the intel-dpdk-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: intel-dpdk-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: "{}" 1
     vlan: <vlan>
     resourceName: intelnics

   1

   Specify an empty object "{}" for the ipam CNI plug-in. DPDK works in userspace mode and does not require an IP address.

   Note

   Please refer to the Configuring SR-IOV additional network section for a detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f intel-dpdk-network.yaml

5. Create the following Pod spec, and then save the YAML in the intel-dpdk-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: dpdk-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: intel-dpdk-network
   spec:
     containers:
     - name: testpmd
       image: <DPDK_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           openshift.io/intelnics: "1" 5
           memory: "1Gi"
           cpu: "4" 6
           hugepages-1Gi: "4Gi" 7
         requests:
           openshift.io/intelnics: "1"
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where the SriovNetwork object intel-dpdk-network is created. If you would like to create the pod in a different namespace, change target_namespace in both the Pod spec and the SriovNetowrk object.

   2

   Specify the DPDK image which includes your application and the DPDK library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside container.

   4

   Mount a hugepage volume to the DPDK pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Optional: Specify the number of DPDK devices allocated to DPDK pod. This resource request and limit, if not explicitly specified, will be automatically added by the SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by the SR-IOV Operator. It is enabled by default and can be disabled by setting enableInjector option to false in the default SriovOperatorConfig CR.

   6

   Specify the number of CPUs. The DPDK pod usually requires exclusive CPUs to be allocated from the kubelet. This is achieved by setting CPU Manager policy to static and creating a pod with Guaranteed QoS.

   7

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to the DPDK pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes. For example, adding kernel arguments default_hugepagesz=1GB, hugepagesz=1G and hugepages=16 will result in 16*1Gi hugepages be allocated during system boot.

6. Create the DPDK pod by running the following command:

   $ oc create -f intel-dpdk-pod.yaml

Procedure

1. Create the following SriovNetworkNodePolicy object, and then save the YAML in the mlx-dpdk-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: mlx-dpdk-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: mlxnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "15b3"
       deviceID: "1015" 1
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: netdevice 2
     isRdma: true 3

   1

   Specify the device hex code of the SR-IOV network device. The only allowed values for Mellanox cards are 1015, 1017.

   2

   Specify the driver type for the virtual functions to netdevice. Mellanox SR-IOV VF can work in DPDK mode without using the vfio-pci device type. VF device appears as a kernel network interface inside a container.

   3

   Enable RDMA mode. This is required by Mellanox cards to work in DPDK mode.

   Note

   Please refer to Configuring SR-IOV network devices section for detailed explanation on each option in SriovNetworkNodePolicy.

   When applying the configuration specified in a SriovNetworkNodePolicy object, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   After the configuration update is applied, all the pods in the openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f mlx-dpdk-node-policy.yaml

3. Create the following SriovNetwork object, and then save the YAML in the mlx-dpdk-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: mlx-dpdk-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: |- 1
       ...
     vlan: <vlan>
     resourceName: mlxnics

   1

   Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.

   Note

   Please refer to Configuring SR-IOV additional network section for detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f mlx-dpdk-network.yaml

5. Create the following Pod spec, and then save the YAML in the mlx-dpdk-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: dpdk-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: mlx-dpdk-network
   spec:
     containers:
     - name: testpmd
       image: <DPDK_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK","NET_RAW"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           openshift.io/mlxnics: "1" 5
           memory: "1Gi"
           cpu: "4" 6
           hugepages-1Gi: "4Gi" 7
         requests:
           openshift.io/mlxnics: "1"
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where SriovNetwork object mlx-dpdk-network is created. If you would like to create the pod in a different namespace, change target_namespace in both Pod spec and SriovNetowrk object.

   2

   Specify the DPDK image which includes your application and the DPDK library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside the container and NET_RAW for the application to access the network interface.

   4

   Mount the hugepage volume to the DPDK pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Optional: Specify the number of DPDK devices allocated to the DPDK pod. This resource request and limit, if not explicitly specified, will be automatically added by SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by SR-IOV Operator. It is enabled by default and can be disabled by setting the enableInjector option to false in the default SriovOperatorConfig CR.

   6

   Specify the number of CPUs. The DPDK pod usually requires exclusive CPUs be allocated from kubelet. This is achieved by setting CPU Manager policy to static and creating a pod with Guaranteed QoS.

   7

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to DPDK pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes.

6. Create the DPDK pod by running the following command:

   $ oc create -f mlx-dpdk-pod.yaml

Procedure

1. Create the following SriovNetworkNodePolicy object, and then save the YAML in the mlx-rdma-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: mlx-rdma-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: mlxnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "15b3"
       deviceID: "1015" 1
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: netdevice 2
     isRdma: true 3

   1

   Specify the device hex code of SR-IOV network device. The only allowed values for Mellanox cards are 1015, 1017.

   2

   Specify the driver type for the virtual functions to netdevice.

   3

   Enable RDMA mode.

   Note

   Please refer to the Configuring SR-IOV network devices section for a detailed explanation on each option in SriovNetworkNodePolicy.

   When applying the configuration specified in a SriovNetworkNodePolicy object, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   After the configuration update is applied, all the pods in the openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f mlx-rdma-node-policy.yaml

3. Create the following SriovNetwork object, and then save the YAML in the mlx-rdma-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: mlx-rdma-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: |- 1
       ...
     vlan: <vlan>
     resourceName: mlxnics

   1

   Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.

   Note

   Please refer to Configuring SR-IOV additional network section for detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy object by running the following command:

   $ oc create -f mlx-rdma-network.yaml

5. Create the following Pod spec, and then save the YAML in the mlx-rdma-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: rdma-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: mlx-rdma-network
   spec:
     containers:
     - name: testpmd
       image: <RDMA_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           memory: "1Gi"
           cpu: "4" 5
           hugepages-1Gi: "4Gi" 6
         requests:
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where SriovNetwork object mlx-rdma-network is created. If you would like to create the pod in a different namespace, change target_namespace in both Pod spec and SriovNetowrk object.

   2

   Specify the RDMA image which includes your application and RDMA library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside the container.

   4

   Mount the hugepage volume to RDMA pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Specify number of CPUs. The RDMA pod usually requires exclusive CPUs be allocated from the kubelet. This is achieved by setting CPU Manager policy to static and create pod with Guaranteed QoS.

   6

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to the RDMA pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes.

6. Create the RDMA pod by running the following command:

   $ oc create -f mlx-rdma-pod.yaml

1. Available only as a Technology Preview feature in OpenShift Container Platform 4.4.
2. Egress firewall is also known as egress network policy in OpenShift SDN. This is not the same as network policy egress.
3. Does not support egress rules and some ipBlock rules.

Procedure

1. Update the NetNamespace object with the egress IP address using the following JSON:

    $ oc patch netnamespace <project_name> --type=merge -p \ 1
     '{
       "egressIPs": [
         "<ip_address>" 2
       ]
     }'

   1

   Specify the name of the project.

   2

   Specify a single egress IP address. Using multiple IP addresses is not supported.

   For example, to assign project1 to an IP address of 192.168.1.100 and project2 to an IP address of 192.168.1.101:

   $ oc patch netnamespace project1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100"]}'
   $ oc patch netnamespace project2 --type=merge -p \
     '{"egressIPs": ["192.168.1.101"]}'

2. Indicate which nodes can host egress IP addresses by setting the egressCIDRs parameter for each host using the following JSON:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressCIDRs": [
         "<ip_address_range_1>", "<ip_address_range_2>" 2
       ]
     }'

   1

   Specify a node name.

   2

   Specify one or more IP address ranges in CIDR format.

   For example, to set node1 and node2 to host egress IP addresses in the range 192.168.1.0 to 192.168.1.255:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'
   $ oc patch hostsubnet node2 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'

   OpenShift Container Platform automatically assigns specific egress IP addresses to available nodes in a balanced way. In this case, it assigns the egress IP address 192.168.1.100 to node1 and the egress IP address 192.168.1.101 to node2 or vice versa.

Procedure

1. Update the NetNamespace object by specifying the following JSON object with the desired IP addresses:

   $ oc patch netnamespace <project> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address>"
         ]
     }'

   1

   Specify the name of the project.

   2

   Specify one or more egress IP addresses. The egressIPs parameter is an array.

   For example, to assign the project1 project to an IP address of 192.168.1.100:

   $ oc patch netnamespace project1 --type=merge \
     -p '{"egressIPs": ["192.168.1.100"]}'

   You can set egressIPs to two or more IP addresses on different nodes to provide high availability. If multiple egress IP addresses are set, pods use the first IP in the list for egress, but if the node hosting that IP address fails, pods switch to using the next IP in the list after a short delay.

2. Manually assign the egress IP to the node hosts. Set the egressIPs parameter on the HostSubnet object on the node host. Using the following JSON, include as many IPs as you want to assign to that node host:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address_1>",
         "<ip_address_N>"
         ]
     }'

   1

   Specify the name of the node.

   2

   Specify one or more egress IP addresses. The egressIPs field is an array.

   For example, to specify that node1 should have the egress IPs 192.168.1.100, 192.168.1.101, and 192.168.1.102:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100", "192.168.1.101", "192.168.1.102"]}'

   In the previous example, all egress traffic for project1 will be routed to the node hosting the specified egress IP, and then connected (using NAT) to that IP address.

Procedure

1. Create a policy rule:

   1. Create a <policy-name>.yaml file where <policy-name> describes the egress policy rules.
   2. In the file you created, define an egress policy object.

2. Enter the following command to create the policy object:

   $ oc create -f <policy-name>.yaml -n <project>

   In the following example, a new EgressNetworkPolicy object is created in a project named project1:

   $ oc create -f default-rules.yaml -n project1

   Example output

   egressnetworkpolicy.network.openshift.io/default-rules created

3. Optional: Save the <policy-name>.yaml so that you can make changes later.

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Optional: If you did not save a copy of the EgressNetworkPolicy object when you created the egress network firewall, enter the following command to create a copy.

   $ oc get -n <project> \ 1
     egressnetworkpolicy <name> \ 2
     -o yaml > <filename>.yaml 3

   1

   Replace <project> with the name of the project

   2

   Replace <name> with the name of the object.

   3

   Replace <filename> with the name of the file to save the YAML.

3. Enter the following command to replace the EgressNetworkPolicy object. Replace <filename> with the name of the file containing the updated EgressNetworkPolicy object.

   $ oc replace -f <filename>.yaml

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Enter the following command to delete the EgressNetworkPolicy object. Replace <project> with the name of the project and <name> with the name of the object.

   $ oc delete -n <project> egressnetworkpolicy <name>

Procedure

1. Create an egress router pod.

2. To ensure that other pods can find the IP address of the egress router pod, create a service to point to the egress router pod, as in the following example:

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-1
   spec:
     ports:
     - name: http
       port: 80
     - name: https
       port: 443
     type: ClusterIP
     selector:
       name: egress-1

   Your pods can now connect to this service. Their connections are redirected to the corresponding ports on the external server, using the reserved egress IP address.

Procedure

1. Create an egress router pod.

2. To ensure that other pods can find the IP address of the egress router pod, create a service to point to the egress router pod, as in the following example:

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-1
   spec:
     ports:
     - name: http-proxy
       port: 8080 1
     type: ClusterIP
     selector:
       name: egress-1

   1

   Ensure the http port is set to 8080.

3. To configure the client pod (not the egress proxy pod) to use the HTTP proxy, set the http_proxy or https_proxy variables:

   apiVersion: v1
   kind: Pod
   metadata:
     name: app-1
     labels:
       name: app-1
   spec:
     containers:
       env:
       - name: http_proxy
         value: http://egress-1:8080/ 1
       - name: https_proxy
         value: http://egress-1:8080/
       ...

   1

   The service created in the previous step.

   Note

   Using the http_proxy and https_proxy environment variables is not necessary for all setups. If the above does not create a working setup, then consult the documentation for the tool or software you are running in the pod.

Procedure

1. Create an egress router pod.

2. Create a service for the egress router pod:

   1. Create a file named egress-router-service.yaml that contains the following YAML. Set spec.ports to the list of ports that you defined previously for the EGRESS_DNS_PROXY_DESTINATION environment variable.

      apiVersion: v1
      kind: Service
      metadata:
        name: egress-dns-svc
      spec:
        ports:
          ...
        type: ClusterIP
        selector:
          name: egress-dns-proxy

      For example:

      apiVersion: v1
      kind: Service
      metadata:
        name: egress-dns-svc
      spec:
        ports:
        - name: con1
          protocol: TCP
          port: 80
          targetPort: 80
        - name: con2
          protocol: TCP
          port: 100
          targetPort: 100
        type: ClusterIP
        selector:
          name: egress-dns-proxy

   2. To create the service, enter the following command:

      $ oc create -f egress-router-service.yaml

      Pods can now connect to this service. The connections are proxied to the corresponding ports on the external server, using the reserved egress IP address.

Procedure

1. Create a file containing the mapping data for the egress router pod, as in the following example:

   # Egress routes for Project "Test", version 3
   
   80   tcp 203.0.113.25
   
   8080 tcp 203.0.113.26 80
   8443 tcp 203.0.113.26 443
   
   # Fallback
   203.0.113.27

   You can put blank lines and comments into this file.

2. Create a ConfigMap object from the file:

   $ oc delete configmap egress-routes --ignore-not-found

   $ oc create configmap egress-routes \
     --from-file=destination=my-egress-destination.txt

   In the previous command, the egress-routes value is the name of the ConfigMap object to create and my-egress-destination.txt is the name of the file that the data is read from.

3. Create an egress router pod definition and specify the configMapKeyRef stanza for the EGRESS_DESTINATION field in the environment stanza:

   ...
   env:
   - name: EGRESS_DESTINATION
     valueFrom:
       configMapKeyRef:
         name: egress-routes
         key: destination
   ...

1. Change your current project to the project that you enabled multicast for. Replace <project> with the project name.

   $ oc project <project>

2. Create a pod to act as a multicast receiver:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: mlistener
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: mlistener
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat hostname && sleep inf"]
         ports:
           - containerPort: 30102
             name: mlistener
             protocol: UDP
   EOF

3. Create a pod to act as a multicast sender:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: msender
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: msender
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat && sleep inf"]
   EOF

4. Start the multicast listener.

   1. Get the IP address for the Pod:

      $ POD_IP=$(oc get pods mlistener -o jsonpath='{.status.podIP}')

   2. To start the multicast listener, in a new terminal window or tab, enter the following command:

      $ oc exec mlistener -i -t -- \
          socat UDP4-RECVFROM:30102,ip-add-membership=224.1.0.1:$POD_IP,fork EXEC:hostname

5. Start the multicast transmitter.

   1. Get the pod network IP address range:

      $ CIDR=$(oc get Network.config.openshift.io cluster \
          -o jsonpath='{.status.clusterNetwork[0].cidr}')

   2. To send a multicast message, enter the following command:

      $ oc exec msender -i -t -- \
          /bin/bash -c "echo | socat STDIO UDP4-DATAGRAM:224.1.0.1:30102,range=$CIDR,ip-multicast-ttl=64"

      If multicast is working, the previous command returns the following output:

      mlistener

Procedure

1. Use the following command to join projects to an existing project network:

   $ oc adm pod-network join-projects --to=<project1> <project2> <project3>

   Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option to specify projects based upon an associated label.

2. Optional: Run the following command to view the pod networks that you have joined together:

   $ oc get netnamespaces

   Projects in the same pod-network have the same network ID in the NETID column.

Procedure

1. Edit the Network.operator.openshift.io custom resource (CR) by running the following command:

   $ oc edit network.operator.openshift.io cluster

2. Modify the kubeProxyConfig parameter in the CR with your changes to the kube-proxy configuration, such as in the following example CR:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     kubeProxyConfig:
       iptablesSyncPeriod: 30s
       proxyArguments:
         iptables-min-sync-period: ["30s"]

3. Save the file and exit the text editor.

   The syntax is validated by the oc command when you save the file and exit the editor. If your modifications contain a syntax error, the editor opens the file and displays an error message.

4. Enter the following command to confirm the configuration update:

   $ oc get networks.operator.openshift.io -o yaml

   Example output

   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: Network
     metadata:
       name: cluster
     spec:
       clusterNetwork:
       - cidr: 10.128.0.0/14
         hostPrefix: 23
       defaultNetwork:
         type: OpenShiftSDN
       kubeProxyConfig:
         iptablesSyncPeriod: 30s
         proxyArguments:
           iptables-min-sync-period:
           - 30s
       serviceNetwork:
       - 172.30.0.0/16
     status: {}
   kind: List

5. Optional: Enter the following command to confirm that the Cluster Network Operator accepted the configuration change:

   $ oc get clusteroperator network

   Example output

   NAME      VERSION     AVAILABLE   PROGRESSING   DEGRADED   SINCE
   network   4.1.0-0.9   True        False         False      1m

   The AVAILABLE field is True when the configuration update is applied successfully.

1. Available only as a Technology Preview feature in OpenShift Container Platform 4.4.
2. Egress firewall is also known as egress network policy in OpenShift SDN. This is not the same as network policy egress.
3. Does not support egress rules and some ipBlock rules.

1. Change your current project to the project that you enabled multicast for. Replace <project> with the project name.

   $ oc project <project>

2. Create a pod to act as a multicast receiver:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: mlistener
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: mlistener
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat hostname && sleep inf"]
         ports:
           - containerPort: 30102
             name: mlistener
             protocol: UDP
   EOF

3. Create a pod to act as a multicast sender:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: msender
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: msender
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat && sleep inf"]
   EOF

4. Start the multicast listener.

   1. Get the IP address for the Pod:

      $ POD_IP=$(oc get pods mlistener -o jsonpath='{.status.podIP}')

   2. To start the multicast listener, in a new terminal window or tab, enter the following command:

      $ oc exec mlistener -i -t -- \
          socat UDP4-RECVFROM:30102,ip-add-membership=224.1.0.1:$POD_IP,fork EXEC:hostname

5. Start the multicast transmitter.

   1. Get the pod network IP address range:

      $ CIDR=$(oc get Network.config.openshift.io cluster \
          -o jsonpath='{.status.clusterNetwork[0].cidr}')

   2. To send a multicast message, enter the following command:

      $ oc exec msender -i -t -- \
          /bin/bash -c "echo | socat STDIO UDP4-DATAGRAM:224.1.0.1:30102,range=$CIDR,ip-multicast-ttl=64"

      If multicast is working, the previous command returns the following output:

      mlistener

Procedure

1. Using the oc annotate command, add the timeout to the route:

   $ oc annotate route <route_name> \
       --overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit> 1

   1

   Supported time units are microseconds (us), milliseconds (ms), seconds (s), minutes (m), hours (h), or days (d).

   The following example sets a timeout of two seconds on a route named myroute:

   $ oc annotate route myroute --overwrite haproxy.router.openshift.io/timeout=2s