Networking

Procedure

1. Create a policy rule:

   1. Create a <policy-name>.yaml file where <policy-name> describes the policy rule.

   2. In the file you just created define a policy object, such as in the following example:

      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: <policy-name> 1
      spec:
        podSelector:
        ingress: []

      1

      Specify a name for the policy object.

2. Run the following command to create the policy object:

   $ oc create -f <policy-name>.yaml -n <project>

   In the following example, a new NetworkPolicy object is created in a project named project1:

   $ oc create -f default-deny.yaml -n project1

   Example output

   networkpolicy "default-deny" created

Procedure

1. Optional: List the current NetworkPolicy objects.

   1. If you want to list the policy objects in a specific namespace, enter the following command. Replace <namespace> with the namespace for a project.

      $ oc get networkpolicy -n <namespace>

   2. If you want to list the policy objects for the entire cluster, enter the following command:

      $ oc get networkpolicy --all-namespaces

2. Edit the NetworkPolicy object.

   1. If you saved the NetworkPolicy in a file, edit the file and make any necessary changes, and then enter the following command. Replace <policy-file> with the name of the file containing the object definition.

      $ oc apply -f <policy-file>.yaml

   2. If you need to update the NetworkPolicy object directly, you can enter the following command. Replace <policy-name> with the name of the NetworkPolicy object and <namespace> with the name of the project where the object exists.

      $ oc edit <policy-name> -n <namespace>

3. Confirm that the NetworkPolicy object is updated. Replace <namespace> with the name of the project where the object exists.

   $ oc get networkpolicy -n <namespace> -o yaml

Procedure

1. Log in as a user with cluster-admin privileges.

2. Generate the default project template:

   $ oc adm create-bootstrap-project-template -o yaml > template.yaml

3. Use a text editor to modify the generated template.yaml file by adding objects or modifying existing objects.

4. The project template must be created in the openshift-config namespace. Load your modified template:

   $ oc create -f template.yaml -n openshift-config

5. Edit the project configuration resource using the web console or CLI.

   • Using the web console:

     1. Navigate to the Administration → Cluster Settings page.
     2. Click Global Configuration to view all configuration resources.
     3. Find the entry for Project and click Edit YAML.

   • Using the CLI:

     1. Edit the project.config.openshift.io/cluster resource:

        $ oc edit project.config.openshift.io/cluster

6. Update the spec section to include the projectRequestTemplate and name parameters, and set the name of your uploaded project template. The default name is project-request.

   Project configuration resource with custom project template

   apiVersion: config.openshift.io/v1
   kind: Project
   metadata:
     ...
   spec:
     projectRequestTemplate:
       name: <template_name>

7. After you save your changes, create a new project to verify that your changes were successfully applied.

Procedure

1. Edit the default template for a new project by running the following command:

   $ oc edit template <project_template> -n openshift-config

   Replace <project_template> with the name of the default template that you configured for your cluster. The default template name is project-request.

2. In the template, add each NetworkPolicy object as an element to the objects parameter. The objects parameter accepts a collection of one or more objects.

   In the following example, the objects parameter collection includes several NetworkPolicy objects:

   objects:
   - apiVersion: networking.k8s.io/v1
     kind: NetworkPolicy
     metadata:
       name: allow-from-same-namespace
     spec:
       podSelector:
       ingress:
       - from:
         - podSelector: {}
   - apiVersion: networking.k8s.io/v1
     kind: NetworkPolicy
     metadata:
       name: allow-from-openshift-ingress
     spec:
       ingress:
       - from:
         - namespaceSelector:
             matchLabels:
               network.openshift.io/policy-group: ingress
       podSelector: {}
       policyTypes:
       - Ingress
   ...

3. Optional: Create a new project to confirm that your network policy objects are created successfully by running the following commands:

   1. Create a new project:

      $ oc new-project <project> 1

      1

      Replace <project> with the name for the project you are creating.

   2. Confirm that the network policy objects in the new project template exist in the new project:

      $ oc get networkpolicy
      NAME                           POD-SELECTOR   AGE
      allow-from-openshift-ingress   <none>         7s
      allow-from-same-namespace      <none>         7s

Procedure

1. Create the following NetworkPolicy objects:

   1. A policy named allow-from-openshift-ingress:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-ingress
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                network.openshift.io/policy-group: ingress
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

   2. A policy named allow-from-openshift-monitoring:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-monitoring
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                network.openshift.io/policy-group: monitoring
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

   3. A policy named allow-same-namespace:

      $ cat << EOF| oc create -f -
      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: allow-same-namespace
      spec:
        podSelector:
        ingress:
        - from:
          - podSelector: {}
      EOF

2. If the default Ingress Controller configuration has the spec.endpointPublishingStrategy: HostNetwork value set, you must apply a label to the default OpenShift Container Platform namespace to allow network traffic between the Ingress Controller and the project:

   1. Determine if your default Ingress Controller uses the HostNetwork endpoint publishing strategy:

      $ oc get --namespace openshift-ingress-operator ingresscontrollers/default \
        --output jsonpath='{.status.endpointPublishingStrategy.type}'

   2. If the previous command reports the endpoint publishing strategy as HostNetwork, set a label on the default namespace:

      $ oc label namespace default 'network.openshift.io/policy-group=ingress'

3. Confirm that the NetworkPolicy object exists in your current project by running the following command:

   $ oc get networkpolicy <policy-name> -o yaml

   In the following example, the allow-from-openshift-ingress NetworkPolicy object is displayed:

   $ oc get -n project1 networkpolicy allow-from-openshift-ingress -o yaml

   Example output

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: allow-from-openshift-ingress
     namespace: project1
   spec:
     ingress:
     - from:
       - namespaceSelector:
           matchLabels:
             network.openshift.io/policy-group: ingress
     podSelector: {}
     policyTypes:
     - Ingress

Procedure

1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the Pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that Pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition CR.

      2

      Specify the namespace where the NetworkAttachmentDefinition CR is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the Pod, enter the following command. Replace <name> with the name of the Pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the Pod.

   $ oc get pod <name> -o yaml

   In the following example, the example-pod Pod is attached to the net1 additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 1
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the Pod. The annotation value is stored as a plain text value.

1. Edit the Pod resource definition by running the following command. Replace <name> with the name of the Pod to edit.

   $ oc edit pod <name>

2. Update the annotations mapping to remove the additional network from the Pod by performing one of the following actions:

   • To remove all additional networks from a Pod, remove the k8s.v1.cni.cncf.io/networks parameter from the Pod resource definition as in the following example:

     apiVersion: v1
     kind: Pod
     metadata:
       name: example-pod
       annotations: {}
     spec:
       containers:
       - name: example-pod
         command: ["/bin/bash", "-c", "sleep 2000000000000"]
         image: centos/tools

   • To remove a specific additional network from a Pod, update the k8s.v1.cni.cncf.io/networks parameter by removing the name of the NetworkAttachmentDefinition for the additional network.

3. Optional: Confirm that the Pod is no longer attached to the additional network by running the following command. Replace <name> with the name of the Pod.

   $ oc describe pod <name>

   In the following example, the example-pod Pod is attached only to the default cluster network.

   $ oc describe pod example-pod

   Example output

   Name:               example-pod
   ...
   Annotations:        k8s.v1.cni.cncf.io/networks-status:
                         [{
                             "name": "openshift-sdn",
                             "interface": "eth0",
                             "ips": [
                                 "10.131.0.13"
                             ],
                             "default": true, 1
                             "dns": {}
                         }]
   Status:             Running
   ...

   1

   Only the default cluster network is attached to the Pod.

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the bridge CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "bridge",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "191.168.1.7"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the macvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: SimpleMacvlan
       simpleMacvlanConfig:
         ipamConfig:
           type: static
           staticIPAMConfig:
             addresses:
             - address: 10.1.1.7

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the ipvlan CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "ipvlan",
         "master": "eth1",
         "mode": "l2",
         "ipam": {
           "type": "static",
           "addresses": [
             {
               "address": "191.168.1.7"
             }
           ]
         }
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Edit the CNO CR by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR that you are creating by adding the configuration for the additional network you are creating, as in the following example CR.

   The following YAML configures the host-device CNI plug-in:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: 1
     - name: test-network-1
       namespace: test-1
       type: Raw
       rawCNIConfig: '{
         "cniVersion": "0.3.1",
         "name": "test-network-1",
         "type": "host-device",
         "device": "eth1"
       }'

   1

   Specify the configuration for the additional network attachment definition.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO created the NetworkAttachmentDefinition CR by running the following command. There might be a delay before the CNO creates the CR.

   $ oc get network-attachment-definitions -n <namespace>

   Example output

   NAME                 AGE
   test-network-1       14m

1. Run the following command to edit the Cluster Network Operator (CNO) CR in your default text editor:

   $ oc edit networks.operator.openshift.io cluster

2. In the additionalNetworks collection, update the additional network with your changes.
3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the CNO updated the NetworkAttachmentDefinition CR by running the following command. Replace <network-name> with the name of the additional network to display. There might be a delay before the CNO updates the NetworkAttachmentDefinition CR to reflect your changes.

   $ oc get network-attachment-definitions <network-name> -o yaml

   For example, the following console output displays a NetworkAttachmentDefinition that is named net1:

   $ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
   { "cniVersion": "0.3.1", "type": "macvlan",
   "master": "ens5",
   "mode": "bridge",
   "ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }

1. Edit the Cluster Network Operator (CNO) in your default text editor by running the following command:

   $ oc edit networks.operator.openshift.io cluster

2. Modify the CR by removing the configuration from the additionalNetworks collection for the network attachment definition you are removing.

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     additionalNetworks: [] 1

   1

   If you are removing the configuration mapping for the only additional network attachment definition in the additionalNetworks collection, you must specify an empty collection.

3. Save your changes and quit the text editor to commit your changes.

4. Optional: Confirm that the additional network CR was deleted by running the following command:

   $ oc get network-attachment-definition --all-namespaces

Procedure

1. Create the following PtpConfig CR, and then save the YAML in the <name>-ptp-config.yaml file. Replace <name> with the name for this configuration.

   apiVersion: ptp.openshift.io/v1
   kind: PtpConfig
   metadata:
     name: <name> 1
     namespace: openshift-ptp 2
   spec:
     profile: 3
     - name: "profile1" 4
       interface: "ens787f1" 5
       ptp4lOpts: "-s -2" 6
       phc2sysOpts: "-a -r" 7
     recommend: 8
     - profile: "profile1" 9
       priority: 10 10
       match: 11
       - nodeLabel: "node-role.kubernetes.io/worker" 12
         nodeName: "dev-worker-0" 13

   1

   Specify a name for the PtpConfig CR.

   2

   Specify the namespace where the PTP Operator is installed.

   3

   Specify an array of one or more profile objects.

   4

   Specify the name of a profile object which is used to uniquely identify a profile object.

   5

   Specify the network interface name to use by the ptp4l service, for example ens787f1.

   6

   Specify system config options for the ptp4l service, for example -s -2. This should not include the interface name -i <interface> and service config file -f /etc/ptp4l.conf because these will be automatically appended.

   7

   Specify system config options for the phc2sys service, for example -a -r.

   8

   Specify an array of one or more recommend objects which define rules on how the profile should be applied to nodes.

   9

   Specify the profile object name defined in the profile section.

   10

   Specify the priority with an integer value between 0 and 99. A larger number gets lower priority, so a priority of 99 is lower than a priority of 10. If a node can be matched with multiple profiles according to rules defined in the match field, the profile with the higher priority will be applied to that node.

   11

   Specify match rules with nodeLabel or nodeName.

   12

   Specify nodeLabel with the key of node.Labels from the node object.

   13

   Specify nodeName with node.Name from the node object.

2. Create the CR by running the following command:

   $ oc create -f <filename> 1

   1

   Replace <filename> with the name of the file you created in the previous step.

3. Optional: Check that the PtpConfig profile is applied to nodes that match with nodeLabel or nodeName.

   $ oc get pods -n openshift-ptp -o wide

   Example output

   NAME                            READY   STATUS    RESTARTS   AGE   IP               NODE           NOMINATED NODE   READINESS GATES
   linuxptp-daemon-4xkbb           1/1     Running   0          43m   192.168.111.15   dev-worker-0   <none>           <none>
   linuxptp-daemon-tdspf           1/1     Running   0          43m   192.168.111.11   dev-master-0   <none>           <none>
   ptp-operator-657bbb64c8-2f8sj   1/1     Running   0          43m   10.128.0.116     dev-master-0   <none>           <none>
   
   $ oc logs linuxptp-daemon-4xkbb -n openshift-ptp
   I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
   I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
   I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
   I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: profile1 1
   I1115 09:41:17.117616 4143292 daemon.go:102] Interface: ens787f1    2
   I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -s -2       3
   I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r     4
   I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------
   I1115 09:41:18.117934 4143292 daemon.go:186] Starting phc2sys...
   I1115 09:41:18.117985 4143292 daemon.go:187] phc2sys cmd: &{Path:/usr/sbin/phc2sys Args:[/usr/sbin/phc2sys -a -r] Env:[] Dir: Stdin:<nil> Stdout:<nil> Stderr:<nil> ExtraFiles:[] SysProcAttr:<nil> Process:<nil> ProcessState:<nil> ctx:<nil> lookPathErr:<nil> finished:false childFiles:[] closeAfterStart:[] closeAfterWait:[] goroutine:[] errch:<nil> waitDone:<nil>}
   I1115 09:41:19.118175 4143292 daemon.go:186] Starting ptp4l...
   I1115 09:41:19.118209 4143292 daemon.go:187] ptp4l cmd: &{Path:/usr/sbin/ptp4l Args:[/usr/sbin/ptp4l -m -f /etc/ptp4l.conf -i ens787f1 -s -2] Env:[] Dir: Stdin:<nil> Stdout:<nil> Stderr:<nil> ExtraFiles:[] SysProcAttr:<nil> Process:<nil> ProcessState:<nil> ctx:<nil> lookPathErr:<nil> finished:false childFiles:[] closeAfterStart:[] closeAfterWait:[] goroutine:[] errch:<nil> waitDone:<nil>}
   ptp4l[102189.864]: selected /dev/ptp5 as PTP clock
   ptp4l[102189.886]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
   ptp4l[102189.886]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE

   1

   Profile Name is the name that is applied to node dev-worker-0.

   2

   Interface is the PTP device specified in the profile1 interface field. The ptp4l service runs on this interface.

   3

   Ptp4lOpts are the ptp4l sysconfig options specified in profile1 Ptp4lOpts field.

   4

   Phc2sysOpts are the phc2sys sysconfig options specified in profile1 Phc2sysOpts field.

Procedure

1. Create the following SriovNetworkNodePolicy CR, and then save the YAML in the <name>-sriov-node-network.yaml file. Replace <name> with the name for this configuration.

2. Create the SriovNetworkNodePolicy CR. Replace <name> with the name for this configuration.

   $ oc create -f <name>-sriov-node-network.yaml

   After applying the configuration update, all the Pods in sriov-network-operator namespace transition to the Running status.

3. To verify that the SR-IOV network device is configured, enter the following command. Replace <node_name> with the name of a node with the SR-IOV network device that you just configured.

   $ oc get sriovnetworknodestates -n openshift-sriov-network-operator <node_name> -o jsonpath='{.status.syncStatus}'

Procedure

1. Create the following SriovNetwork CR, and then save the YAML in the <name>-sriov-network.yaml file. Replace <name> with a name for this additional network.

2. To create the CR object, enter the following command. Replace <name> with a name for this additional network.

   $ oc create -f <name>-sriov-network.yaml

3. Optional: To confirm that the NetworkAttachmentDefinition CR associated with the SriovNetwork CR that you created in the previous step exists, enter the following command. Replace <namespace> with the namespace you specified in the SriovNetwork CR.

   $ oc get net-attach-def -n <namespace>

Procedure

1. Add an annotation to the Pod object. Only one of the following annotation formats can be used:

   1. To attach an additional network without any customization, add an annotation with the following format. Replace <network> with the name of the additional network to associate with the Pod:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] 1

      1

      To specify more than one additional network, separate each network with a comma. Do not include whitespace between the comma. If you specify the same additional network multiple times, that Pod will have multiple network interfaces attached to that network.

   2. To attach an additional network with customizations, add an annotation with the following format:

      metadata:
        annotations:
          k8s.v1.cni.cncf.io/networks: |-
            [
              {
                "name": "<network>", 1
                "namespace": "<namespace>", 2
                "default-route": ["<default-route>"] 3
              }
            ]

      1

      Specify the name of the additional network defined by a NetworkAttachmentDefinition CR.

      2

      Specify the namespace where the NetworkAttachmentDefinition CR is defined.

      3

      Optional: Specify an override for the default route, such as 192.168.17.1.

2. To create the Pod, enter the following command. Replace <name> with the name of the Pod.

   $ oc create -f <name>.yaml

3. Optional: To Confirm that the annotation exists in the Pod CR, enter the following command, replacing <name> with the name of the Pod.

   $ oc get pod <name> -o yaml

   In the following example, the example-pod Pod is attached to the net1 additional network:

   $ oc get pod example-pod -o yaml
   apiVersion: v1
   kind: Pod
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/networks: macvlan-bridge
       k8s.v1.cni.cncf.io/networks-status: |- 1
         [{
             "name": "openshift-sdn",
             "interface": "eth0",
             "ips": [
                 "10.128.2.14"
             ],
             "default": true,
             "dns": {}
         },{
             "name": "macvlan-bridge",
             "interface": "net1",
             "ips": [
                 "20.2.2.100"
             ],
             "mac": "22:2f:60:a5:f8:00",
             "dns": {}
         }]
     name: example-pod
     namespace: default
   spec:
     ...
   status:
     ...

   1

   The k8s.v1.cni.cncf.io/networks-status parameter is a JSON array of objects. Each object describes the status of an additional network attached to the Pod. The annotation value is stored as a plain text value.

Procedure

1. Create the following SR-IOV pod spec, and then save the YAML in the <name>-sriov-pod.yaml file. Replace <name> with a name for this pod.

   The following example shows an SR-IOV pod spec:

   apiVersion: v1
   kind: Pod
   metadata:
     name: sample-pod
     annotations:
       k8s.v1.cni.cncf.io/networks: <name> 1
   spec:
     containers:
     - name: sample-container
       image: <image> 2
       command: ["sleep", "infinity"]
       resources:
         limits:
           memory: "1Gi" 3
           cpu: "2" 4
         requests:
           memory: "1Gi"
           cpu: "2"

   1

   Replace <name> with the name of the SR-IOV network attachment definition CR.

   2

   Replace <image> with the name of the sample-pod image.

   3

   To create the SR-IOV pod with guaranteed QoS, set memory limits equal to memory requests.

   4

   To create the SR-IOV pod with guaranteed QoS, set cpu limits equals to cpu requests.

2. Create the sample SR-IOV pod by running the following command:

   $ oc create -f <filename> 1

   1

   Replace <filename> with the name of the file you created in the previous step.

3. Confirm that the sample-pod is configured with guaranteed QoS.

   $ oc describe pod sample-pod

4. Confirm that the sample-pod is allocated with exclusive CPUs.

   $ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus

5. Confirm that the SR-IOV device and CPUs that are allocated for the sample-pod are on the same NUMA node.

   $ oc exec sample-pod -- cat /sys/fs/cgroup/cpuset/cpuset.cpus

Procedure

1. Create a SriovNetworkNodePolicy custom resource (CR):

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: policy-example
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: example
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     numVfs: 4
     nicSelector:
       vendor: "8086"
       pfNames: ['ens803f0']
       rootDevices: ['0000:86:00.0']

2. Create a SriovNetwork CR:

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: net-example
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: default
     ipam: | 1
       {
         "type": "host-local", 2
         "subnet": "10.56.217.0/24",
         "rangeStart": "10.56.217.171",
         "rangeEnd": "10.56.217.181",
         "routes": [
           {"dst": "224.0.0.0/5"},
           {"dst": "232.0.0.0/5"}
         ],
         "gateway": "10.56.217.1"
       }
     resourceName: example

   1 2

   If you choose to configure DHCP as IPAM, ensure that you provision the following default routes through your DHCP server: 224.0.0.0/5 and 232.0.0.0/5. This is to override the static multicast route set by the default network provider.

3. Create a Pod with multicast application:

   apiVersion: v1
   kind: Pod
   metadata:
     name: testpmd
     namespace: default
     annotations:
       k8s.v1.cni.cncf.io/networks: nic1
   spec:
     containers:
     - name: example
       image: rhel7:latest
       securityContext:
         capabilities:
           add: ["NET_ADMIN"] 1
       command: [ "sleep", "infinity"]

   1

   The NET_ADMIN capability is required only if your application needs to assign the multicast IP address to the SR-IOV interface. Otherwise, it can be omitted.

Procedure

1. Create the following SriovNetworkNodePolicy CR, and then save the YAML in the intel-dpdk-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: intel-dpdk-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: intelnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "8086"
       deviceID: "158b"
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: vfio-pci 1

   1

   Specify the driver type for the virtual functions to vfio-pci.

   Note

   Please refer to the Configuring SR-IOV network devices section for a detailed explanation on each option in SriovNetworkNodePolicy.

   + When applying the configuration specified in a SriovNetworkNodePolicy CR, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   + After the configuration update is applied, all the Pods in openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f intel-dpdk-node-policy.yaml

3. Create the following SriovNetwork CR, and then save the YAML in the intel-dpdk-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: intel-dpdk-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: "{}" 1
     vlan: <vlan>
     resourceName: intelnics

   1

   Specify an empty object "{}" for the ipam CNI plug-in. DPDK works in userspace mode and does not require an IP address.

   Note

   Please refer to the Configuring SR-IOV additional network section for a detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f intel-dpdk-network.yaml

5. Create the following Pod spec, and then save the YAML in the intel-dpdk-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: dpdk-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: intel-dpdk-network
   spec:
     containers:
     - name: testpmd
       image: <DPDK_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           openshift.io/intelnics: "1" 5
           memory: "1Gi"
           cpu: "4" 6
           hugepages-1Gi: "4Gi" 7
         requests:
           openshift.io/intelnics: "1"
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where the SriovNetwork CR intel-dpdk-network is created. If you would like to create the Pod in a different namespace, change target_namespace in both the Pod spec and the SriovNetowrk CR.

   2

   Specify the DPDK image which includes your application and the DPDK library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside container.

   4

   Mount a hugepage volume to the DPDK Pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Optional: Specify the number of DPDK devices allocated to DPDK Pod. This resource request and limit, if not explicitly specified, will be automatically added by the SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by the SR-IOV Operator. It is enabled by default and can be disabled by setting enableInjector option to false in the default SriovOperatorConfig CR.

   6

   Specify the number of CPUs. The DPDK Pod usually requires exclusive CPUs to be allocated from the kubelet. This is achieved by setting CPU Manager policy to static and creating a Pod with Guaranteed QoS.

   7

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to the DPDK Pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes. For example, adding kernel arguments default_hugepagesz=1GB, hugepagesz=1G and hugepages=16 will result in 16*1Gi hugepages be allocated during system boot.

6. Create the DPDK Pod by running the following command:

   $ oc create -f intel-dpdk-pod.yaml

Procedure

1. Create the following SriovNetworkNodePolicy CR, and then save the YAML in the mlx-dpdk-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: mlx-dpdk-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: mlxnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "15b3"
       deviceID: "1015" 1
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: netdevice 2
     isRdma: true 3

   1

   Specify the device hex code of the SR-IOV network device. The only allowed values for Mellanox cards are 1015, 1017.

   2

   Specify the driver type for the virtual functions to netdevice. Mellanox SR-IOV VF can work in DPDK mode without using the vfio-pci device type. VF device appears as a kernel network interface inside a container.

   3

   Enable RDMA mode. This is required by Mellanox cards to work in DPDK mode.

   Note

   Please refer to Configuring SR-IOV network devices section for detailed explanation on each option in SriovNetworkNodePolicy.

   + When applying the configuration specified in a SriovNetworkNodePolicy CR, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   + After the configuration update is applied, all the Pods in the openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f mlx-dpdk-node-policy.yaml

3. Create the following SriovNetwork CR, and then save the YAML in the mlx-dpdk-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: mlx-dpdk-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: |- 1
       ...
     vlan: <vlan>
     resourceName: mlxnics

   1

   Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.

   Note

   Please refer to Configuring SR-IOV additional network section for detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f mlx-dpdk-network.yaml

5. Create the following Pod spec, and then save the YAML in the mlx-dpdk-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: dpdk-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: mlx-dpdk-network
   spec:
     containers:
     - name: testpmd
       image: <DPDK_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           openshift.io/mlxnics: "1" 5
           memory: "1Gi"
           cpu: "4" 6
           hugepages-1Gi: "4Gi" 7
         requests:
           openshift.io/mlxnics: "1"
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where SriovNetwork CR mlx-dpdk-network is created. If you would like to create the Pod in a different namespace, change target_namespace in both Pod spec and SriovNetowrk CR.

   2

   Specify the DPDK image which includes your application and the DPDK library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside the container.

   4

   Mount the hugepage volume to the DPDK Pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Optional: Specify the number of DPDK devices allocated to the DPDK Pod. This resource request and limit, if not explicitly specified, will be automatically added by SR-IOV network resource injector. The SR-IOV network resource injector is an admission controller component managed by SR-IOV Operator. It is enabled by default and can be disabled by setting the enableInjector option to false in the default SriovOperatorConfig CR.

   6

   Specify the number of CPUs. The DPDK Pod usually requires exclusive CPUs be allocated from kubelet. This is achieved by setting CPU Manager policy to static and creating a Pod with Guaranteed QoS.

   7

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to DPDK Pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes.

6. Create the DPDK Pod by running the following command:

   $ oc create -f mlx-dpdk-pod.yaml

Procedure

1. Create the following SriovNetworkNodePolicy CR, and then save the YAML in the mlx-rdma-node-policy.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy
   metadata:
     name: mlx-rdma-node-policy
     namespace: openshift-sriov-network-operator
   spec:
     resourceName: mlxnics
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: "true"
     priority: <priority>
     numVfs: <num>
     nicSelector:
       vendor: "15b3"
       deviceID: "1015" 1
       pfNames: ["<pf_name>", ...]
       rootDevices: ["<pci_bus_id>", "..."]
     deviceType: netdevice 2
     isRdma: true 3

   1

   Specify the device hex code of SR-IOV network device. The only allowed values for Mellanox cards are 1015, 1017.

   2

   Specify the driver type for the virtual functions to netdevice.

   3

   Enable RDMA mode.

   Note

   Please refer to the Configuring SR-IOV network devices section for a detailed explanation on each option in SriovNetworkNodePolicy.

   + When applying the configuration specified in a SriovNetworkNodePolicy CR, the SR-IOV Operator may drain the nodes, and in some cases, reboot nodes. It may take several minutes for a configuration change to apply. Ensure that there are enough available nodes in your cluster to handle the evicted workload beforehand.

   + After the configuration update is applied, all the Pods in the openshift-sriov-network-operator namespace will change to a Running status.

2. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f mlx-rdma-node-policy.yaml

3. Create the following SriovNetwork CR, and then save the YAML in the mlx-rdma-network.yaml file.

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetwork
   metadata:
     name: mlx-rdma-network
     namespace: openshift-sriov-network-operator
   spec:
     networkNamespace: <target_namespace>
     ipam: |- 1
       ...
     vlan: <vlan>
     resourceName: mlxnics

   1

   Specify a configuration object for the ipam CNI plug-in as a YAML block scalar. The plug-in manages IP address assignment for the attachment definition.

   Note

   Please refer to Configuring SR-IOV additional network section for detailed explanation on each option in SriovNetwork.

4. Create the SriovNetworkNodePolicy CR by running the following command:

   $ oc create -f mlx-rdma-network.yaml

5. Create the following Pod spec, and then save the YAML in the mlx-rdma-pod.yaml file.

   apiVersion: v1
   kind: Pod
   metadata:
     name: rdma-app
     namespace: <target_namespace> 1
     annotations:
       k8s.v1.cni.cncf.io/networks: mlx-rdma-network
   spec:
     containers:
     - name: testpmd
       image: <RDMA_image> 2
       securityContext:
        capabilities:
           add: ["IPC_LOCK"] 3
       volumeMounts:
       - mountPath: /dev/hugepages 4
         name: hugepage
       resources:
         limits:
           memory: "1Gi"
           cpu: "4" 5
           hugepages-1Gi: "4Gi" 6
         requests:
           memory: "1Gi"
           cpu: "4"
           hugepages-1Gi: "4Gi"
       command: ["sleep", "infinity"]
     volumes:
     - name: hugepage
       emptyDir:
         medium: HugePages

   1

   Specify the same target_namespace where SriovNetwork CR mlx-rdma-network is created. If you would like to create the Pod in a different namespace, change target_namespace in both Pod spec and SriovNetowrk CR.

   2

   Specify the RDMA image which includes your application and RDMA library used by application.

   3

   Specify the IPC_LOCK capability which is required by the application to allocate hugepage memory inside the container.

   4

   Mount the hugepage volume to RDMA Pod under /dev/hugepages. The hugepage volume is backed by the emptyDir volume type with the medium being Hugepages.

   5

   Specify number of CPUs. The RDMA Pod usually requires exclusive CPUs be allocated from the kubelet. This is achieved by setting CPU Manager policy to static and create Pod with Guaranteed QoS.

   6

   Specify hugepage size hugepages-1Gi or hugepages-2Mi and the quantity of hugepages that will be allocated to the RDMA Pod. Configure 2Mi and 1Gi hugepages separately. Configuring 1Gi hugepage requires adding kernel arguments to Nodes.

6. Create the RDMA Pod by running the following command:

   $ oc create -f mlx-rdma-pod.yaml

Procedure

1. Update the NetNamespace resource with the egress IP address using the following JSON:

    $ oc patch netnamespace <project_name> --type=merge -p \ 1
     '{
       "egressIPs": [
         "<ip_address>" 2
       ]
     }'

   1

   Specify the name of the project.

   2

   Specify a single egress IP address. Using multiple IP addresses is not supported.

   For example, to assign project1 to an IP address of 192.168.1.100 and project2 to an IP address of 192.168.1.101:

   $ oc patch netnamespace project1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100"]}'
   $ oc patch netnamespace project2 --type=merge -p \
     '{"egressIPs": ["192.168.1.101"]}'

2. Indicate which nodes can host egress IP addresses by setting the egressCIDRs parameter for each host using the following JSON:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressCIDRs": [
         "<ip_address_range_1>", "<ip_address_range_2>" 2
       ]
     }'

   1

   Specify a node name.

   2

   Specify one or more IP address ranges in CIDR format.

   For example, to set node1 and node2 to host egress IP addresses in the range 192.168.1.0 to 192.168.1.255:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'
   $ oc patch hostsubnet node2 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'

   OpenShift Container Platform automatically assigns specific egress IP addresses to available nodes in a balanced way. In this case, it assigns the egress IP address 192.168.1.100 to node1 and the egress IP address 192.168.1.101 to node2 or vice versa.

Procedure

1. Update the NetNamespace resource by specifying the following JSON object with the desired IP addresses:

   $ oc patch netnamespace <project> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address>"
         ]
     }'

   1

   Specify the name of the project.

   2

   Specify one or more egress IP addresses. The egressIPs parameter is an array.

   For example, to assign the project1 project to an IP address of 192.168.1.100:

   $ oc patch netnamespace project1 --type=merge \
     -p '{"egressIPs": ["192.168.1.100"]}'

   You can set egressIPs to two or more IP addresses on different nodes to provide high availability. If multiple egress IP addresses are set, pods use the first IP in the list for egress, but if the node hosting that IP address fails, pods switch to using the next IP in the list after a short delay.

2. Manually assign the egress IP to the node hosts. Set the egressIPs parameter on the HostSubnet object on the node host. Using the following JSON, include as many IPs as you want to assign to that node host:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address_1>",
         "<ip_address_N>"
         ]
     }'

   1

   Specify the name of the node.

   2

   Specify one or more egress IP addresses. The egressIPs field is an array.

   For example, to specify that node1 should have the egress IPs 192.168.1.100, 192.168.1.101, and 192.168.1.102:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100", "192.168.1.101", "192.168.1.102"]}'

   In the previous example, all egress traffic for project1 will be routed to the node hosting the specified egress IP, and then connected (using NAT) to that IP address.

Procedure

1. Create a policy rule:

   1. Create a <policy-name>.yaml file where <policy-name> describes the egress policy rules.
   2. In the file you created, define an egress policy object.

2. Enter the following command to create the policy object:

   $ oc create -f <policy-name>.yaml -n <project>

   In the following example, a new EgressNetworkPolicy object is created in a project named project1:

   $ oc create -f default-rules.yaml -n project1

   Example output

   egressnetworkpolicy.network.openshift.io/default-rules created

3. Optional: Save the <policy-name>.yaml so that you can make changes later.

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Optional: If you did not save a copy of the EgressNetworkPolicy object when you created the egress network firewall, enter the following command to create a copy.

   $ oc get -n <project> \ 1
     egressnetworkpolicy <name> \ 2
     -o yaml > <filename>.yaml 3

   1

   Replace <project> with the name of the project

   2

   Replace <name> with the name of the object.

   3

   Replace <filename> with the name of the file to save the YAML.

3. Enter the following command to replace the EgressNetworkPolicy object. Replace <filename> with the name of the file containing the updated EgressNetworkPolicy object.

   $ oc replace -f <filename>.yaml

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Enter the following command to delete the EgressNetworkPolicy object. Replace <project> with the name of the project and <name> with the name of the object.

   $ oc delete -n <project> egressnetworkpolicy <name>

Procedure

1. Use the following command to join projects to an existing project network:

   $ oc adm pod-network join-projects --to=<project1> <project2> <project3>

   Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option to specify projects based upon an associated label.

2. Optional: Run the following command to view the pod networks that you have joined together:

   $ oc get netnamespaces

   Projects in the same pod-network have the same network ID in the NETID column.

Procedure

1. Edit the Network.operator.openshift.io Custom Resource (CR) by running the following command:

   $ oc edit network.operator.openshift.io cluster

2. Modify the kubeProxyConfig parameter in the CR with your changes to the kube-proxy configuration, such as in the following example CR:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     kubeProxyConfig:
       iptablesSyncPeriod: 30s
       proxyArguments:
         iptables-min-sync-period: ["30s"]

3. Save the file and exit the text editor.

   The syntax is validated by the oc command when you save the file and exit the editor. If your modifications contain a syntax error, the editor opens the file and displays an error message.

4. Enter the following command to confirm the configuration update:

   $ oc get networks.operator.openshift.io -o yaml

   Example output

   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: Network
     metadata:
       name: cluster
     spec:
       clusterNetwork:
       - cidr: 10.128.0.0/14
         hostPrefix: 23
       defaultNetwork:
         type: OpenShiftSDN
       kubeProxyConfig:
         iptablesSyncPeriod: 30s
         proxyArguments:
           iptables-min-sync-period:
           - 30s
       serviceNetwork:
       - 172.30.0.0/16
     status: {}
   kind: List

5. Optional: Enter the following command to confirm that the Cluster Network Operator accepted the configuration change:

   $ oc get clusteroperator network

   Example output

   NAME      VERSION     AVAILABLE   PROGRESSING   DEGRADED   SINCE
   network   4.1.0-0.9   True        False         False      1m

   The AVAILABLE field is True when the configuration update is applied successfully.

Procedure

1. Using the oc annotate command, add the timeout to the route:

   $ oc annotate route <route_name> \
       --overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit> 1

   1

   Supported time units are microseconds (us), milliseconds (ms), seconds (s), minutes (m), hours (h), or days (d).

   The following example sets a timeout of two seconds on a route named myroute:

   $ oc annotate route myroute --overwrite haproxy.router.openshift.io/timeout=2s

Procedure

1. Create a new project for your service:

   $ oc new-project <project_name>

   For example:

   $ oc new-project myproject

2. Use the oc new-app command to create a service. For example:

   $ oc new-app \
       -e MYSQL_USER=admin \
       -e MYSQL_PASSWORD=redhat \
       -e MYSQL_DATABASE=mysqldb \
       registry.redhat.io/rhscl/mysql-80-rhel7

3. Run the following command to see that the new service is created:

   $ oc get svc -n myproject

   Example output

   NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
   mysql-80-rhel7   ClusterIP   172.30.63.31   <none>        3306/TCP   4m55s

   By default, the new service does not have an external IP address.

1. Log in to OpenShift Container Platform.

2. Log in to the project where the service you want to expose is located:

   $ oc project project1

3. Run the following command to expose the route:

   $ oc expose service <service_name>

   For example:

   $ oc expose service mysql-80-rhel7

   Example output

   route "mysql-80-rhel7" exposed

4. Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service:

   $ curl <pod_ip>:<port>

   For example:

   $ curl 172.30.131.89:3306

   The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connected to the service.

   If you have a MySQL client, log in with the standard CLI command:

   $ mysql -h 172.30.131.89 -u admin -p

   Example output

   Enter password:
   Welcome to the MariaDB monitor.  Commands end with ; or \g.
   
   MySQL [(none)]>

Procedure

1. Edit the router-internal.yaml file:

   # cat router-internal.yaml
   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: sharded
       namespace: openshift-ingress-operator
     spec:
       domain: <apps-sharded.basedomain.example.net>
       nodePlacement:
         nodeSelector:
           matchLabels:
             node-role.kubernetes.io/worker: ""
       routeSelector:
         matchLabels:
           type: sharded
     status: {}
   kind: List
   metadata:
     resourceVersion: ""
     selfLink: ""

2. Apply the Ingress Controller router-internal.yaml file:

   # oc apply -f router-internal.yaml

   The Ingress Controller selects routes in any namespace that have the label type: sharded.

Procedure

1. Edit the router-internal.yaml file:

   # cat router-internal.yaml

   Example output

   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: sharded
       namespace: openshift-ingress-operator
     spec:
       domain: <apps-sharded.basedomain.example.net>
       nodePlacement:
         nodeSelector:
           matchLabels:
             node-role.kubernetes.io/worker: ""
       namespaceSelector:
         matchLabels:
           type: sharded
     status: {}
   kind: List
   metadata:
     resourceVersion: ""
     selfLink: ""

2. Apply the Ingress Controller router-internal.yaml file:

   # oc apply -f router-internal.yaml

   The Ingress Controller selects routes in any namespace that is selected by the namespace selector that have the label type: sharded.

Procedure

1. Create a new project for your service:

   $ oc new-project <project_name>

   For example:

   $ oc new-project myproject

2. Use the oc new-app command to create a service. For example:

   $ oc new-app \
       -e MYSQL_USER=admin \
       -e MYSQL_PASSWORD=redhat \
       -e MYSQL_DATABASE=mysqldb \
       registry.redhat.io/rhscl/mysql-80-rhel7

3. Run the following command to see that the new service is created:

   $ oc get svc -n myproject

   Example output

   NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
   mysql-80-rhel7   ClusterIP   172.30.63.31   <none>        3306/TCP   4m55s

   By default, the new service does not have an external IP address.

1. Log in to OpenShift Container Platform.

2. Log in to the project where the service you want to expose is located:

   $ oc project project1

3. Run the following command to expose the route:

   $ oc expose service <service_name>

   For example:

   $ oc expose service mysql-80-rhel7

   Example output

   route "mysql-80-rhel7" exposed

4. Use a tool, such as cURL, to make sure you can reach the service using the cluster IP address for the service:

   $ curl <pod_ip>:<port>

   For example:

   $ curl 172.30.131.89:3306

   The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connected to the service.

   If you have a MySQL client, log in with the standard CLI command:

   $ mysql -h 172.30.131.89 -u admin -p

   Example output

   Enter password:
   Welcome to the MariaDB monitor.  Commands end with ; or \g.
   
   MySQL [(none)]>

1. Log in to OpenShift Container Platform.

2. Load the project where the service you want to expose is located.

   $ oc project project1

3. Open a text file on the master node and paste the following text, editing the file as needed:

   Sample load balancer configuration file

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-2 1
   spec:
     ports:
     - name: db
       port: 3306 2
     loadBalancerIP:
     type: LoadBalancer 3
     selector:
       name: mysql 4

   1

   Enter a descriptive name for the load balancer service.

   2

   Enter the same port that the service you want to expose is listening on.

   3

   Enter loadbalancer as the type.

   4

   Enter the name of the service.

4. Save and exit the file.

5. Run the following command to create the service:

   $ oc create -f <file-name>

   For example:

   $ oc create -f mysql-lb.yaml

6. Execute the following command to view the new service:

   $ oc get svc

   Example output

   NAME       TYPE           CLUSTER-IP      EXTERNAL-IP                             PORT(S)          AGE
   egress-2   LoadBalancer   172.30.22.226   ad42f5d8b303045-487804948.example.com   3306:30357/TCP   15m

   The service has an external IP address automatically assigned if there is a cloud provider enabled.

7. On the master, use a tool, such as cURL, to make sure you can reach the service using the public IP address:

   $ curl <public-ip>:<port>

   For example:

   $ curl 172.29.121.74:3306

   The examples in this section use a MySQL service, which requires a client application. If you get a string of characters with the Got packets out of order message, you are connecting with the service:

   If you have a MySQL client, log in with the standard CLI command:

   $ mysql -h 172.30.131.89 -u admin -p

   Example output

   Enter password:
   Welcome to the MariaDB monitor.  Commands end with ; or \g.
   
   MySQL [(none)]>