Chapter 12. OpenShift SDN default CNI network provider

1. Available only as a Technology Preview feature in OpenShift Container Platform 4.5.
2. Egress firewall is also known as egress network policy in OpenShift SDN. This is not the same as network policy egress.
3. Does not support egress rules and some ipBlock rules.

Procedure

1. Update the NetNamespace object with the egress IP address using the following JSON:

    $ oc patch netnamespace <project_name> --type=merge -p \ 1
     '{
       "egressIPs": [
         "<ip_address>" 2
       ]
     }'

   1

   Specify the name of the project.

   2

   Specify a single egress IP address. Using multiple IP addresses is not supported.

   For example, to assign project1 to an IP address of 192.168.1.100 and project2 to an IP address of 192.168.1.101:

   $ oc patch netnamespace project1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100"]}'
   $ oc patch netnamespace project2 --type=merge -p \
     '{"egressIPs": ["192.168.1.101"]}'

   Note

   Because OpenShift SDN manages the NetNamespace object, you can make changes only by modifying the existing NetNamespace object. Do not create a new NetNamespace object.

2. Indicate which nodes can host egress IP addresses by setting the egressCIDRs parameter for each host using the following JSON:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressCIDRs": [
         "<ip_address_range_1>", "<ip_address_range_2>" 2
       ]
     }'

   1

   Specify a node name.

   2

   Specify one or more IP address ranges in CIDR format.

   For example, to set node1 and node2 to host egress IP addresses in the range 192.168.1.0 to 192.168.1.255:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'
   $ oc patch hostsubnet node2 --type=merge -p \
     '{"egressCIDRs": ["192.168.1.0/24"]}'

   OpenShift Container Platform automatically assigns specific egress IP addresses to available nodes in a balanced way. In this case, it assigns the egress IP address 192.168.1.100 to node1 and the egress IP address 192.168.1.101 to node2 or vice versa.

Procedure

1. Update the NetNamespace object by specifying the following JSON object with the desired IP addresses:

   $ oc patch netnamespace <project> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address>"
         ]
     }'

   1

   Specify the name of the project.

   2

   Specify one or more egress IP addresses. The egressIPs parameter is an array.

   For example, to assign the project1 project to an IP address of 192.168.1.100:

   $ oc patch netnamespace project1 --type=merge \
     -p '{"egressIPs": ["192.168.1.100"]}'

   You can set egressIPs to two or more IP addresses on different nodes to provide high availability. If multiple egress IP addresses are set, pods use the first IP in the list for egress, but if the node hosting that IP address fails, pods switch to using the next IP in the list after a short delay.

   Note

   Because OpenShift SDN manages the NetNamespace object, you can make changes only by modifying the existing NetNamespace object. Do not create a new NetNamespace object.

2. Manually assign the egress IP to the node hosts. Set the egressIPs parameter on the HostSubnet object on the node host. Using the following JSON, include as many IPs as you want to assign to that node host:

   $ oc patch hostsubnet <node_name> --type=merge -p \ 1
     '{
       "egressIPs": [ 2
         "<ip_address_1>",
         "<ip_address_N>"
         ]
     }'

   1

   Specify the name of the node.

   2

   Specify one or more egress IP addresses. The egressIPs field is an array.

   For example, to specify that node1 should have the egress IPs 192.168.1.100, 192.168.1.101, and 192.168.1.102:

   $ oc patch hostsubnet node1 --type=merge -p \
     '{"egressIPs": ["192.168.1.100", "192.168.1.101", "192.168.1.102"]}'

   In the previous example, all egress traffic for project1 will be routed to the node hosting the specified egress IP, and then connected (using NAT) to that IP address.

Procedure

1. Create a policy rule:

   1. Create a <policy-name>.yaml file where <policy-name> describes the egress policy rules.
   2. In the file you created, define an egress policy object.

2. Enter the following command to create the policy object:

   $ oc create -f <policy-name>.yaml -n <project>

   In the following example, a new EgressNetworkPolicy object is created in a project named project1:

   $ oc create -f default-rules.yaml -n project1

   Example output

   egressnetworkpolicy.network.openshift.io/default-rules created

3. Optional: Save the <policy-name>.yaml so that you can make changes later.

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Optional: If you did not save a copy of the EgressNetworkPolicy object when you created the egress network firewall, enter the following command to create a copy.

   $ oc get -n <project> \ 1
     egressnetworkpolicy <name> \ 2
     -o yaml > <filename>.yaml 3

   1

   Replace <project> with the name of the project

   2

   Replace <name> with the name of the object.

   3

   Replace <filename> with the name of the file to save the YAML.

3. Enter the following command to replace the EgressNetworkPolicy object. Replace <filename> with the name of the file containing the updated EgressNetworkPolicy object.

   $ oc replace -f <filename>.yaml

1. Find the name of the EgressNetworkPolicy object for the project. Replace <project> with the name of the project.

   $ oc get -n <project> egressnetworkpolicy

2. Enter the following command to delete the EgressNetworkPolicy object. Replace <project> with the name of the project and <name> with the name of the object.

   $ oc delete -n <project> egressnetworkpolicy <name>

Procedure

1. Create an egress router pod.

2. To ensure that other pods can find the IP address of the egress router pod, create a service to point to the egress router pod, as in the following example:

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-1
   spec:
     ports:
     - name: http
       port: 80
     - name: https
       port: 443
     type: ClusterIP
     selector:
       name: egress-1

   Your pods can now connect to this service. Their connections are redirected to the corresponding ports on the external server, using the reserved egress IP address.

Procedure

1. Create an egress router pod.

2. To ensure that other pods can find the IP address of the egress router pod, create a service to point to the egress router pod, as in the following example:

   apiVersion: v1
   kind: Service
   metadata:
     name: egress-1
   spec:
     ports:
     - name: http-proxy
       port: 8080 1
     type: ClusterIP
     selector:
       name: egress-1

   1

   Ensure the http port is set to 8080.

3. To configure the client pod (not the egress proxy pod) to use the HTTP proxy, set the http_proxy or https_proxy variables:

   apiVersion: v1
   kind: Pod
   metadata:
     name: app-1
     labels:
       name: app-1
   spec:
     containers:
       env:
       - name: http_proxy
         value: http://egress-1:8080/ 1
       - name: https_proxy
         value: http://egress-1:8080/
       ...

   1

   The service created in the previous step.

   Note

   Using the http_proxy and https_proxy environment variables is not necessary for all setups. If the above does not create a working setup, then consult the documentation for the tool or software you are running in the pod.

Procedure

1. Create an egress router pod.

2. Create a service for the egress router pod:

   1. Create a file named egress-router-service.yaml that contains the following YAML. Set spec.ports to the list of ports that you defined previously for the EGRESS_DNS_PROXY_DESTINATION environment variable.

      apiVersion: v1
      kind: Service
      metadata:
        name: egress-dns-svc
      spec:
        ports:
          ...
        type: ClusterIP
        selector:
          name: egress-dns-proxy

      For example:

      apiVersion: v1
      kind: Service
      metadata:
        name: egress-dns-svc
      spec:
        ports:
        - name: con1
          protocol: TCP
          port: 80
          targetPort: 80
        - name: con2
          protocol: TCP
          port: 100
          targetPort: 100
        type: ClusterIP
        selector:
          name: egress-dns-proxy

   2. To create the service, enter the following command:

      $ oc create -f egress-router-service.yaml

      Pods can now connect to this service. The connections are proxied to the corresponding ports on the external server, using the reserved egress IP address.

Procedure

1. Create a file containing the mapping data for the egress router pod, as in the following example:

   # Egress routes for Project "Test", version 3
   
   80   tcp 203.0.113.25
   
   8080 tcp 203.0.113.26 80
   8443 tcp 203.0.113.26 443
   
   # Fallback
   203.0.113.27

   You can put blank lines and comments into this file.

2. Create a ConfigMap object from the file:

   $ oc delete configmap egress-routes --ignore-not-found

   $ oc create configmap egress-routes \
     --from-file=destination=my-egress-destination.txt

   In the previous command, the egress-routes value is the name of the ConfigMap object to create and my-egress-destination.txt is the name of the file that the data is read from.

3. Create an egress router pod definition and specify the configMapKeyRef stanza for the EGRESS_DESTINATION field in the environment stanza:

   ...
   env:
   - name: EGRESS_DESTINATION
     valueFrom:
       configMapKeyRef:
         name: egress-routes
         key: destination
   ...

1. Change your current project to the project that you enabled multicast for. Replace <project> with the project name.

   $ oc project <project>

2. Create a pod to act as a multicast receiver:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: mlistener
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: mlistener
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat hostname && sleep inf"]
         ports:
           - containerPort: 30102
             name: mlistener
             protocol: UDP
   EOF

3. Create a pod to act as a multicast sender:

   $ cat <<EOF| oc create -f -
   apiVersion: v1
   kind: Pod
   metadata:
     name: msender
     labels:
       app: multicast-verify
   spec:
     containers:
       - name: msender
         image: registry.access.redhat.com/ubi8
         command: ["/bin/sh", "-c"]
         args:
           ["dnf -y install socat && sleep inf"]
   EOF

4. Start the multicast listener.

   1. Get the IP address for the Pod:

      $ POD_IP=$(oc get pods mlistener -o jsonpath='{.status.podIP}')

   2. To start the multicast listener, in a new terminal window or tab, enter the following command:

      $ oc exec mlistener -i -t -- \
          socat UDP4-RECVFROM:30102,ip-add-membership=224.1.0.1:$POD_IP,fork EXEC:hostname

5. Start the multicast transmitter.

   1. Get the pod network IP address range:

      $ CIDR=$(oc get Network.config.openshift.io cluster \
          -o jsonpath='{.status.clusterNetwork[0].cidr}')

   2. To send a multicast message, enter the following command:

      $ oc exec msender -i -t -- \
          /bin/bash -c "echo | socat STDIO UDP4-DATAGRAM:224.1.0.1:30102,range=$CIDR,ip-multicast-ttl=64"

      If multicast is working, the previous command returns the following output:

      mlistener

Procedure

1. Use the following command to join projects to an existing project network:

   $ oc adm pod-network join-projects --to=<project1> <project2> <project3>

   Alternatively, instead of specifying specific project names, you can use the --selector=<project_selector> option to specify projects based upon an associated label.

2. Optional: Run the following command to view the pod networks that you have joined together:

   $ oc get netnamespaces

   Projects in the same pod-network have the same network ID in the NETID column.

Procedure

1. Edit the Network.operator.openshift.io custom resource (CR) by running the following command:

   $ oc edit network.operator.openshift.io cluster

2. Modify the kubeProxyConfig parameter in the CR with your changes to the kube-proxy configuration, such as in the following example CR:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     kubeProxyConfig:
       iptablesSyncPeriod: 30s
       proxyArguments:
         iptables-min-sync-period: ["30s"]

3. Save the file and exit the text editor.

   The syntax is validated by the oc command when you save the file and exit the editor. If your modifications contain a syntax error, the editor opens the file and displays an error message.

4. Enter the following command to confirm the configuration update:

   $ oc get networks.operator.openshift.io -o yaml

   Example output

   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: Network
     metadata:
       name: cluster
     spec:
       clusterNetwork:
       - cidr: 10.128.0.0/14
         hostPrefix: 23
       defaultNetwork:
         type: OpenShiftSDN
       kubeProxyConfig:
         iptablesSyncPeriod: 30s
         proxyArguments:
           iptables-min-sync-period:
           - 30s
       serviceNetwork:
       - 172.30.0.0/16
     status: {}
   kind: List

5. Optional: Enter the following command to confirm that the Cluster Network Operator accepted the configuration change:

   $ oc get clusteroperator network

   Example output

   NAME      VERSION     AVAILABLE   PROGRESSING   DEGRADED   SINCE
   network   4.1.0-0.9   True        False         False      1m

   The AVAILABLE field is True when the configuration update is applied successfully.