Chapter 7. Post-installation network configuration

Procedure

1. Create the following NetworkPolicy objects:

   1. A policy named allow-from-openshift-ingress.

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-ingress
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                policy-group.network.openshift.io/ingress: ""
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

      Note

      policy-group.network.openshift.io/ingress: "" is the preferred namespace selector label for OpenShift SDN. You can use the network.openshift.io/policy-group: ingress namespace selector label, but this is a legacy label.

   2. A policy named allow-from-openshift-monitoring:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-openshift-monitoring
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                network.openshift.io/policy-group: monitoring
        podSelector: {}
        policyTypes:
        - Ingress
      EOF

   3. A policy named allow-same-namespace:

      $ cat << EOF| oc create -f -
      kind: NetworkPolicy
      apiVersion: networking.k8s.io/v1
      metadata:
        name: allow-same-namespace
      spec:
        podSelector:
        ingress:
        - from:
          - podSelector: {}
      EOF

   4. A policy named allow-from-kube-apiserver-operator:

      $ cat << EOF| oc create -f -
      apiVersion: networking.k8s.io/v1
      kind: NetworkPolicy
      metadata:
        name: allow-from-kube-apiserver-operator
      spec:
        ingress:
        - from:
          - namespaceSelector:
              matchLabels:
                kubernetes.io/metadata.name: openshift-kube-apiserver-operator
            podSelector:
              matchLabels:
                app: kube-apiserver-operator
        policyTypes:
        - Ingress
      EOF

      For more details, see New kube-apiserver-operator webhook controller validating health of webhook.

2. Optional: To confirm that the network policies exist in your current project, enter the following command:

   $ oc describe networkpolicy

   Example output

   Name:         allow-from-openshift-ingress
   Namespace:    example1
   Created on:   2020-06-09 00:28:17 -0400 EDT
   Labels:       <none>
   Annotations:  <none>
   Spec:
     PodSelector:     <none> (Allowing the specific traffic to all pods in this namespace)
     Allowing ingress traffic:
       To Port: <any> (traffic allowed to all ports)
       From:
         NamespaceSelector: network.openshift.io/policy-group: ingress
     Not affecting egress traffic
     Policy Types: Ingress
   
   
   Name:         allow-from-openshift-monitoring
   Namespace:    example1
   Created on:   2020-06-09 00:29:57 -0400 EDT
   Labels:       <none>
   Annotations:  <none>
   Spec:
     PodSelector:     <none> (Allowing the specific traffic to all pods in this namespace)
     Allowing ingress traffic:
       To Port: <any> (traffic allowed to all ports)
       From:
         NamespaceSelector: network.openshift.io/policy-group: monitoring
     Not affecting egress traffic
     Policy Types: Ingress

Procedure

1. Log in as a user with cluster-admin privileges.

2. Generate the default project template:

   $ oc adm create-bootstrap-project-template -o yaml > template.yaml

3. Use a text editor to modify the generated template.yaml file by adding objects or modifying existing objects.

4. The project template must be created in the openshift-config namespace. Load your modified template:

   $ oc create -f template.yaml -n openshift-config

5. Edit the project configuration resource using the web console or CLI.

   • Using the web console:

     1. Navigate to the Administration Cluster Settings page.
     2. Click Configuration to view all configuration resources.
     3. Find the entry for Project and click Edit YAML.

   • Using the CLI:

     1. Edit the project.config.openshift.io/cluster resource:

        $ oc edit project.config.openshift.io/cluster

6. Update the spec section to include the projectRequestTemplate and name parameters, and set the name of your uploaded project template. The default name is project-request.

   Project configuration resource with custom project template

   apiVersion: config.openshift.io/v1
   kind: Project
   metadata:
     ...
   spec:
     projectRequestTemplate:
       name: <template_name>

7. After you save your changes, create a new project to verify that your changes were successfully applied.

1. Show the port:

   $ openstack port show <cluster_name>-<cluster_ID>-ingress-port

2. Attach the port to the IP address:

   $ openstack floating ip set --port <ingress_port_ID> <apps_FIP>

3. Add a wildcard A record for *apps. to your DNS file:

   *.apps.<cluster_name>.<base_domain>  IN  A  <apps_FIP>

Procedure

1. From a command line, open the Cluster Network Operator (CNO) CR for editing:

   $ oc edit networks.operator.openshift.io cluster

2. Edit the settings to meet your requirements. The following file is provided as an example:

   apiVersion: operator.openshift.io/v1
   kind: Network
   metadata:
     name: cluster
   spec:
     clusterNetwork:
     - cidr: 10.128.0.0/14
       hostPrefix: 23
     serviceNetwork:
     - 172.30.0.0/16
     defaultNetwork:
       type: Kuryr
       kuryrConfig:
         enablePortPoolsPrepopulation: false 1
         poolMinPorts: 1 2
         poolBatchPorts: 3 3
         poolMaxPorts: 5 4

   1

   Set enablePortPoolsPrepopulation to true to make Kuryr create Neutron ports when the first pod that is configured to use the dedicated network for pods is created in a namespace. This setting raises the Neutron ports quota but can reduce the time that is required to spawn pods. The default value is false.

   2

   Kuryr creates new ports for a pool if the number of free ports in that pool is lower than the value of poolMinPorts. The default value is 1.

   3

   poolBatchPorts controls the number of new ports that are created if the number of free ports is lower than the value of poolMinPorts. The default value is 3.

   4

   If the number of free ports in a pool is higher than the value of poolMaxPorts, Kuryr deletes them until the number matches that value. Setting the value to 0 disables this upper bound, preventing pools from shrinking. The default value is 0.

3. Save your changes and quit the text editor to commit your changes.

Procedure

1. From a command line, enter the following command to disable the admission webhook:

   $ oc patch sriovoperatorconfig default --type=merge -n openshift-sriov-network-operator --patch '{ "spec": { "enableOperatorWebhook": false } }'

2. Create an SriovNetworkNodePolicy policy for the two hw-offload type VF interfaces that are on your cluster:

   The first virtual function interface

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy 1
   metadata:
     name: "hwoffload9"
     namespace: openshift-sriov-network-operator
   spec:
     deviceType: netdevice
     isRdma: true
     nicSelector:
       pfNames: 2
       - ens6
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: 'true'
     numVfs: 1
     priority: 99
     resourceName: "hwoffload9"

   1

   Insert the SriovNetworkNodePolicy value here.

   2

   Both interfaces must include physical function (PF) names.

   The second virtual function interface

   apiVersion: sriovnetwork.openshift.io/v1
   kind: SriovNetworkNodePolicy 1
   metadata:
     name: "hwoffload10"
     namespace: openshift-sriov-network-operator
   spec:
     deviceType: netdevice
     isRdma: true
     nicSelector:
       pfNames: 2
       - ens5
     nodeSelector:
       feature.node.kubernetes.io/network-sriov.capable: 'true'
     numVfs: 1
     priority: 99
     resourceName: "hwoffload10"

   1

   Insert the SriovNetworkNodePolicy value here.

   2

   Both interfaces must include physical function (PF) names.

3. Create NetworkAttachmentDefinition resources for the two interfaces:

   A NetworkAttachmentDefinition resource for the first interface

   apiVersion: k8s.cni.cncf.io/v1
   kind: NetworkAttachmentDefinition
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/resourceName: openshift.io/hwoffload9
     name: hwoffload9
     namespace: default
   spec:
       config: '{ "cniVersion":"0.3.1", "name":"hwoffload9","type":"host-device","device":"ens6"
       }'

   A NetworkAttachmentDefinition resource for the second interface

   apiVersion: k8s.cni.cncf.io/v1
   kind: NetworkAttachmentDefinition
   metadata:
     annotations:
       k8s.v1.cni.cncf.io/resourceName: openshift.io/hwoffload10
     name: hwoffload10
     namespace: default
   spec:
       config: '{ "cniVersion":"0.3.1", "name":"hwoffload10","type":"host-device","device":"ens5"
       }'

4. Use the interfaces that you created with a pod. For example:

   A pod that uses the two OVS offload interfaces

   apiVersion: v1
   kind: Pod
   metadata:
     name: dpdk-testpmd
     namespace: default
     annotations:
       irq-load-balancing.crio.io: disable
       cpu-quota.crio.io: disable
       k8s.v1.cni.cncf.io/networks: '[
         {
          "name": "hwoffload9",
          "namespace": "default"
         },
         {
          "name": "hwoffload10",
          "namespace": "default"
         }
       ]'
   spec:
     restartPolicy: Never
     containers:
     - name: dpdk-testpmd
       image: quay.io/krister/centos8_nfv-container-dpdk-testpmd:latest