Chapter 27. Configuring ingress cluster traffic

Procedure

1. Modify the default Ingress Controller by running the following command:

   $ oc edit ingresscontroller -n openshift-ingress-operator default

2. Edit the Ingress Controller to contain a

   namespaceSelector

   that excludes the routes with any of the

   finance

   ,

   ops

   , and

   dev

   labels:

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: default
     namespace: openshift-ingress-operator
   spec:
     namespaceSelector:
       matchExpressions:
         - key: name
           operator: NotIn
           values:
             - finance
             - ops
             - dev

Procedure

1. Edit the

   router-internal.yaml

   file:

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: sharded
     namespace: openshift-ingress-operator
   spec:
     domain: <apps-sharded.basedomain.example.net> 

   1

   
     nodePlacement:
       nodeSelector:
         matchLabels:
           node-role.kubernetes.io/worker: ""
     routeSelector:
       matchLabels:
         type: sharded

   1

   Specify a domain to be used by the Ingress Controller. This domain must be different from the default Ingress Controller domain.

2. Apply the Ingress Controller

   router-internal.yaml

   file:

   # oc apply -f router-internal.yaml

   The Ingress Controller selects routes in any namespace that have the label

   type: sharded

   .

3. Create a new route using the domain configured in the

   router-internal.yaml

   :

   $ oc expose svc <service-name> --hostname <route-name>.apps-sharded.basedomain.example.net

Procedure

1. Edit the

   router-internal.yaml

   file:

   $ cat router-internal.yaml

   Example output

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: sharded
     namespace: openshift-ingress-operator
   spec:
     domain: <apps-sharded.basedomain.example.net> 

   1

   
     nodePlacement:
       nodeSelector:
         matchLabels:
           node-role.kubernetes.io/worker: ""
     namespaceSelector:
       matchLabels:
         type: sharded

   1

   Specify a domain to be used by the Ingress Controller. This domain must be different from the default Ingress Controller domain.

2. Apply the Ingress Controller

   router-internal.yaml

   file:

   $ oc apply -f router-internal.yaml

   The Ingress Controller selects routes in any namespace that is selected by the namespace selector that have the label

   type: sharded

   .

3. Create a new route using the domain configured in the

   router-internal.yaml

   :

   $ oc expose svc <service-name> --hostname <route-name>.apps-sharded.basedomain.example.net

Procedure

1. Create a project called

   hello-openshift

   by running the following command:

   $ oc new-project hello-openshift

2. Create a pod in the project by running the following command:

   $ oc create -f https://raw.githubusercontent.com/openshift/origin/master/examples/hello-openshift/hello-pod.json

3. Create a service called

   hello-openshift

   by running the following command:

   $ oc expose pod/hello-openshift

4. Create a route definition called

   hello-openshift-route.yaml

   :

   YAML definition of the created route for sharding:

   apiVersion: route.openshift.io/v1
   kind: Route
   metadata:
     labels:
       type: sharded 

   1

   
     name: hello-openshift-edge
     namespace: hello-openshift
   spec:
     subdomain: hello-openshift 

   2

   
     tls:
       termination: edge
     to:
       kind: Service
       name: hello-openshift

   1

   Both the label key and its corresponding label value must match the ones specified in the Ingress Controller. In this example, the Ingress Controller has the label key and value type: sharded.

   2

   The route will be exposed using the value of the subdomain field. When you specify the subdomain field, you must leave the hostname unset. If you specify both the host and subdomain fields, then the route will use the value of the host field, and ignore the subdomain field.

5. Use

   hello-openshift-route.yaml

   to create a route to the

   hello-openshift

   application by running the following command:

   $ oc -n hello-openshift create -f hello-openshift-route.yaml

Procedure

1. Create a custom resource (CR) file for the Ingress Controller:

   Example of a CR file that defines information for the IngressController object

   apiVersion: v1
   items:
   - apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: <custom_ic_name> 

   1

   
       namespace: openshift-ingress-operator
     spec:
       replicas: 1
       domain: <custom_ic_domain_name> 

   2

   
       nodePlacement:
         nodeSelector:
           matchLabels:
             <key>: <value> 

   3

   
       namespaceSelector:
        matchLabels:
          <key>: <value> 

   4

   
       endpointPublishingStrategy:
         type: NodePortService
   # ...

   1

   Specify the a custom name for the IngressController CR.

   2

   The DNS name that the Ingress Controller services. As an example, the default ingresscontroller domain is apps.ipi-cluster.example.com, so you would specify the <custom_ic_domain_name> as nodeportsvc.ipi-cluster.example.com.

   3

   Specify the label for the nodes that include the custom Ingress Controller.

   4

   Specify the label for a set of namespaces. Substitute <key>:<value> with a map of key-value pairs where <key> is a unique name for the new label and <value> is its value. For example: ingresscontroller: custom-ic.

2. Add a label to a node by using the

   oc label node

   command:

   $ oc label node <node_name> <key>=<value> 

   1

   1

   Where <value> must match the key-value pair specified in the nodePlacement section of your IngressController CR.

3. Create the

   IngressController

   object:

   $ oc create -f <ingress_controller_cr>.yaml

4. Find the port for the service created for the

   IngressController

   CR:

   $ oc get svc -n openshift-ingress

   Example output that shows port 80:32432/TCP for the router-nodeport-custom-ic3 service

   NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                                     AGE
   router-internal-default      ClusterIP   172.30.195.74    <none>        80/TCP,443/TCP,1936/TCP                     223d
   router-nodeport-custom-ic3   NodePort    172.30.109.219   <none>        80:32432/TCP,443:31366/TCP,1936:30499/TCP   155m

5. To create a new project, enter the following command:

   $ oc new-project <project_name>

6. To label the new namespace, enter the following command:

   $ oc label namespace <project_name> <key>=<value> 

   1

   1

   Where <key>=<value> must match the value in the namespaceSelector section of your Ingress Controller CR.

7. Create a new application in your cluster:

   $ oc new-app --image=<image_name> 

   1

   1

   An example of <image_name> is quay.io/openshifttest/hello-openshift:multiarch.

8. Create a

   Route

   object for a service, so that the pod can use the service to expose the application external to the cluster.

   $ oc expose svc/<service_name> --hostname=<svc_name>-<project_name>.<custom_ic_domain_name> 

   1

   Note

   You must specify the domain name of your custom Ingress Controller in the

   --hostname

   argument. If you do not do this, the Ingress Operator uses the default Ingress Controller to serve all the routes for your cluster.

9. Check that the route has the

   Admitted

   status and that it includes metadata for the custom Ingress Controller:

   $ oc get route/hello-openshift -o json | jq '.status.ingress'

   Example output

   # ...
   {
     "conditions": [
       {
         "lastTransitionTime": "2024-05-17T18:25:41Z",
         "status": "True",
         "type": "Admitted"
       }
     ],
     [
       {
         "host": "hello-openshift.nodeportsvc.ipi-cluster.example.com",
         "routerCanonicalHostname": "router-nodeportsvc.nodeportsvc.ipi-cluster.example.com",
         "routerName": "nodeportsvc", "wildcardPolicy": "None"
       }
     ],
   }

10. Update the default

    IngressController

    CR to prevent the default Ingress Controller from managing the

    NodePort

    -type

    Service

    . The default Ingress Controller will continue to monitor all other cluster traffic.

    $ oc patch --type=merge -n openshift-ingress-operator ingresscontroller/default --patch '{"spec":{"namespaceSelector":{"matchExpressions":[{"key":"<key>","operator":"NotIn","values":["<value>]}]}}}'

Verification

1. Verify that the DNS entry can route inside and outside of your cluster by entering the following command. The command outputs the IP address of the node that received the label from running the

   oc label node

   command earlier in the procedure.

   $ dig +short <svc_name>-<project_name>.<custom_ic_domain_name>

2. To verify that your cluster uses the IP addresses from external DNS servers for DNS resolution, check the connection of your cluster by entering the following command:

   $ curl <svc_name>-<project_name>.<custom_ic_domain_name>:<port> 

   1

   1 1

   Where <port> is the node port from the NodePort-type Service. Based on example output from the oc get svc -n openshift-ingress command, the 80:32432/TCP HTTP route means that 32432 is the node port.

   Output example

   Hello OpenShift!

Procedure

1. Using the

   oc annotate

   command, add the timeout to the route:

   $ oc annotate route <route_name> \
       --overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit> 

   1

   1

   Supported time units are microseconds (us), milliseconds (ms), seconds (s), minutes (m), hours (h), or days (d).

   The following example sets a timeout of two seconds on a route named

   myroute

   :

   $ oc annotate route myroute --overwrite haproxy.router.openshift.io/timeout=2s

Procedure

1. Set an AWS connection idle timeout of five minutes for the default

   ingresscontroller

   by running the following command:

   $ oc -n openshift-ingress-operator patch ingresscontroller/default \
       --type=merge --patch='{"spec":{"endpointPublishingStrategy": \
       {"type":"LoadBalancerService", "loadBalancer": \
       {"scope":"External", "providerParameters":{"type":"AWS", "aws": \
       {"type":"Classic", "classicLoadBalancer": \
       {"connectionIdleTimeout":"5m"}}}}}}}'

2. Optional: Restore the default value of the timeout by running the following command:

   $ oc -n openshift-ingress-operator patch ingresscontroller/default \
       --type=merge --patch='{"spec":{"endpointPublishingStrategy": \
       {"loadBalancer":{"providerParameters":{"aws":{"classicLoadBalancer": \
       {"connectionIdleTimeout":null}}}}}}}'

Procedure

1. Modify the existing Ingress Controller that you want to switch to using an NLB. This example assumes that your default Ingress Controller has an

   External

   scope and no other customizations:

   Example ingresscontroller.yaml file

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     creationTimestamp: null
     name: default
     namespace: openshift-ingress-operator
   spec:
     endpointPublishingStrategy:
       loadBalancer:
         scope: External
         providerParameters:
           type: AWS
           aws:
             type: NLB
       type: LoadBalancerService

   Note

   If you do not specify a value for the

   spec.endpointPublishingStrategy.loadBalancer.providerParameters.aws.type

   field, the Ingress Controller uses the

   spec.loadBalancer.platform.aws.type

   value from the cluster

   Ingress

   configuration that was set during installation.

   Tip

   If your Ingress Controller has other customizations that you want to update, such as changing the domain, consider force replacing the Ingress Controller definition file instead.

2. Apply the changes to the Ingress Controller YAML file by running the command:

   $ oc apply -f ingresscontroller.yaml

   Expect several minutes of outages while the Ingress Controller updates.

Procedure

1. Modify the existing Ingress Controller that you want to switch to using a CLB. This example assumes that your default Ingress Controller has an

   External

   scope and no other customizations:

   Example ingresscontroller.yaml file

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     creationTimestamp: null
     name: default
     namespace: openshift-ingress-operator
   spec:
     endpointPublishingStrategy:
       loadBalancer:
         scope: External
         providerParameters:
           type: AWS
           aws:
             type: Classic
       type: LoadBalancerService

   Note

   If you do not specify a value for the

   spec.endpointPublishingStrategy.loadBalancer.providerParameters.aws.type

   field, the Ingress Controller uses the

   spec.loadBalancer.platform.aws.type

   value from the cluster

   Ingress

   configuration that was set during installation.

   Tip

   If your Ingress Controller has other customizations that you want to update, such as changing the domain, consider force replacing the Ingress Controller definition file instead.

2. Apply the changes to the Ingress Controller YAML file by running the command:

   $ oc apply -f ingresscontroller.yaml

   Expect several minutes of outages while the Ingress Controller updates.

Procedure

1. Create a file with a new default Ingress Controller. The following example assumes that your default Ingress Controller has an

   External

   scope and no other customizations:

   Example ingresscontroller.yml file

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     creationTimestamp: null
     name: default
     namespace: openshift-ingress-operator
   spec:
     endpointPublishingStrategy:
       loadBalancer:
         scope: External
         providerParameters:
           type: AWS
           aws:
             type: NLB
       type: LoadBalancerService

   If your default Ingress Controller has other customizations, ensure that you modify the file accordingly.

   Tip

   If your Ingress Controller has no other customizations and you are only updating the load balancer type, consider following the procedure detailed in "Switching the Ingress Controller from using a Classic Load Balancer to a Network Load Balancer".

2. Force replace the Ingress Controller YAML file:

   $ oc replace --force --wait -f ingresscontroller.yml

   Wait until the Ingress Controller is replaced. Expect several of minutes of outages.

1. Create the Ingress Controller manifest:

    $ cat ingresscontroller-aws-nlb.yaml

   Example output

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: $my_ingress_controller

   1

   
     namespace: openshift-ingress-operator
   spec:
     domain: $my_unique_ingress_domain

   2

   
     endpointPublishingStrategy:
       type: LoadBalancerService
       loadBalancer:
         scope: External

   3

   
         providerParameters:
           type: AWS
           aws:
             type: NLB

   1

   Replace $my_ingress_controller with a unique name for the Ingress Controller.

   2

   Replace $my_unique_ingress_domain with a domain name that is unique among all Ingress Controllers in the cluster. This variable must be a subdomain of the DNS name <clustername>.<domain>.

   3

   You can replace External with Internal to use an internal NLB.

2. Create the resource in the cluster:

   $ oc create -f ingresscontroller-aws-nlb.yaml

1. Change to the directory that contains the installation program and create the manifests:

   $ ./openshift-install create manifests --dir <installation_directory> 

   1

   1

   For <installation_directory>, specify the name of the directory that contains the install-config.yaml file for your cluster.

2. Create a file that is named

   cluster-ingress-default-ingresscontroller.yaml

   in the

   <installation_directory>/manifests/

   directory:

   $ touch <installation_directory>/manifests/cluster-ingress-default-ingresscontroller.yaml 

   1

   1

   For <installation_directory>, specify the directory name that contains the manifests/ directory for your cluster.

   After creating the file, several network configuration files are in the

   manifests/

   directory, as shown:

   $ ls <installation_directory>/manifests/cluster-ingress-default-ingresscontroller.yaml

   Example output

   cluster-ingress-default-ingresscontroller.yaml

3. Open the

   cluster-ingress-default-ingresscontroller.yaml

   file in an editor and enter a custom resource (CR) that describes the Operator configuration you want:

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     creationTimestamp: null
     name: default
     namespace: openshift-ingress-operator
   spec:
     endpointPublishingStrategy:
       loadBalancer:
         scope: External
         providerParameters:
           type: AWS
           aws:
             type: NLB
       type: LoadBalancerService

4. Save the

   cluster-ingress-default-ingresscontroller.yaml

   file and quit the text editor.
5. Optional: Back up the

   manifests/cluster-ingress-default-ingresscontroller.yaml

   file. The installation program deletes the

   manifests/

   directory when creating the cluster.