Chapter 2. Networking Operators

Procedure

1. After logging into your cluster as an admin user, run the following commands:

   Copy to Clipboard Toggle word wrap

   $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}")
   $ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
   $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o jsonpath='{.spec.serviceAccountIssuer}' | sed  's|^https://||')
   $ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
   $ export SCRATCH="/tmp/${CLUSTER_NAME}/alb-operator"
   $ mkdir -p ${SCRATCH}

2. You can verify that the variables are set by running the following command:

   Copy to Clipboard Toggle word wrap

   $ echo "Cluster name: ${CLUSTER_NAME}, Region: ${REGION}, OIDC Endpoint: ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"

   Example output

   Copy to Clipboard Toggle word wrap

   Cluster name: <cluster_id>, Region: us-east-2, OIDC Endpoint: oidc.op1.openshiftapps.com/<oidc_id>, AWS Account ID: <aws_id>

Procedure

1. Set the environmental variables to the proper values for your ROSA deployment:

   Copy to Clipboard Toggle word wrap

   $ export VPC_ID=<vpc-id>
   $ export PUBLIC_SUBNET_IDS="<public-subnet-a-id> <public-subnet-b-id> <public-subnet-c-id>"
   $ export PRIVATE_SUBNET_IDS="<private-subnet-a-id> <private-subnet-b-id> <private-subnet-c-id>"

2. Add a tag to your cluster’s VPC with the cluster name:

   Copy to Clipboard Toggle word wrap

   $ aws ec2 create-tags --resources ${VPC_ID} --tags Key=kubernetes.io/cluster/${CLUSTER_NAME},Value=owned --region ${REGION}

3. Add a tag to your public subnets:

   Copy to Clipboard Toggle word wrap

   $ aws ec2 create-tags \
        --resources ${PUBLIC_SUBNET_IDS} \
        --tags Key=kubernetes.io/role/elb,Value='' \
        --region ${REGION}

4. Add a tag to your private subnets:

   Copy to Clipboard Toggle word wrap

   $ aws ec2 create-tags \
        --resources ${PRIVATE_SUBNET_IDS} \
        --tags Key=kubernetes.io/role/internal-elb,Value='' \
        --region ${REGION}

1. Create a new project:

   Copy to Clipboard Toggle word wrap

   $ oc new-project hello-world

2. Deploy a hello world application:

   Copy to Clipboard Toggle word wrap

   $ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift

3. Configure a NodePort service for the AWS ALB to connect to:

   Copy to Clipboard Toggle word wrap

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: hello-openshift-nodeport
     namespace: hello-world
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: NodePort
     selector:
       deployment: hello-openshift
   EOF

4. Deploy an AWS ALB using the AWS Load Balancer Operator:

   Copy to Clipboard Toggle word wrap

   $ cat << EOF | oc apply -f -
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: hello-openshift-alb
     namespace: hello-world
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
   spec:
     ingressClassName: alb
     rules:
       - http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: hello-openshift-nodeport
                   port:
                     number: 80
   EOF

5. Curl the AWS ALB Ingress endpoint to verify the hello world application is accessible:

   Note

   AWS ALB provisioning takes a few minutes. If you receive an error that says curl: (6) Could not resolve host, please wait and try again.

   Copy to Clipboard Toggle word wrap

   $ INGRESS=$(oc -n hello-world get ingress hello-openshift-alb \
       -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   $ curl "http://${INGRESS}"

   Example output

   Copy to Clipboard Toggle word wrap

   Hello OpenShift!

6. Deploy an AWS NLB for your hello world application:

   Copy to Clipboard Toggle word wrap

   $ cat << EOF | oc apply -f -
   apiVersion: v1
   kind: Service
   metadata:
     name: hello-openshift-nlb
     namespace: hello-world
     annotations:
       service.beta.kubernetes.io/aws-load-balancer-type: external
       service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
       service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: LoadBalancer
     selector:
       deployment: hello-openshift
   EOF

7. Test the AWS NLB endpoint:

   Note

   NLB provisioning takes a few minutes. If you receive an error that says curl: (6) Could not resolve host, please wait and try again.

   Copy to Clipboard Toggle word wrap

   $ NLB=$(oc -n hello-world get service hello-openshift-nlb \
     -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
   $ curl "http://${NLB}"

   Example output

   Copy to Clipboard Toggle word wrap

   Hello OpenShift!

1. Create a Secret resource containing the custom certificate in the openshift-ingress namespace using the tls.crt and tls.key files.

   Copy to Clipboard Toggle word wrap

   $ oc --namespace openshift-ingress create secret tls custom-certs-default --cert=tls.crt --key=tls.key

2. Update the IngressController CR to reference the new certificate secret:

   Copy to Clipboard Toggle word wrap

   $ oc patch --type=merge --namespace openshift-ingress-operator ingresscontrollers/default \
     --patch '{"spec":{"defaultCertificate":{"name":"custom-certs-default"}}}'

3. Verify the update was effective:

   Copy to Clipboard Toggle word wrap

   $ echo Q |\
     openssl s_client -connect console-openshift-console.apps.<domain>:443 -showcerts 2>/dev/null |\
     openssl x509 -noout -subject -issuer -enddate

   where:

   <domain>

   Specifies the base domain name for your cluster.

   Example output

   Copy to Clipboard Toggle word wrap

   subject=C = US, ST = NC, L = Raleigh, O = RH, OU = OCP4, CN = *.apps.example.com
   issuer=C = US, ST = NC, L = Raleigh, O = RH, OU = OCP4, CN = example.com
   notAfter=May 10 08:32:45 2022 GM

   Tip

   You can alternatively apply the following YAML to set a custom default certificate:

   Copy to Clipboard Toggle word wrap

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: default
     namespace: openshift-ingress-operator
   spec:
     defaultCertificate:
       name: custom-certs-default

   The certificate secret name should match the value used to update the CR.

Procedure

1. Create a service account to authenticate with Thanos by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc create -n openshift-ingress-operator serviceaccount thanos && oc describe -n openshift-ingress-operator serviceaccount thanos

   Example output

   Copy to Clipboard Toggle word wrap

   Name:                thanos
   Namespace:           openshift-ingress-operator
   Labels:              <none>
   Annotations:         <none>
   Image pull secrets:  thanos-dockercfg-kfvf2
   Mountable secrets:   thanos-dockercfg-kfvf2
   Tokens:              <none>
   Events:              <none>

2. Manually create the service account secret token with the following command:

   Copy to Clipboard Toggle word wrap

   $ oc apply -f - <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: thanos-token
     namespace: openshift-ingress-operator
     annotations:
       kubernetes.io/service-account.name: thanos
   type: kubernetes.io/service-account-token
   EOF

3. Define a TriggerAuthentication object within the openshift-ingress-operator namespace by using the service account’s token.

   1. Create the TriggerAuthentication object and pass the value of the secret variable to the TOKEN parameter:

      Copy to Clipboard Toggle word wrap

      $ oc apply -f - <<EOF
      apiVersion: keda.sh/v1alpha1
      kind: TriggerAuthentication
      metadata:
        name: keda-trigger-auth-prometheus
        namespace: openshift-ingress-operator
      spec:
        secretTargetRef:
        - parameter: bearerToken
          name: thanos-token
          key: token
        - parameter: ca
          name: thanos-token
          key: ca.crt
      EOF

4. Create and apply a role for reading metrics from Thanos:

   1. Create a new role, thanos-metrics-reader.yaml, that reads metrics from pods and nodes:

      thanos-metrics-reader.yaml

      Copy to Clipboard Toggle word wrap

      apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      metadata:
        name: thanos-metrics-reader
        namespace: openshift-ingress-operator
      rules:
      - apiGroups:
        - ""
        resources:
        - pods
        - nodes
        verbs:
        - get
      - apiGroups:
        - metrics.k8s.io
        resources:
        - pods
        - nodes
        verbs:
        - get
        - list
        - watch
      - apiGroups:
        - ""
        resources:
        - namespaces
        verbs:
        - get

   2. Apply the new role by running the following command:

      Copy to Clipboard Toggle word wrap

      $ oc apply -f thanos-metrics-reader.yaml

5. Add the new role to the service account by entering the following commands:

   Copy to Clipboard Toggle word wrap

   $ oc adm policy -n openshift-ingress-operator add-role-to-user thanos-metrics-reader -z thanos --role-namespace=openshift-ingress-operator

   Copy to Clipboard Toggle word wrap

   $ oc adm policy -n openshift-ingress-operator add-cluster-role-to-user cluster-monitoring-view -z thanos

   Note

   The argument add-cluster-role-to-user is only required if you use cross-namespace queries. The following step uses a query from the kube-metrics namespace which requires this argument.

6. Create a new ScaledObject YAML file, ingress-autoscaler.yaml, that targets the default Ingress Controller deployment:

   Example ScaledObject definition

   Copy to Clipboard Toggle word wrap

   apiVersion: keda.sh/v1alpha1
   kind: ScaledObject
   metadata:
     name: ingress-scaler
     namespace: openshift-ingress-operator
   spec:
     scaleTargetRef: 

   1

   
       apiVersion: operator.openshift.io/v1
       kind: IngressController
       name: default
       envSourceContainerName: ingress-operator
     minReplicaCount: 1
     maxReplicaCount: 20 

   2

   
     cooldownPeriod: 1
     pollingInterval: 1
     triggers:
     - type: prometheus
       metricType: AverageValue
       metadata:
         serverAddress: https://thanos-querier.openshift-monitoring.svc.cluster.local:9091 

   3

   
         namespace: openshift-ingress-operator 

   4

   
         metricName: 'kube-node-role'
         threshold: '1'
         query: 'sum(kube_node_role{role="worker",service="kube-state-metrics"})' 

   5

   
         authModes: "bearer"
       authenticationRef:
         name: keda-trigger-auth-prometheus

   1

   The custom resource that you are targeting. In this case, the Ingress Controller.

   2

   Optional: The maximum number of replicas. If you omit this field, the default maximum is set to 100 replicas.

   3

   The Thanos service endpoint in the openshift-monitoring namespace.

   4

   The Ingress Operator namespace.

   5

   This expression evaluates to however many worker nodes are present in the deployed cluster.

   Important

   If you are using cross-namespace queries, you must target port 9091 and not port 9092 in the serverAddress field. You also must have elevated privileges to read metrics from this port.

7. Apply the custom resource definition by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc apply -f ingress-autoscaler.yaml

Procedure

1. View the current number of available replicas for the default IngressController:

   Copy to Clipboard Toggle word wrap

   $ oc get -n openshift-ingress-operator ingresscontrollers/default -o jsonpath='{$.status.availableReplicas}'

   Example output

   Copy to Clipboard Toggle word wrap

   2

2. Scale the default IngressController to the desired number of replicas using the oc patch command. The following example scales the default IngressController to 3 replicas:

   Copy to Clipboard Toggle word wrap

   $ oc patch -n openshift-ingress-operator ingresscontroller/default --patch '{"spec":{"replicas": 3}}' --type=merge

   Example output

   Copy to Clipboard Toggle word wrap

   ingresscontroller.operator.openshift.io/default patched

3. Verify that the default IngressController scaled to the number of replicas that you specified:

   Copy to Clipboard Toggle word wrap

   $ oc get -n openshift-ingress-operator ingresscontrollers/default -o jsonpath='{$.status.availableReplicas}'

   Example output

   Copy to Clipboard Toggle word wrap

   3

   Tip

   You can alternatively apply the following YAML to scale an Ingress Controller to three replicas:

   Copy to Clipboard Toggle word wrap

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: default
     namespace: openshift-ingress-operator
   spec:
     replicas: 3               

   1

   1

   If you need a different amount of replicas, change the replicas value.

Procedure

1. Create an IngressController custom resource (CR) in a file named <name>-ingress-controller.yaml, such as in the following example:

   Copy to Clipboard Toggle word wrap

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     namespace: openshift-ingress-operator
     name: <name> 

   1

   
   spec:
     domain: <domain> 

   2

   
     endpointPublishingStrategy:
       type: LoadBalancerService
       loadBalancer:
         scope: Internal 

   3

   1

   Replace <name> with a name for the IngressController object.

   2

   Specify the domain for the application published by the controller.

   3

   Specify a value of Internal to use an internal load balancer.

2. Create the Ingress Controller defined in the previous step by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc create -f <name>-ingress-controller.yaml 

   1

   1

   Replace <name> with the name of the IngressController object.

3. Optional: Confirm that the Ingress Controller was created by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc --all-namespaces=true get ingresscontrollers

Procedure

1. Configure the default Ingress Controller for your cluster to be internal by deleting and recreating it.

   Copy to Clipboard Toggle word wrap

   $ oc replace --force --wait --filename - <<EOF
   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     namespace: openshift-ingress-operator
     name: default
   spec:
     endpointPublishingStrategy:
       type: LoadBalancerService
       loadBalancer:
         scope: Internal
   EOF

Procedure

1. Configure the wildcard policy.

   1. Use the following command to edit the IngressController resource:

      Copy to Clipboard Toggle word wrap

      $ oc edit IngressController

   2. Under spec, set the wildcardPolicy field to WildcardsDisallowed or WildcardsAllowed:

      Copy to Clipboard Toggle word wrap

      spec:
        routeAdmission:
          wildcardPolicy: WildcardsDisallowed # or WildcardsAllowed

Procedure

1. Edit the Ingress Controller resource:

   Copy to Clipboard Toggle word wrap

   $ oc -n openshift-ingress-operator edit ingresscontroller/default

2. Replace the X-SSL-Client-Der HTTP request header with the X-Forwarded-Client-Cert HTTP request header:

   Copy to Clipboard Toggle word wrap

   apiVersion: operator.openshift.io/v1
   kind: IngressController
   metadata:
     name: default
     namespace: openshift-ingress-operator
   spec:
     httpHeaders:
       actions: 

   1

   
         request: 

   2

   
         - name: X-Forwarded-Client-Cert 

   3

   
           action:
             type: Set 

   4

   
             set:
              value: "%{+Q}[ssl_c_der,base64]" 

   5

   
         - name: X-SSL-Client-Der
           action:
             type: Delete

   1

   The list of actions you want to perform on the HTTP headers.

   2

   The type of header you want to change. In this case, a request header.

   3

   The name of the header you want to change. For a list of available headers you can set or delete, see HTTP header configuration.

   4

   The type of action being taken on the header. This field can have the value Set or Delete.

   5

   When setting HTTP headers, you must provide a value. The value can be a string from a list of available directives for that header, for example DENY, or it can be a dynamic value that will be interpreted using HAProxy’s dynamic value syntax. In this case, a dynamic value is added.

   Note

   For setting dynamic header values for HTTP responses, allowed sample fetchers are res.hdr and ssl_c_der. For setting dynamic header values for HTTP requests, allowed sample fetchers are req.hdr and ssl_c_der. Both request and response dynamic values can use the lower and base64 converters.

3. Save the file to apply the changes.

Procedure

1. Configure the HTTPHeaders field for the Ingress Controller.

   1. Use the following command to edit the IngressController resource:

      Copy to Clipboard Toggle word wrap

      $ oc edit IngressController

   2. Under spec, set the HTTPHeaders policy field to Append, Replace, IfNone, or Never:

      Copy to Clipboard Toggle word wrap

      apiVersion: operator.openshift.io/v1
      kind: IngressController
      metadata:
        name: default
        namespace: openshift-ingress-operator
      spec:
        httpHeaders:
          forwardedHeaderPolicy: Append

Procedure

1. Edit the Ingress Controller resource by entering the following command in your CLI:

   Copy to Clipboard Toggle word wrap

   $ oc -n openshift-ingress-operator edit ingresscontroller/default

2. Set the PROXY configuration:

   • If your Ingress Controller uses the HostNetwork endpoint publishing strategy type, set the spec.endpointPublishingStrategy.hostNetwork.protocol subfield to PROXY:

     Sample hostNetwork configuration to PROXY

     Copy to Clipboard Toggle word wrap

     # ...
       spec:
         endpointPublishingStrategy:
           hostNetwork:
             protocol: PROXY
           type: HostNetwork
     # ...

   • If your Ingress Controller uses the NodePortService endpoint publishing strategy type, set the spec.endpointPublishingStrategy.nodePort.protocol subfield to PROXY:

     Sample nodePort configuration to PROXY

     Copy to Clipboard Toggle word wrap

     # ...
       spec:
         endpointPublishingStrategy:
           nodePort:
             protocol: PROXY
           type: NodePortService
     # ...

   • If your Ingress Controller uses the Private endpoint publishing strategy type, set the spec.endpointPublishingStrategy.private.protocol subfield to PROXY:

     Sample private configuration to PROXY

     Copy to Clipboard Toggle word wrap

     # ...
       spec:
         endpointPublishingStrategy:
           private:
             protocol: PROXY
         type: Private
     # ...

Procedure

1. Configure the appsDomain field by specifying an alternative default domain for user-created routes.

   1. Edit the ingress cluster resource:

      Copy to Clipboard Toggle word wrap

      $ oc edit ingresses.config/cluster -o yaml

   2. Edit the YAML file:

      Sample appsDomain configuration to test.example.com

      Copy to Clipboard Toggle word wrap

      apiVersion: config.openshift.io/v1
      kind: Ingress
      metadata:
        name: cluster
      spec:
        domain: apps.example.com            

      1

      
        appsDomain: <test.example.com>      

      2

      1

      Specifies the default domain. You cannot modify the default domain after installation.

      2

      Optional: Domain for Red Hat OpenShift Service on AWS infrastructure to use for application routes. Instead of the default prefix, apps, you can use an alternative prefix like test.

2. Verify that an existing route contains the domain name specified in the appsDomain field by exposing the route and verifying the route domain change:

   Note

   Wait for the openshift-apiserver finish rolling updates before exposing the route.

   1. Expose the route:

      Copy to Clipboard Toggle word wrap

      $ oc expose service hello-openshift
      route.route.openshift.io/hello-openshift exposed

      Example output

      Copy to Clipboard Toggle word wrap

      $ oc get routes
      NAME              HOST/PORT                                   PATH   SERVICES          PORT       TERMINATION   WILDCARD
      hello-openshift   hello_openshift-<my_project>.test.example.com
      hello-openshift   8080-tcp                 None

Procedure

1. Configure the httpCompression field for the Ingress Controller.

   1. Use the following command to edit the IngressController resource:

      Copy to Clipboard Toggle word wrap

      $ oc edit -n openshift-ingress-operator ingresscontrollers/default

   2. Under spec, set the httpCompression policy field to mimeTypes and specify a list of MIME types that should have compression applied:

      Copy to Clipboard Toggle word wrap

      apiVersion: operator.openshift.io/v1
      kind: IngressController
      metadata:
        name: default
        namespace: openshift-ingress-operator
      spec:
        httpCompression:
          mimeTypes:
          - "text/html"
          - "text/css; charset=utf-8"
          - "application/json"
         ...

Procedure

1. Get the router pod name by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc get pods -n openshift-ingress

   Example output

   Copy to Clipboard Toggle word wrap

   NAME                              READY   STATUS    RESTARTS   AGE
   router-default-76bfffb66c-46qwp   1/1     Running   0          11h

2. Get the router’s username and password, which the router pod stores in the /var/lib/haproxy/conf/metrics-auth/statsUsername and /var/lib/haproxy/conf/metrics-auth/statsPassword files:

   1. Get the username by running the following command:

      Copy to Clipboard Toggle word wrap

      $ oc rsh <router_pod_name> cat metrics-auth/statsUsername

   2. Get the password by running the following command:

      Copy to Clipboard Toggle word wrap

      $ oc rsh <router_pod_name> cat metrics-auth/statsPassword

3. Get the router IP and metrics certificates by running the following command:

   Copy to Clipboard Toggle word wrap

   $ oc describe pod <router_pod>

4. Get the raw statistics in Prometheus format by running the following command:

   Copy to Clipboard Toggle word wrap

   $ curl -u <user>:<password> http://<router_IP>:<stats_port>/metrics

5. Access the metrics securely by running the following command:

   Copy to Clipboard Toggle word wrap

   $ curl -u user:password https://<router_IP>:<stats_port>/metrics -k

6. Access the default stats port, 1936, by running the following command:

   Copy to Clipboard Toggle word wrap

   $ curl -u <user>:<password> http://<router_IP>:<stats_port>/metrics

   Example 2.1. Example output

   Copy to Clipboard Toggle word wrap

   ...
   # HELP haproxy_backend_connections_total Total number of connections.
   # TYPE haproxy_backend_connections_total gauge
   haproxy_backend_connections_total{backend="http",namespace="default",route="hello-route"} 0
   haproxy_backend_connections_total{backend="http",namespace="default",route="hello-route-alt"} 0
   haproxy_backend_connections_total{backend="http",namespace="default",route="hello-route01"} 0
   ...
   # HELP haproxy_exporter_server_threshold Number of servers tracked and the current threshold value.
   # TYPE haproxy_exporter_server_threshold gauge
   haproxy_exporter_server_threshold{type="current"} 11
   haproxy_exporter_server_threshold{type="limit"} 500
   ...
   # HELP haproxy_frontend_bytes_in_total Current total of incoming bytes.
   # TYPE haproxy_frontend_bytes_in_total gauge
   haproxy_frontend_bytes_in_total{frontend="fe_no_sni"} 0
   haproxy_frontend_bytes_in_total{frontend="fe_sni"} 0
   haproxy_frontend_bytes_in_total{frontend="public"} 119070
   ...
   # HELP haproxy_server_bytes_in_total Current total of incoming bytes.
   # TYPE haproxy_server_bytes_in_total gauge
   haproxy_server_bytes_in_total{namespace="",pod="",route="",server="fe_no_sni",service=""} 0
   haproxy_server_bytes_in_total{namespace="",pod="",route="",server="fe_sni",service=""} 0
   haproxy_server_bytes_in_total{namespace="default",pod="docker-registry-5-nk5fz",route="docker-registry",server="10.130.0.89:5000",service="docker-registry"} 0
   haproxy_server_bytes_in_total{namespace="default",pod="hello-rc-vkjqx",route="hello-route",server="10.130.0.90:8080",service="hello-svc-1"} 0
   ...

7. Launch the stats window by entering the following URL in a browser:

   Copy to Clipboard Toggle word wrap

   http://<user>:<password>@<router_IP>:<stats_port>

8. Optional: Get the stats in CSV format by entering the following URL in a browser:

   Copy to Clipboard Toggle word wrap

   http://<user>:<password>@<router_ip>:1936/metrics;csv

Procedure

1. Create a config map named my-custom-error-code-pages in the openshift-config namespace:

   Copy to Clipboard Toggle word wrap

   $ oc -n openshift-config create configmap my-custom-error-code-pages \
   --from-file=error-page-503.http \
   --from-file=error-page-404.http

   Important

   If you do not specify the correct format for the custom error code response page, a router pod outage occurs. To resolve this outage, you must delete or correct the config map and delete the affected router pods so they can be recreated with the correct information.

2. Patch the Ingress Controller to reference the my-custom-error-code-pages config map by name:

   Copy to Clipboard Toggle word wrap

   $ oc patch -n openshift-ingress-operator ingresscontroller/default --patch '{"spec":{"httpErrorCodePages":{"name":"my-custom-error-code-pages"}}}' --type=merge

   The Ingress Operator copies the my-custom-error-code-pages config map from the openshift-config namespace to the openshift-ingress namespace. The Operator names the config map according to the pattern, <your_ingresscontroller_name>-errorpages, in the openshift-ingress namespace.

3. Display the copy:

   Copy to Clipboard Toggle word wrap

   $ oc get cm default-errorpages -n openshift-ingress

   Example output

   Copy to Clipboard Toggle word wrap

   NAME                       DATA   AGE
   default-errorpages         2      25s  

   1

   1

   The example config map name is default-errorpages because the default Ingress Controller custom resource (CR) was patched.

4. Confirm that the config map containing the custom error response page mounts on the router volume where the config map key is the filename that has the custom HTTP error code response:

   • For 503 custom HTTP custom error code response:

     Copy to Clipboard Toggle word wrap

     $ oc -n openshift-ingress rsh <router_pod> cat /var/lib/haproxy/conf/error_code_pages/error-page-503.http

   • For 404 custom HTTP custom error code response:

     Copy to Clipboard Toggle word wrap

     $ oc -n openshift-ingress rsh <router_pod> cat /var/lib/haproxy/conf/error_code_pages/error-page-404.http

1. Create a test project and application:

   Copy to Clipboard Toggle word wrap

    $ oc new-project test-ingress

   Copy to Clipboard Toggle word wrap

   $ oc new-app django-psql-example

2. For 503 custom http error code response:

   1. Stop all the pods for the application.

   2. Run the following curl command or visit the route hostname in the browser:

      Copy to Clipboard Toggle word wrap

      $ curl -vk <route_hostname>

3. For 404 custom http error code response:

   1. Visit a non-existent route or an incorrect route.

   2. Run the following curl command or visit the route hostname in the browser:

      Copy to Clipboard Toggle word wrap

      $ curl -vk <route_hostname>

4. Check if the errorfile attribute is properly in the haproxy.config file:

   Copy to Clipboard Toggle word wrap

   $ oc -n openshift-ingress rsh <router> cat /var/lib/haproxy/conf/haproxy.config | grep errorfile

Procedure

1. To create the openshift-ingress-node-firewall namespace, enter the following command:

   Copy to Clipboard Toggle word wrap

   $ cat << EOF| oc create -f -
   apiVersion: v1
   kind: Namespace
   metadata:
     labels:
       pod-security.kubernetes.io/enforce: privileged
       pod-security.kubernetes.io/enforce-version: v1.24
     name: openshift-ingress-node-firewall
   EOF

2. To create an OperatorGroup CR, enter the following command:

   Copy to Clipboard Toggle word wrap

   $ cat << EOF| oc create -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: ingress-node-firewall-operators
     namespace: openshift-ingress-node-firewall
   EOF

3. Subscribe to the Ingress Node Firewall Operator.

   1. To create a Subscription CR for the Ingress Node Firewall Operator, enter the following command:

      Copy to Clipboard Toggle word wrap

      $ cat << EOF| oc create -f -
      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: ingress-node-firewall-sub
        namespace: openshift-ingress-node-firewall
      spec:
        name: ingress-node-firewall
        channel: stable
        source: redhat-operators
        sourceNamespace: openshift-marketplace
      EOF

4. To verify that the Operator is installed, enter the following command:

   Copy to Clipboard Toggle word wrap

   $ oc get ip -n openshift-ingress-node-firewall

   Example output

   Copy to Clipboard Toggle word wrap

   NAME            CSV                                         APPROVAL    APPROVED
   install-5cvnz   ingress-node-firewall.4.0-202211122336   Automatic   true

5. To verify the version of the Operator, enter the following command:

   Copy to Clipboard Toggle word wrap

   $ oc get csv -n openshift-ingress-node-firewall

   Example output

   Copy to Clipboard Toggle word wrap

   NAME                                        DISPLAY                          VERSION               REPLACES                                    PHASE
   ingress-node-firewall.4.0-202211122336   Ingress Node Firewall Operator   4.0-202211122336   ingress-node-firewall.4.0-202211102047   Succeeded

Procedure

1. Install the Ingress Node Firewall Operator:

   1. In the Red Hat OpenShift Service on AWS web console, click Operators OperatorHub.
   2. Select Ingress Node Firewall Operator from the list of available Operators, and then click Install.
   3. On the Install Operator page, under Installed Namespace, select Operator recommended Namespace.
   4. Click Install.

2. Verify that the Ingress Node Firewall Operator is installed successfully:

   1. Navigate to the Operators Installed Operators page.

   2. Ensure that Ingress Node Firewall Operator is listed in the openshift-ingress-node-firewall project with a Status of InstallSucceeded.

      Note

      During installation an Operator might display a Failed status. If the installation later succeeds with an InstallSucceeded message, you can ignore the Failed message.

      If the Operator does not have a Status of InstallSucceeded, troubleshoot using the following steps:

      • Inspect the Operator Subscriptions and Install Plans tabs for any failures or errors under Status.
      • Navigate to the Workloads Pods page and check the logs for pods in the openshift-ingress-node-firewall project.

      • Check the namespace of the YAML file. If the annotation is missing, you can add the annotation workload.openshift.io/allowed=management to the Operator namespace with the following command:

        Copy to Clipboard Toggle word wrap

        $ oc annotate ns/openshift-ingress-node-firewall workload.openshift.io/allowed=management

        Note

        For single-node OpenShift clusters, the openshift-ingress-node-firewall namespace requires the workload.openshift.io/allowed=management annotation.