Chapter 6. Ingress Operator in OpenShift Container Platform

• All the available nodes in the cluster have their own, externally accessible IP addresses. The service running in the cluster is bound to the unique NodePort for all the nodes.
• When the client connects to a node that is down, for example, by connecting the 10.0.128.4 IP address in the graphic, the node port directly connects the client to an available node that is running the service. In this scenario, no load balancing is required. As the image shows, the 10.0.128.4 address is down and another IP address must be used instead.

Prerequisites

• You must have a certificate/key pair in PEM-encoded files, where the certificate is signed by a trusted certificate authority or by a private trusted certificate authority that you configured in a custom PKI.

• Your certificate meets the following requirements:

  • The certificate is valid for the ingress domain.
  • The certificate uses the subjectAltName extension to specify a wildcard domain, such as *.apps.ocp4.example.com.

• You must have an IngressController CR. You may use the default one:

  $ oc --namespace openshift-ingress-operator get ingresscontrollers

  Example output

  NAME      AGE
  default   10m

Prerequisites

• You have access to the cluster as a user with the cluster-admin role.
• You have installed the OpenShift CLI (oc).
• You previously configured a custom default certificate for the Ingress Controller.

Procedure

• To remove the custom certificate and restore the certificate that ships with OpenShift Container Platform, enter the following command:

  $ oc patch -n openshift-ingress-operator ingresscontrollers/default \
    --type json -p $'- op: remove\n  path: /spec/defaultCertificate'

  There can be a delay while the cluster reconciles the new certificate configuration.

Verification

• To confirm that the original cluster certificate is restored, enter the following command:

  $ echo Q | \
    openssl s_client -connect console-openshift-console.apps.<domain>:443 -showcerts 2>/dev/null | \
    openssl x509 -noout -subject -issuer -enddate

  where:

  <domain>

  Specifies the base domain name for your cluster.

  Example output

  subject=CN = *.apps.<domain>
  issuer=CN = ingress-operator@1620633373
  notAfter=May 10 10:44:36 2023 GMT

Prerequisites

• Log in as a user with cluster-admin privileges.

• To configure Ingress access logging, you must specify a destination using spec.logging.access.destination. To specify logging to a sidecar container, you must specify Container spec.logging.access.destination.type. The following example is an Ingress Controller definition that logs to a Container destination:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Container

• When you configure the Ingress Controller to log to a sidecar, the operator creates a container named logs inside the Ingress Controller Pod:

  $ oc -n openshift-ingress logs deployment.apps/router-default -c logs

  Example output

  2020-05-11T19:11:50.135710+00:00 router-default-57dfc6cd95-bpmk6 router-default-57dfc6cd95-bpmk6 haproxy[108]: 174.19.21.82:39654 [11/May/2020:19:11:50.133] public be_http:hello-openshift:hello-openshift/pod:hello-openshift:hello-openshift:10.128.2.12:8080 0/0/1/0/1 200 142 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"

• To configure Ingress access logging, you must specify a destination using spec.logging.access.destination. To specify logging to a Syslog endpoint destination, you must specify Syslog for spec.logging.access.destination.type. If the destination type is Syslog, you must also specify a destination endpoint using spec.logging.access.destination.syslog.endpoint and you can specify a facility using spec.logging.access.destination.syslog.facility. The following example is an Ingress Controller definition that logs to a Syslog destination:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Syslog
          syslog:
            address: 1.2.3.4
            port: 10514

  Note

  The syslog destination port must be UDP.

• You can specify spec.logging.access.httpLogFormat to customize the log format. The following example is an Ingress Controller definition that logs to a syslog endpoint with IP address 1.2.3.4 and port 10514:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Syslog
          syslog:
            address: 1.2.3.4
            port: 10514
        httpLogFormat: '%ci:%cp [%t] %ft %b/%s %B %bq %HM %HU %HV'

• To disable Ingress access logging, leave spec.logging or spec.logging.access empty:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access: null

Prerequisites

• The following assumes that you already created an Ingress Controller.

Procedure

• Update the Ingress Controller to increase the number of threads:

  $ oc -n openshift-ingress-operator patch ingresscontroller/default --type=merge -p '{"spec":{"tuningOptions": {"threadCount": 8}}}'

  Note

  If you have a node that is capable of running large amounts of resources, you can configure spec.nodePlacement.nodeSelector with labels that match the capacity of the intended node, and configure spec.tuningOptions.threadCount to an appropriately high value.

• Balance Ingress Controllers, or routers, with several routes to speed up responses to changes.
• Allocate certain routes to have different reliability guarantees than other routes.
• Allow certain Ingress Controllers to have different policies defined.
• Allow only specific routes to use additional features.
• Expose different routes on different addresses so that internal and external users can see different routes, for example.

• You can load load balance externally, using the cloud provider load balancer, or internally, using the OpenShift Ingress Controller Load Balancer.
• You can use the single IP address of the load balancer and more familiar ports, such as 8080 and 4200 as shown on the cluster depicted in the graphic.
• Traffic from the external load balancer is directed at the pods, and managed by the load balancer, as depicted in the instance of a down node. See the Kubernetes Services documentation for implementation details.

Prerequisites

• Install the OpenShift CLI (oc).
• Log in as a user with cluster-admin privileges.

Prerequisites

• You deployed an OpenShift Container Platform cluster on GCP infrastructure.
• You configured an Ingress Controller to use an internal load balancer.
• You installed the OpenShift CLI (oc).

Prerequisites

• Install the OpenShift CLI (oc).
• Log in as a user with cluster-admin privileges.

Prerequisites

• Cluster administrator privileges.

Procedure

• Edit the .spec.routeAdmission field of the ingresscontroller resource variable using the following command:

  $ oc -n openshift-ingress-operator patch ingresscontroller/default --patch '{"spec":{"routeAdmission":{"namespaceOwnership":"InterNamespaceAllowed"}}}' --type=merge

  Sample Ingress Controller configuration

  spec:
    routeAdmission:
      namespaceOwnership: InterNamespaceAllowed
  ...

  Tip

  You can alternatively apply the following YAML to configure the route admission policy:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    routeAdmission:
      namespaceOwnership: InterNamespaceAllowed

• Configure an external proxy that injects the X-Forwarded-For header into each request before forwarding it to an Ingress Controller.

  To configure the Ingress Controller to pass the header through unmodified, you specify the never policy. The Ingress Controller then never sets the headers, and applications receive only the headers that the external proxy provides.

• Configure the Ingress Controller to pass the X-Forwarded-For header that your external proxy sets on external cluster requests through unmodified.

  To configure the Ingress Controller to set the X-Forwarded-For header on internal cluster requests, which do not go through the external proxy, specify the if-none policy. If an HTTP request already has the header set through the external proxy, then the Ingress Controller preserves it. If the header is absent because the request did not come through the proxy, then the Ingress Controller adds the header.

• Configure an application-specific external proxy that injects the X-Forwarded-For header.

  To configure an Ingress Controller to pass the header through unmodified for an application’s Route, without affecting the policy for other Routes, add an annotation haproxy.router.openshift.io/set-forwarded-headers: if-none or haproxy.router.openshift.io/set-forwarded-headers: never on the Route for the application.

  Note

  You can set the haproxy.router.openshift.io/set-forwarded-headers annotation on a per route basis, independent from the globally set value for the Ingress Controller.

• To enable HTTP/2 on an Ingress Controller, enter the oc annotate command:

  $ oc -n openshift-ingress-operator annotate ingresscontrollers/<ingresscontroller_name> ingress.operator.openshift.io/default-enable-http2=true

  Replace <ingresscontroller_name> with the name of the Ingress Controller to annotate.

• To enable HTTP/2 for the entire cluster, enter the oc annotate command:

  $ oc annotate ingresses.config/cluster ingress.operator.openshift.io/default-enable-http2=true

  Tip

  You can alternatively apply the following YAML to add the annotation:

  apiVersion: config.openshift.io/v1
  kind: Ingress
  metadata:
    name: cluster
    annotations:
      ingress.operator.openshift.io/default-enable-http2: "true"

Prerequisites

• You created an Ingress Controller.

Prerequisites

• You deployed an OpenShift Container Platform cluster.
• You installed the oc command line interface.

Prerequisites

• You have installed the OpenShift CLI (oc).
• You have access to the cluster as a user with the cluster-admin role.

• Specify an HTTP header to be capitalized by entering the oc patch command.

  1. Enter the oc patch command to change the HTTP host header to Host:

     $ oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{"spec":{"httpHeaders":{"headerNameCaseAdjustments":["Host"]}}}'

  2. Annotate the route of the application:

     $ oc annotate routes/my-application haproxy.router.openshift.io/h1-adjust-case=true

     The Ingress Controller then adjusts the host request header as specified.

• Specify adjustments using the HeaderNameCaseAdjustments field by configuring the Ingress Controller YAML file.

  1. The following example Ingress Controller YAML adjusts the host header to Host for HTTP/1 requests to appropriately annotated routes:

     Example Ingress Controller YAML

     apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: default
       namespace: openshift-ingress-operator
     spec:
       httpHeaders:
         headerNameCaseAdjustments:
         - Host

  2. The following example route enables HTTP response header name case adjustments using the haproxy.router.openshift.io/h1-adjust-case annotation:

     Example route YAML

     apiVersion: route.openshift.io/v1
     kind: Route
     metadata:
       annotations:
         haproxy.router.openshift.io/h1-adjust-case: true 1
       name: my-application
       namespace: my-application
     spec:
       to:
         kind: Service
         name: my-application

     1

     Set haproxy.router.openshift.io/h1-adjust-case to true.