Chapter 6. Ingress Operator in OpenShift Container Platform

• All the available nodes in the cluster have their own, externally accessible IP addresses. The service running in the cluster is bound to the unique NodePort for all the nodes.
• When the client connects to a node that is down, for example, by connecting the

  10.0.128.4

  IP address in the graphic, the node port directly connects the client to an available node that is running the service. In this scenario, no load balancing is required. As the image shows, the

  10.0.128.4

  address is down and another IP address must be used instead.

Prerequisites

• You must have a certificate/key pair in PEM-encoded files, where the certificate is signed by a trusted certificate authority or by a private trusted certificate authority that you configured in a custom PKI.

• Your certificate meets the following requirements:

  • The certificate is valid for the ingress domain.
  • The certificate uses the

    subjectAltName

    extension to specify a wildcard domain, such as

    *.apps.ocp4.example.com

    .

• You must have an

  IngressController

  CR. You may use the default one:

  $ oc --namespace openshift-ingress-operator get ingresscontrollers

  Example output

  NAME      AGE
  default   10m

Prerequisites

• You have access to the cluster as a user with the

  cluster-admin

  role.
• You have installed the OpenShift CLI (

  oc

  ).
• You previously configured a custom default certificate for the Ingress Controller.

Procedure

• To remove the custom certificate and restore the certificate that ships with OpenShift Container Platform, enter the following command:

  $ oc patch -n openshift-ingress-operator ingresscontrollers/default \
    --type json -p $'- op: remove\n  path: /spec/defaultCertificate'

  There can be a delay while the cluster reconciles the new certificate configuration.

Verification

• To confirm that the original cluster certificate is restored, enter the following command:

  $ echo Q | \
    openssl s_client -connect console-openshift-console.apps.<domain>:443 -showcerts 2>/dev/null | \
    openssl x509 -noout -subject -issuer -enddate

  where:

  <domain>

  Specifies the base domain name for your cluster.

  Example output

  subject=CN = *.apps.<domain>
  issuer=CN = ingress-operator@1620633373
  notAfter=May 10 10:44:36 2023 GMT

Prerequisites

• Log in as a user with

  cluster-admin

  privileges.

• To configure Ingress access logging, you must specify a destination using

  spec.logging.access.destination

  . To specify logging to a sidecar container, you must specify

  Container

  spec.logging.access.destination.type

  . The following example is an Ingress Controller definition that logs to a

  Container

  destination:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Container

• When you configure the Ingress Controller to log to a sidecar, the operator creates a container named

  logs

  inside the Ingress Controller Pod:

  $ oc -n openshift-ingress logs deployment.apps/router-default -c logs

  Example output

  2020-05-11T19:11:50.135710+00:00 router-default-57dfc6cd95-bpmk6 router-default-57dfc6cd95-bpmk6 haproxy[108]: 174.19.21.82:39654 [11/May/2020:19:11:50.133] public be_http:hello-openshift:hello-openshift/pod:hello-openshift:hello-openshift:10.128.2.12:8080 0/0/1/0/1 200 142 - - --NI 1/1/0/0/0 0/0 "GET / HTTP/1.1"

• To configure Ingress access logging, you must specify a destination using

  spec.logging.access.destination

  . To specify logging to a Syslog endpoint destination, you must specify

  Syslog

  for

  spec.logging.access.destination.type

  . If the destination type is

  Syslog

  , you must also specify a destination endpoint using

  spec.logging.access.destination.syslog.endpoint

  and you can specify a facility using

  spec.logging.access.destination.syslog.facility

  . The following example is an Ingress Controller definition that logs to a

  Syslog

  destination:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Syslog
          syslog:
            address: 1.2.3.4
            port: 10514

  Note

  The

  syslog

  destination port must be UDP.

• You can specify

  spec.logging.access.httpLogFormat

  to customize the log format. The following example is an Ingress Controller definition that logs to a

  syslog

  endpoint with IP address 1.2.3.4 and port 10514:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access:
        destination:
          type: Syslog
          syslog:
            address: 1.2.3.4
            port: 10514
        httpLogFormat: '%ci:%cp [%t] %ft %b/%s %B %bq %HM %HU %HV'

• To disable Ingress access logging, leave

  spec.logging

  or

  spec.logging.access

  empty:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    replicas: 2
    logging:
      access: null

Prerequisites

• The following assumes that you already created an Ingress Controller.

Procedure

• Update the Ingress Controller to increase the number of threads:

  $ oc -n openshift-ingress-operator patch ingresscontroller/default --type=merge -p '{"spec":{"tuningOptions": {"threadCount": 8}}}'

  Note

  If you have a node that is capable of running large amounts of resources, you can configure

  spec.nodePlacement.nodeSelector

  with labels that match the capacity of the intended node, and configure

  spec.tuningOptions.threadCount

  to an appropriately high value.

• Balance Ingress Controllers, or routers, with several routes to speed up responses to changes.
• Allocate certain routes to have different reliability guarantees than other routes.
• Allow certain Ingress Controllers to have different policies defined.
• Allow only specific routes to use additional features.
• Expose different routes on different addresses so that internal and external users can see different routes, for example.

• You can load load balance externally, using the cloud provider load balancer, or internally, using the OpenShift Ingress Controller Load Balancer.
• You can use the single IP address of the load balancer and more familiar ports, such as 8080 and 4200 as shown on the cluster depicted in the graphic.
• Traffic from the external load balancer is directed at the pods, and managed by the load balancer, as depicted in the instance of a down node. See the Kubernetes Services documentation for implementation details.

Prerequisites

• Install the OpenShift CLI (

  oc

  ).
• Log in as a user with

  cluster-admin

  privileges.

Prerequisites

• You deployed an OpenShift Container Platform cluster on GCP infrastructure.
• You configured an Ingress Controller to use an internal load balancer.
• You installed the OpenShift CLI (

  oc

  ).

Prerequisites

• Install the OpenShift CLI (

  oc

  ).
• Log in as a user with

  cluster-admin

  privileges.

Prerequisites

• Cluster administrator privileges.

Procedure

• Edit the

  .spec.routeAdmission

  field of the

  ingresscontroller

  resource variable using the following command:

  $ oc -n openshift-ingress-operator patch ingresscontroller/default --patch '{"spec":{"routeAdmission":{"namespaceOwnership":"InterNamespaceAllowed"}}}' --type=merge

  Sample Ingress Controller configuration

  spec:
    routeAdmission:
      namespaceOwnership: InterNamespaceAllowed
  ...

  Tip

  You can alternatively apply the following YAML to configure the route admission policy:

  apiVersion: operator.openshift.io/v1
  kind: IngressController
  metadata:
    name: default
    namespace: openshift-ingress-operator
  spec:
    routeAdmission:
      namespaceOwnership: InterNamespaceAllowed

• Configure an external proxy that injects the

  X-Forwarded-For

  header into each request before forwarding it to an Ingress Controller.

  To configure the Ingress Controller to pass the header through unmodified, you specify the

  never

  policy. The Ingress Controller then never sets the headers, and applications receive only the headers that the external proxy provides.

• Configure the Ingress Controller to pass the

  X-Forwarded-For

  header that your external proxy sets on external cluster requests through unmodified.

  To configure the Ingress Controller to set the

  X-Forwarded-For

  header on internal cluster requests, which do not go through the external proxy, specify the

  if-none

  policy. If an HTTP request already has the header set through the external proxy, then the Ingress Controller preserves it. If the header is absent because the request did not come through the proxy, then the Ingress Controller adds the header.

• Configure an application-specific external proxy that injects the

  X-Forwarded-For

  header.

  To configure an Ingress Controller to pass the header through unmodified for an application’s Route, without affecting the policy for other Routes, add an annotation

  haproxy.router.openshift.io/set-forwarded-headers: if-none

  or

  haproxy.router.openshift.io/set-forwarded-headers: never

  on the Route for the application.
  Note

  You can set the

  haproxy.router.openshift.io/set-forwarded-headers

  annotation on a per route basis, independent from the globally set value for the Ingress Controller.

• To enable HTTP/2 on an Ingress Controller, enter the

  oc annotate

  command:

  $ oc -n openshift-ingress-operator annotate ingresscontrollers/<ingresscontroller_name> ingress.operator.openshift.io/default-enable-http2=true

  Replace

  <ingresscontroller_name>

  with the name of the Ingress Controller to annotate.

• To enable HTTP/2 for the entire cluster, enter the

  oc annotate

  command:

  $ oc annotate ingresses.config/cluster ingress.operator.openshift.io/default-enable-http2=true

  Tip

  You can alternatively apply the following YAML to add the annotation:

  apiVersion: config.openshift.io/v1
  kind: Ingress
  metadata:
    name: cluster
    annotations:
      ingress.operator.openshift.io/default-enable-http2: "true"

Prerequisites

• You created an Ingress Controller.

Prerequisites

• You deployed an OpenShift Container Platform cluster.
• You installed the

  oc

  command line interface.

Prerequisites

• You have installed the OpenShift CLI (

  oc

  ).
• You have access to the cluster as a user with the

  cluster-admin

  role.

• Specify an HTTP header to be capitalized by entering the

  oc patch

  command.

  1. Enter the

     oc patch

     command to change the HTTP

     host

     header to

     Host

     :

     $ oc -n openshift-ingress-operator patch ingresscontrollers/default --type=merge --patch='{"spec":{"httpHeaders":{"headerNameCaseAdjustments":["Host"]}}}'

  2. Annotate the route of the application:

     $ oc annotate routes/my-application haproxy.router.openshift.io/h1-adjust-case=true

     The Ingress Controller then adjusts the

     host

     request header as specified.

• Specify adjustments using the

  HeaderNameCaseAdjustments

  field by configuring the Ingress Controller YAML file.

  1. The following example Ingress Controller YAML adjusts the

     host

     header to

     Host

     for HTTP/1 requests to appropriately annotated routes:

     Example Ingress Controller YAML

     apiVersion: operator.openshift.io/v1
     kind: IngressController
     metadata:
       name: default
       namespace: openshift-ingress-operator
     spec:
       httpHeaders:
         headerNameCaseAdjustments:
         - Host

  2. The following example route enables HTTP response header name case adjustments using the

     haproxy.router.openshift.io/h1-adjust-case

     annotation:

     Example route YAML

     apiVersion: route.openshift.io/v1
     kind: Route
     metadata:
       annotations:
         haproxy.router.openshift.io/h1-adjust-case: true 

     1

     
       name: my-application
       namespace: my-application
     spec:
       to:
         kind: Service
         name: my-application

     1

     Set haproxy.router.openshift.io/h1-adjust-case to true.