Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Ingress and load balancing

OpenShift Dedicated 4

Exposing services and managing external traffic in OpenShift Dedicated

Red Hat OpenShift Documentation Team

Abstract

This document explains how to configure routes, manage ingress traffic, and implement various load balancing solutions in OpenShift Dedicated.

Chapter 1. Routes
Copy link

1.1. Creating basic routes
Copy link

If you have unencrypted HTTP, you can create a basic route with a route object.

1.1.1. Creating an HTTP-based route
Copy link

You can use the following procedure to create a simple HTTP-based route to a web application, using the hello-openshift application as an example.

You can create a route to host your application at a public URL. The route can either be secure or unsecured, depending on the network security configuration of your application. An HTTP-based route is an unsecured route that uses the basic HTTP routing protocol and exposes a service on an unsecured application port.

Prerequisites

  • You installed the OpenShift CLI (oc).
  • You are logged in as an administrator.
  • You have a web application that exposes a port and a TCP endpoint listening for traffic on the port.

Procedure

  1. Create a project called hello-openshift by running the following command: 

    $ oc new-project hello-openshift
    Copy to Clipboard Toggle word wrap

  2. Create a pod in the project by running the following command: 

    $ oc create -f https://raw.githubusercontent.com/openshift/origin/master/examples/hello-openshift/hello-pod.json
    Copy to Clipboard Toggle word wrap

  3. Create a service called hello-openshift by running the following command: 

    $ oc expose pod/hello-openshift
    Copy to Clipboard Toggle word wrap

  4. Create an unsecured route to the hello-openshift application by running the following command: 

    $ oc expose svc hello-openshift
    Copy to Clipboard Toggle word wrap

Verification

  • To verify that the route resource that you created, run the following command: 

    $ oc get routes -o yaml hello-openshift
    Copy to Clipboard Toggle word wrap

    Example YAML definition of the created unsecured route

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: hello-openshift
spec:
  host: www.example.com
  port:
    targetPort: 8080
  to:
    kind: Service
    name: hello-openshift
    Copy to Clipboard Toggle word wrap

    where:

    host
    Specifies an alias DNS record that points to the service. This field can be any valid DNS name, such as www.example.com. The DNS name must follow DNS952 subdomain conventions. If not specified, a route name is automatically generated.
    targetPort

    Specifies the target port on pods that is selected by the service that this route points to.

    Note

    To display your default ingress domain, run the following command: 

    $ oc get ingresses.config/cluster -o jsonpath={.spec.domain}
    Copy to Clipboard Toggle word wrap

1.1.2. Path-based routes
Copy link

To serve multiple applications by using a single hostname, configure path-based routes. This HTTP-based configuration directs traffic to specific services by comparing the URL path component, ensuring requests match the most specific route defined.

The following table shows example routes and their accessibility:

Expand
Table 1.1. Route availability
RouteWhen compared toAccessible

www.example.com/test

www.example.com/test

Yes

www.example.com

No

www.example.com/test and www.example.com

www.example.com/test

Yes

www.example.com

Yes

www.example.com

www.example.com/text

Yes (Matched by the host, not the route)

www.example.com

Yes

Example of an unsecured route with a path

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: route-unsecured
spec:
  host: www.example.com
  path: "/test"
  to:
    kind: Service
    name: service-name
Copy to Clipboard Toggle word wrap

  • spec.host: Specifies the path attribute for a path-based route.
Note

Path-based routing is not available when using passthrough TLS, as the router does not terminate TLS in that case and cannot read the contents of the request.

1.2. Securing routes
Copy link

You can secure a route with HTTP strict transport security (HSTS).

1.2.1. HTTP Strict Transport Security
Copy link

To enhance security and optimize website performance, use the HTTP Strict Transport Security (HSTS) policy. This mechanism signals browsers to use only HTTPS traffic on the route host, eliminating the need for HTTP redirects and speeding up user interactions.

When HSTS policy is enforced, HSTS adds a Strict Transport Security header to HTTP and HTTPS responses from the site. You can use the insecureEdgeTerminationPolicy value in a route to redirect HTTP to HTTPS. When HSTS is enforced, the client changes all requests from the HTTP URL to HTTPS before the request is sent, eliminating the need for a redirect.

Cluster administrators can configure HSTS to do the following:

  • Enable HSTS per-route
  • Disable HSTS per-route
  • Enforce HSTS per-domain, for a set of domains, or use namespace labels in combination with domains
Important

HSTS works only with secure routes, either edge-terminated or re-encrypt. The configuration is ineffective on HTTP or passthrough routes.

To enforce secure HTTPS connections for specific applications, enable HTTP Strict Transport Security (HSTS) on a per-route basis. Applying the haproxy.router.openshift.io/hsts_header annotation to edge and re-encrypt routes ensures that browsers reject unencrypted traffic.

Prerequisites

  • You are logged in to the cluster with a user with administrator privileges for the project.
  • You installed the OpenShift CLI (oc).

Procedure

  • To enable HSTS on a route, add the haproxy.router.openshift.io/hsts_header value to the edge-terminated or re-encrypt route. You can use the oc annotate tool to do this by running the following command. To properly run the command, ensure that the semicolon (;) in the haproxy.router.openshift.io/hsts_header route annotation is also surrounded by double quotation marks ("").

    Example annotate command that sets the maximum age to 31536000 ms (approximately 8.5 hours)

    $ oc annotate route <route_name> -n <namespace> --overwrite=true "haproxy.router.openshift.io/hsts_header=max-age=31536000;\
includeSubDomains;preload"
    Copy to Clipboard Toggle word wrap

    Example route configured with an annotation

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/hsts_header: max-age=31536000;includeSubDomains;preload
# ...
spec:
  host: def.abc.com
  tls:
    termination: "reencrypt"
    ...
  wildcardPolicy: "Subdomain"
# ...
    Copy to Clipboard Toggle word wrap

    where:

    max-age
    Specifies the measurement of the length of time, in seconds, for the HSTS policy. If set to 0, it negates the policy.
    includeSubDomains
    Specifies that all subdomains of the host must have the same HSTS policy as the host. Optional parameter.
    preload
    Specifies that the site is included in the HSTS preload list when max-age is greater than 0. For example, sites such as Google can construct a list of sites that have preload set. Browsers can then use these lists to determine which sites they can communicate with over HTTPS, even before they have interacted with the site. Without preload set, browsers must have interacted with the site over HTTPS, at least once, to get the header. Optional parameter.

To allow unencrypted connections or troubleshoot access issues, disable HTTP Strict Transport Security (HSTS) for a specific route. Setting the max-age route annotation to 0 instructs browsers to stop enforcing HTTPS requirements on the route host.

Prerequisites

  • You are logged in to the cluster with a user with administrator privileges for the project.
  • You installed the OpenShift CLI (oc).

Procedure

  • To disable HSTS, enter the following to set the max-age value in the route annotation to 0: 

    $ oc annotate route <route_name> -n <namespace> --overwrite=true "haproxy.router.openshift.io/hsts_header"="max-age=0"
    Copy to Clipboard Toggle word wrap
    Tip

    You can alternatively apply the following YAML to create the config map:

    Example of disabling HSTS per-route

    kind: Route
apiVersion: route.openshift.io/v1
metadata:
  annotations:
    haproxy.router.openshift.io/hsts_header: max-age=0
    Copy to Clipboard Toggle word wrap

  • To disable HSTS for every route in a namespace, enter the following command: 

    $ oc annotate route --all -n <namespace> --overwrite=true "haproxy.router.openshift.io/hsts_header"="max-age=0"
    Copy to Clipboard Toggle word wrap

Verification

  • To query the annotation for all routes, enter the following command: 

    $ oc get route  --all-namespaces -o go-template='{{range .items}}{{if .metadata.annotations}}{{$a := index .metadata.annotations "haproxy.router.openshift.io/hsts_header"}}{{$n := .metadata.name}}{{with $a}}Name: {{$n}} HSTS: {{$a}}{{"\n"}}{{else}}{{""}}{{end}}{{end}}{{end}}'
    Copy to Clipboard Toggle word wrap

    Example output

    Name: routename HSTS: max-age=0
    Copy to Clipboard Toggle word wrap

To enforce HTTP Strict Transport Security (HSTS) per-domain for secure routes, add a requiredHSTSPolicies record to the Ingress spec to capture the configuration of the HSTS policy.

If you configure a requiredHSTSPolicy to enforce HSTS, then any newly created route must be configured with a compliant HSTS policy annotation.

Note

To handle upgraded clusters with non-compliant HSTS routes, you can update the manifests at the source and apply the updates.

Note

You cannot use oc expose route or oc create route commands to add a route in a domain that enforces HSTS, because the API for these commands does not accept annotations.

Important

HSTS cannot be applied to insecure, or non-TLS routes, even if HSTS is requested for all routes globally.

Prerequisites

  • You are logged in to the cluster with a user with administrator privileges for the project.
  • You installed the OpenShift CLI (oc).

Procedure

  1. Edit the Ingress configuration YAML by running the following command and updating fields as needed: 

    $ oc edit ingresses.config.openshift.io/cluster
    Copy to Clipboard Toggle word wrap

    Example HSTS policy

    apiVersion: config.openshift.io/v1
kind: Ingress
metadata:
  name: cluster
spec:
  domain: 'hello-openshift-default.apps.username.devcluster.openshift.com'
  requiredHSTSPolicies:
    1 
    
  - domainPatterns:
    2 
    
    - '*hello-openshift-default.apps.username.devcluster.openshift.com'
    - '*hello-openshift-default2.apps.username.devcluster.openshift.com'
    namespaceSelector:
    3 
    
      matchLabels:
        myPolicy: strict
    maxAge:
    4 
    
      smallestMaxAge: 1
      largestMaxAge: 31536000
    preloadPolicy: RequirePreload
    5 
    
    includeSubDomainsPolicy: RequireIncludeSubDomains
    6 
    
  - domainPatterns:
    - 'abc.example.com'
    - '*xyz.example.com'
    namespaceSelector:
      matchLabels: {}
    maxAge: {}
    preloadPolicy: NoOpinion
    includeSubDomainsPolicy: RequireNoIncludeSubDomains
    Copy to Clipboard Toggle word wrap

    1
    Required. requiredHSTSPolicies are validated in order, and the first matching domainPatterns applies.
    2
    Required. You must specify at least one domainPatterns hostname. Any number of domains can be listed. You can include multiple sections of enforcing options for different domainPatterns.
    3
    Optional. If you include namespaceSelector, it must match the labels of the project where the routes reside, to enforce the set HSTS policy on the routes. Routes that only match the namespaceSelector and not the domainPatterns are not validated.
    4
    Required. max-age measures the length of time, in seconds, that the HSTS policy is in effect. This policy setting allows for a smallest and largest max-age to be enforced.
    • The largestMaxAge value must be between 0 and 2147483647. It can be left unspecified, which means no upper limit is enforced.
    • The smallestMaxAge value must be between 0 and 2147483647. Enter 0 to disable HSTS for troubleshooting, otherwise enter 1 if you never want HSTS to be disabled. It can be left unspecified, which means no lower limit is enforced.
    5
    Optional. Including preload in haproxy.router.openshift.io/hsts_header allows external services to include this site in their HSTS preload lists. Browsers can then use these lists to determine which sites they can communicate with over HTTPS, before they have interacted with the site. Without preload set, browsers need to interact at least once with the site to get the header. preload can be set with one of the following:
    • RequirePreload: preload is required by the RequiredHSTSPolicy.
    • RequireNoPreload: preload is forbidden by the RequiredHSTSPolicy.
    • NoOpinion: preload does not matter to the RequiredHSTSPolicy.
    6
    Optional. includeSubDomainsPolicy can be set with one of the following:
    • RequireIncludeSubDomains: includeSubDomains is required by the RequiredHSTSPolicy.
    • RequireNoIncludeSubDomains: includeSubDomains is forbidden by the RequiredHSTSPolicy.
    • NoOpinion: includeSubDomains does not matter to the RequiredHSTSPolicy.

  2. You can apply HSTS to all routes in the cluster or in a particular namespace by entering the oc annotate command.

    • To apply HSTS to all routes in the cluster, enter the oc annotate command. For example: 

      $ oc annotate route --all --all-namespaces --overwrite=true "haproxy.router.openshift.io/hsts_header"="max-age=31536000"
      Copy to Clipboard Toggle word wrap

    • To apply HSTS to all routes in a particular namespace, enter the oc annotate command. For example: 

      $ oc annotate route --all -n my-namespace --overwrite=true "haproxy.router.openshift.io/hsts_header"="max-age=31536000"
      Copy to Clipboard Toggle word wrap

Verification

You can review the HSTS policy you configured. For example:

  • To review the maxAge set for required HSTS policies, enter the following command: 

    $ oc get clusteroperator/ingress -n openshift-ingress-operator -o jsonpath='{range .spec.requiredHSTSPolicies[*]}{.spec.requiredHSTSPolicies.maxAgePolicy.largestMaxAge}{"\n"}{end}'
    Copy to Clipboard Toggle word wrap

  • To review the HSTS annotations on all routes, enter the following command: 

    $ oc get route  --all-namespaces -o go-template='{{range .items}}{{if .metadata.annotations}}{{$a := index .metadata.annotations "haproxy.router.openshift.io/hsts_header"}}{{$n := .metadata.name}}{{with $a}}Name: {{$n}} HSTS: {{$a}}{{"\n"}}{{else}}{{""}}{{end}}{{end}}{{end}}'
    Copy to Clipboard Toggle word wrap

    Example output

    Name: <_routename_> HSTS: max-age=31536000;preload;includeSubDomains
    Copy to Clipboard Toggle word wrap

1.3. Configuring routes
Copy link

To customise route configuration for specific traffic behaviors, apply annotations, headers, and cookies. By using these mechanisms, you can define granular routing rules, extending standard capabilities to meet complex application requirements.

1.3.1. Configuring route timeouts
Copy link

You can configure the default timeouts for an existing route when you have services in need of a low timeout, which is required for Service Level Availability (SLA) purposes, or a high timeout, for cases with a slow back end.

Important

If you configured a user-managed external load balancer in front of your OpenShift Dedicated cluster, ensure that the timeout value for the user-managed external load balancer is higher than the timeout value for the route. This configuration prevents network congestion issues over the network that your cluster uses.

Prerequisites

  • You deployed an Ingress Controller on a running cluster.

Procedure

  • Using the oc annotate command, add the timeout to the route: 

    $ oc annotate route <route_name> \
    --overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit>
    Copy to Clipboard Toggle word wrap

  • <timeout>: Supported time units are microseconds (us), milliseconds (ms), seconds (s), minutes (m), hours (h), or days (d).

    The following example sets a timeout of two seconds on a route named myroute: 

    $ oc annotate route myroute --overwrite haproxy.router.openshift.io/timeout=2s
    Copy to Clipboard Toggle word wrap

1.3.2. HTTP header configuration
Copy link

To customize request and response headers for your applications, configure the Ingress Controller or apply specific route annotations. Understanding the interaction between these configuration methods ensures you effectively manage global and route-specific header policies.

You can also set certain headers by using route annotations. The various ways of configuring headers can present challenges when working together.

Note

You can only set or delete headers within an IngressController or Route CR, you cannot append them. If an HTTP header is set with a value, that value must be complete and not require appending in the future. In situations where it makes sense to append a header, such as the X-Forwarded-For header, use the spec.httpHeaders.forwardedHeaderPolicy field, instead of spec.httpHeaders.actions.

Order of precedence

When the same HTTP header is modified both in the Ingress Controller and in a route, HAProxy prioritizes the actions in certain ways depending on whether it is a request or response header.

  • For HTTP response headers, actions specified in the Ingress Controller are executed after the actions specified in a route. This means that the actions specified in the Ingress Controller take precedence.
  • For HTTP request headers, actions specified in a route are executed after the actions specified in the Ingress Controller. This means that the actions specified in the route take precedence.

For example, a cluster administrator sets the X-Frame-Options response header with the value DENY in the Ingress Controller using the following configuration:

Example IngressController spec

apiVersion: operator.openshift.io/v1
kind: IngressController
# ...
spec:
  httpHeaders:
    actions:
      response:
      - name: X-Frame-Options
        action:
          type: Set
          set:
            value: DENY
Copy to Clipboard Toggle word wrap

A route owner sets the same response header that the cluster administrator set in the Ingress Controller, but with the value SAMEORIGIN using the following configuration:

Example Route spec

apiVersion: route.openshift.io/v1
kind: Route
# ...
spec:
  httpHeaders:
    actions:
      response:
      - name: X-Frame-Options
        action:
          type: Set
          set:
            value: SAMEORIGIN
Copy to Clipboard Toggle word wrap

When both the IngressController spec and Route spec are configuring the X-Frame-Options response header, then the value set for this header at the global level in the Ingress Controller takes precedence, even if a specific route allows frames. For a request header, the Route spec value overrides the IngressController spec value.

This prioritization occurs because the haproxy.config file uses the following logic, where the Ingress Controller is considered the front end and individual routes are considered the back end. The header value DENY applied to the front end configurations overrides the same header with the value SAMEORIGIN that is set in the back end: 

frontend public
  http-response set-header X-Frame-Options 'DENY'

frontend fe_sni
  http-response set-header X-Frame-Options 'DENY'

frontend fe_no_sni
  http-response set-header X-Frame-Options 'DENY'

backend be_secure:openshift-monitoring:alertmanager-main
  http-response set-header X-Frame-Options 'SAMEORIGIN'
Copy to Clipboard Toggle word wrap

Additionally, any actions defined in either the Ingress Controller or a route override values set using route annotations.

Special case headers
The following headers are either prevented entirely from being set or deleted, or allowed under specific circumstances:
Expand
Table 1.2. Special case header configuration options
Header nameConfigurable using IngressController specConfigurable using Route specReason for disallowmentConfigurable using another method

proxy

No

No

The proxy HTTP request header can be used to exploit vulnerable CGI applications by injecting the header value into the HTTP_PROXY environment variable. The proxy HTTP request header is also non-standard and prone to error during configuration.

No

host

No

Yes

When the host HTTP request header is set using the IngressController CR, HAProxy can fail when looking up the correct route.

No

strict-transport-security

No

No

The strict-transport-security HTTP response header is already handled using route annotations and does not need a separate implementation.

Yes: the haproxy.router.openshift.io/hsts_header route annotation

cookie and set-cookie

No

No

The cookies that HAProxy sets are used for session tracking to map client connections to particular back-end servers. Allowing these headers to be set could interfere with HAProxy’s session affinity and restrict HAProxy’s ownership of a cookie.

Yes:

  • the haproxy.router.openshift.io/disable_cookie route annotation
  • the haproxy.router.openshift.io/cookie_name route annotation

You can set or delete certain HTTP request and response headers for compliance purposes or other reasons. You can set or delete these headers either for all routes served by an Ingress Controller or for specific routes.

For example, you might want to enable a web application to serve content in alternate locations for specific routes if that content is written in multiple languages, even if there is a default global location specified by the Ingress Controller serving the routes.

The following procedure creates a route that sets the Content-Location HTTP request header so that the URL associated with the application, https://app.example.com, directs to the location https://app.example.com/lang/en-us. Directing application traffic to this location means that anyone using that specific route is accessing web content written in American English.

Prerequisites

  • You have installed the OpenShift CLI (oc).
  • You are logged into an OpenShift Dedicated cluster as a project administrator.
  • You have a web application that exposes a port and an HTTP or TLS endpoint listening for traffic on the port.

Procedure

  1. Create a route definition and save it in a file called app-example-route.yaml:

    YAML definition of the created route with HTTP header directives

    apiVersion: route.openshift.io/v1
kind: Route
# ...
spec:
  host: app.example.com
  tls:
    termination: edge
  to:
    kind: Service
    name: app-example
  httpHeaders:
    actions:
      response:
      - name: Content-Location
        action:
          type: Set
          set:
            value: /lang/en-us
# ...
    Copy to Clipboard Toggle word wrap

    where:

    actions
    Specifies the list of actions you want to perform on the HTTP headers.
    response
    Specifies the type of header you want to change. In this case, a response header.
    response.name
    Specifies the name of the header you want to change. For a list of available headers you can set or delete, see HTTP header configuration.
    action.type
    Specifies the type of action being taken on the header. This field can have the value Set or Delete.
    set.value
    When setting HTTP headers, you must provide a value. The value can be a string from a list of available directives for that header, for example DENY, or it can be a dynamic value that will be interpreted using HAProxy’s dynamic value syntax. In this case, the value is set to the relative location of the content.

  2. Create a route to your existing web application using the newly created route definition: 

    $ oc -n app-example create -f app-example-route.yaml
    Copy to Clipboard Toggle word wrap

    For HTTP request headers, the actions specified in the route definitions are executed after any actions performed on HTTP request headers in the Ingress Controller. This means that any values set for those request headers in a route will take precedence over the ones set in the Ingress Controller. For more information on the processing order of HTTP headers, see HTTP header configuration.

1.3.4. Using cookies to keep route statefulness
Copy link

To maintain stateful application traffic during pod restarts or scaling events, configure sticky sessions by using cookies. By using this method, you ensure that all incoming traffic reaches the same endpoint, preventing state loss even if the specific endpoint pod changes.

OpenShift Dedicated can use cookies to configure session persistence. The Ingress Controller selects an endpoint to handle any user requests, and creates a cookie for the session. The cookie is passed back in the response to the request and the user sends the cookie back with the next request in the session. The cookie tells the Ingress Controller which endpoint is handling the session, ensuring that client requests use the cookie so that they are routed to the same pod.

Note

Cookies cannot be set on passthrough routes, because the HTTP traffic cannot be seen. Instead, a number is calculated based on the source IP address, which determines the backend.

If backends change, the traffic can be directed to the wrong server, making it less sticky. If you are using a load balancer, which hides source IP, the same number is set for all connections and traffic is sent to the same pod.

1.3.5. Route-specific annotations
Copy link

The Ingress Controller can set the default options for all the routes it exposes. An individual route can override some of these defaults by providing specific configurations in its annotations. Red Hat does not support adding a route annotation to an operator-managed route.

Important

To create an allow list with multiple source IPs or subnets, use a space-delimited list. Any other delimiter type causes the list to be ignored without a warning or error message.

Expand
Table 1.3. Route annotations
VariableDescription

haproxy.router.openshift.io/balance

Sets the load-balancing algorithm. Available options are random, source, roundrobin[1], and leastconn. The default value is source for TLS passthrough routes. For all other routes, the default is random.

haproxy.router.openshift.io/disable_cookies

Disables the use of cookies to track related connections. If set to 'true' or 'TRUE', the balance algorithm is used to choose which back-end serves connections for each incoming HTTP request.

router.openshift.io/cookie_name

Specifies an optional cookie to use for this route. The name must consist of any combination of upper and lower case letters, digits, "_", and "-". The default is the hashed internal key name for the route.

haproxy.router.openshift.io/pod-concurrent-connections

Sets the maximum number of connections that are allowed to a backing pod from a router.
Note: If there are multiple pods, each can have this many connections. If you have multiple routers, there is no coordination among them, each may connect this many times. If not set, or set to 0, there is no limit.

haproxy.router.openshift.io/rate-limit-connections

Setting 'true' or 'TRUE' enables rate limiting functionality which is implemented through stick-tables on the specific backend per route.
Note: Using this annotation provides basic protection against denial-of-service attacks.

haproxy.router.openshift.io/rate-limit-connections.concurrent-tcp

Limits the number of concurrent TCP connections made through the same source IP address. It accepts a numeric value.
Note: Using this annotation provides basic protection against denial-of-service attacks.

haproxy.router.openshift.io/rate-limit-connections.rate-http

Limits the rate at which a client with the same source IP address can make HTTP requests. It accepts a numeric value.
Note: Using this annotation provides basic protection against denial-of-service attacks.

haproxy.router.openshift.io/rate-limit-connections.rate-tcp

Limits the rate at which a client with the same source IP address can make TCP connections. It accepts a numeric value.

router.openshift.io/haproxy.health.check.interval

Sets the interval for the back-end health checks. (TimeUnits)

haproxy.router.openshift.io/ip_allowlist

Sets an allowlist for the route. The allowlist is a space-separated list of IP addresses and CIDR ranges for the approved source addresses. Requests from IP addresses that are not in the allowlist are dropped.

The maximum number of IP addresses and CIDR ranges directly visible in the haproxy.config file is 61. [2]

haproxy.router.openshift.io/hsts_header

Sets a Strict-Transport-Security header for the edge terminated or re-encrypt route.

haproxy.router.openshift.io/rewrite-target

Sets the rewrite path of the request on the backend.

router.openshift.io/cookie-same-site

Sets a value to restrict cookies. The values are:

Lax: the browser does not send cookies on cross-site requests, but does send cookies when users navigate to the origin site from an external site. This is the default browser behavior when the SameSite value is not specified.

Strict: the browser sends cookies only for same-site requests.

None: the browser sends cookies for both cross-site and same-site requests.

This value is applicable to re-encrypt and edge routes only. For more information, see the SameSite cookies documentation.

haproxy.router.openshift.io/set-forwarded-headers

Sets the policy for handling the Forwarded and X-Forwarded-For HTTP headers per route. The values are:

append: appends the header, preserving any existing header. This is the default value.

replace: sets the header, removing any existing header.

never: never sets the header, but preserves any existing header.

if-none: sets the header if it is not already set.

  1. By default, the router reloads every 5 s which resets the balancing connection across pods from the beginning. As a result, the roundrobin state is not preserved across reloads. This algorithm works best when pods have nearly identical computing capabilites and storage capacity. If your application or service has continuously changing endpoints, for example, due to the use of a CI/CD pipeline, uneven balancing can result. In this case, use a different algorithm.

  2. If the number of IP addresses and CIDR ranges in an allowlist exceeds 61, they are written into a separate file that is then referenced from the haproxy.config file. This file is stored in the /var/lib/haproxy/router/allowlists folder.

    Note

    To ensure that the addresses are written to the allowlist, check that the full list of CIDR ranges are listed in the Ingress Controller configuration file. The etcd object size limit restricts how large a route annotation can be. Because of this, it creates a threshold for the maximum number of IP addresses and CIDR ranges that you can include in an allowlist.

A route that allows only one specific IP address

metadata:
  annotations:
    haproxy.router.openshift.io/ip_allowlist: 192.168.1.10
Copy to Clipboard Toggle word wrap

A route that allows several IP addresses

metadata:
  annotations:
    haproxy.router.openshift.io/ip_allowlist: 192.168.1.10 192.168.1.11 192.168.1.12
Copy to Clipboard Toggle word wrap

A route that allows an IP address CIDR network

metadata:
  annotations:
    haproxy.router.openshift.io/ip_allowlist: 192.168.1.0/24
Copy to Clipboard Toggle word wrap

A route that allows both IP an address and IP address CIDR networks

metadata:
  annotations:
    haproxy.router.openshift.io/ip_allowlist: 180.5.61.153 192.168.1.0/24 10.0.0.0/8
Copy to Clipboard Toggle word wrap

A route specifying a rewrite target

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/rewrite-target: /
1 

...
Copy to Clipboard Toggle word wrap

1
Sets / as rewrite path of the request on the backend.

Setting the haproxy.router.openshift.io/rewrite-target annotation on a route specifies that the Ingress Controller should rewrite paths in HTTP requests using this route before forwarding the requests to the backend application. The part of the request path that matches the path specified in spec.path is replaced with the rewrite target specified in the annotation.

The following table provides examples of the path rewriting behavior for various combinations of spec.path, request path, and rewrite target.

Expand
Table 1.4. rewrite-target examples
Route.spec.pathRequest pathRewrite targetForwarded request path

/foo

/foo

/

/

/foo

/foo/

/

/

/foo

/foo/bar

/

/bar

/foo

/foo/bar/

/

/bar/

/foo

/foo

/bar

/bar

/foo

/foo/

/bar

/bar/

/foo

/foo/bar

/baz

/baz/bar

/foo

/foo/bar/

/baz

/baz/bar/

/foo/

/foo

/

N/A (request path does not match route path)

/foo/

/foo/

/

/

/foo/

/foo/bar

/

/bar

Certain special characters in haproxy.router.openshift.io/rewrite-target require special handling because they must be escaped properly. Refer to the following table to understand how these characters are handled.

Expand
Table 1.5. Special character handling
For characterUse charactersNotes

#

\#

Avoid # because it terminates the rewrite expression

%

% or %%

Avoid odd sequences such as %%%

‘

\’

Avoid ‘ because it is ignored

All other valid URL characters can be used without escaping.

1.3.6. Throughput issue troubleshooting methods
Copy link

To diagnose and resolve network throughput issues, such as unusually high latency between specific services, apply troubleshooting methods. Identifying connectivity bottlenecks helps ensure stable application performance within OpenShift Dedicated.

If pod logs do not reveal any cause of the problem, use the following methods to analyze performance issues:

  • Use a packet analyzer, such as ping or tcpdump to analyze traffic between a pod and its node.

    For example, run the tcpdump tool on each pod while reproducing the behavior that led to the issue. Review the captures on both sides to compare send and receive timestamps to analyze the latency of traffic to and from a pod. Latency can occur in OpenShift Dedicated if a node interface is overloaded with traffic from other pods, storage devices, or the data plane. 

    $ tcpdump -s 0 -i any -w /tmp/dump.pcap host <podip 1> && host <podip 2>
    1
    Copy to Clipboard Toggle word wrap

    where:

    podip

    Specifies the IP address for the pod. Run the oc get pod <pod_name> -o wide command to get the IP address of a pod.

    The tcpdump command generates a file at /tmp/dump.pcap containing all traffic between these two pods. You can run the analyzer shortly before the issue is reproduced and stop the analyzer shortly after the issue is finished reproducing to minimize the size of the file. You can also run a packet analyzer between the nodes with: 

    $ tcpdump -s 0 -i any -w /tmp/dump.pcap port 4789
    Copy to Clipboard Toggle word wrap
  • Use a bandwidth measuring tool, such as iperf, to measure streaming throughput and UDP throughput. Locate any bottlenecks by running the tool from the pods first, and then running it from the nodes.
  • In some cases, the cluster might mark the node with the router pod as unhealthy due to latency issues. Use worker latency profiles to adjust the frequency that the cluster waits for a status update from the node before taking action.
  • If your cluster has designated lower-latency and higher-latency nodes, configure the spec.nodePlacement field in the Ingress Controller to control the placement of the router pod.

1.4. Creating advanced routes
Copy link

To secure application traffic and serve custom certificates to clients, configure routes by using edge, passthrough, or re-encrypt TLS termination. By using these methods, you can define granular encryption rules, ensuring that traffic is decrypted and re-encrypted according to your specific security requirements.

To secure traffic by using a custom certificate, configure a route with edge TLS termination by running the oc create route command. This configuration terminates encryption at the Ingress Controller before forwarding traffic to the destination pod.

The route specifies the TLS certificate and key that the Ingress Controller uses for the route.

The procedure creates a Route resource with a custom certificate and edge TLS termination. The procedure assumes that the certificate/key pair are in the tls.crt and tls.key files in the current working directory. You may also specify a CA certificate if needed to complete the certificate chain. Substitute the actual path names for tls.crt, tls.key, and (optionally) ca.crt. Substitute the name of the service that you want to expose for frontend. Substitute the appropriate hostname for www.example.com.

Prerequisites

  • You must have a certificate/key pair in PEM-encoded files, where the certificate is valid for the route host.
  • You might have a separate CA certificate in a PEM-encoded file that completes the certificate chain.
  • You must have a service that you want to expose.
Note

Password protected key files are not supported. To remove a passphrase from a key file, use the following command: 

$ openssl rsa -in password_protected_tls.key -out tls.key
Copy to Clipboard Toggle word wrap

Procedure

  • Create a secure Route resource using edge TLS termination and a custom certificate. 

    $ oc create route edge --service=frontend --cert=tls.crt --key=tls.key --ca-cert=ca.crt --hostname=www.example.com
    Copy to Clipboard Toggle word wrap

    If you examine the resulting Route resource, the resource should have a configuration similar to the following example:

    YAML Definition of the Secure Route

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: frontend
spec:
  host: www.example.com
  to:
    kind: Service
    name: frontend
  tls:
    termination: edge
    key: |-
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
# ...
    Copy to Clipboard Toggle word wrap

    See oc create route edge --help for more options.

To secure traffic by using a custom certificate, configure a route with re-encrypt TLS termination by running the oc create route command. This configuration enables the Ingress Controller to decrypt traffic, and then re-encrypt traffic before forwarding the traffic to the destination pod.

The procedure creates a Route resource with a custom certificate and reencrypt TLS termination. The procedure assumes that the certificate/key pair are in the tls.crt and tls.key files in the current working directory. You must also specify a destination CA certificate to enable the Ingress Controller to trust the service’s certificate. You may also specify a CA certificate if needed to complete the certificate chain. Substitute the actual path names for tls.crt, tls.key, cacert.crt, and (optionally) ca.crt. Substitute the name of the Service resource that you want to expose for frontend. Substitute the appropriate hostname for www.example.com.

Prerequisites

  • You must have a certificate/key pair in PEM-encoded files, where the certificate is valid for the route host.
  • You may have a separate CA certificate in a PEM-encoded file that completes the certificate chain.
  • You must have a separate destination CA certificate in a PEM-encoded file.
  • You must have a service that you want to expose.
Note

Password protected key files are not supported. To remove a passphrase from a key file, use the following command: 

$ openssl rsa -in password_protected_tls.key -out tls.key
Copy to Clipboard Toggle word wrap

Procedure

  • Create a secure Route resource using reencrypt TLS termination and a custom certificate: 

    $ oc create route reencrypt --service=frontend --cert=tls.crt --key=tls.key --dest-ca-cert=destca.crt --ca-cert=ca.crt --hostname=www.example.com
    Copy to Clipboard Toggle word wrap

    If you examine the resulting Route resource, the resource should have a configuration similar to the following example:

    YAML Definition of the Secure Route

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: frontend
spec:
  host: www.example.com
  to:
    kind: Service
    name: frontend
  tls:
    termination: reencrypt
    key: |-
      -----BEGIN PRIVATE KEY-----
      [...]
      -----END PRIVATE KEY-----
    certificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    caCertificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    destinationCACertificate: |-
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
# ...
    Copy to Clipboard Toggle word wrap

    See oc create route reencrypt --help for more options.

1.4.3. Creating a passthrough route
Copy link

To send encrypted traffic directly to the destination without decryption at the router, configure a route with passthrough termination by running the oc create route command. This configuration requires no key or certificate on the route, as the destination pod handles TLS termination.

Prerequisites

  • You must have a service that you want to expose.

Procedure

  • Create a Route resource: 

    $ oc create route passthrough route-passthrough-secured --service=frontend --port=8080
    Copy to Clipboard Toggle word wrap

    If you examine the resulting Route resource, it should look similar to the following:

    A Secured Route Using Passthrough Termination

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: route-passthrough-secured
spec:
  host: www.example.com
  port:
    targetPort: 8080
  tls:
    termination: passthrough
    insecureEdgeTerminationPolicy: None
  to:
    kind: Service
    name: frontend
    Copy to Clipboard Toggle word wrap

    where:

    metadata.name
    Specifies the name of the object, which is limited to 63 characters.
    tls.termination
    Specifies the termination field is set to passthrough. This is the only required tls field.
    tls.insecureEdgeTerminationPolicy

    Specifies the type of edge termination policy. Optional parameter. The only valid values are None, Redirect, or empty for disabled.

    The destination pod is responsible for serving certificates for the traffic at the endpoint. This is currently the only method that can support requiring client certificates, also known as two-way authentication.

To define a route with a custom destination CA certificate, apply the route.openshift.io/destination-ca-certificate-secret annotation to an Ingress object. This configuration ensures the Ingress Controller uses the specified secret to verify the identity of the destination service.

Prerequisites

  • You have a certificate/key pair in PEM-encoded files, where the certificate is valid for the route host.
  • You have a separate CA certificate in a PEM-encoded file that completes the certificate chain.
  • You have a separate destination CA certificate in a PEM-encoded file.
  • You have a service that you want to expose.

Procedure

  1. Create a secret for the destination CA certificate by entering the following command: 

    $ oc create secret generic dest-ca-cert --from-file=tls.crt=<file_path>
    Copy to Clipboard Toggle word wrap

    For example: 

    $ oc -n test-ns create secret generic dest-ca-cert --from-file=tls.crt=tls.crt
    Copy to Clipboard Toggle word wrap

    Example output

    secret/dest-ca-cert created
    Copy to Clipboard Toggle word wrap

  2. Add the route.openshift.io/destination-ca-certificate-secret to the Ingress annotations: 

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frontend
  annotations:
    route.openshift.io/termination: "reencrypt"
    route.openshift.io/destination-ca-certificate-secret: secret-ca-cert
...
    Copy to Clipboard Toggle word wrap

    where:

    destination-ca-certificate-secret

    Specifies the route.openshift.io/destination-ca-certificate-secret annotation. The annotation references a Kubernetes secret.

    The Ingress Controller inserts a secret that is referenced in the annotation into the generated route.

    Example output

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: frontend
  annotations:
    route.openshift.io/termination: reencrypt
    route.openshift.io/destination-ca-certificate-secret: secret-ca-cert
spec:
...
  tls:
    insecureEdgeTerminationPolicy: Redirect
    termination: reencrypt
    destinationCACertificate: |
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
...
    Copy to Clipboard Toggle word wrap

You can configure OpenShift Dedicated routes with third-party certificate management solutions by using the .spec.tls.externalCertificate field of the route API. You can reference externally managed TLS certificates via secrets, eliminating the need for manual certificate management.

By using the externally managed certificate, you can reduce errors to ensure a smoother rollout of certificate updates and enable the OpenShift router to serve renewed certificates promptly. You can use externally managed certificates with both edge routes and re-encrypt routes.

Prerequisites

  • You must have a secret containing a valid certificate or key pair in PEM-encoded format of type kubernetes.io/tls, which includes both tls.key and tls.crt keys. Example command: $ oc create secret tls myapp-tls --cert=server.crt --key=server.key.

Procedure

  1. Create a role object in the same namespace as the secret to allow the router service account read access by running the following command: 

    $ oc create role secret-reader --verb=get,list,watch --resource=secrets --resource-name=<secret-name> \
--namespace=<current-namespace>
    Copy to Clipboard Toggle word wrap
    • <secret-name>: Specify the actual name of your secret.
    • <current-namespace>: Specify the namespace where both your secret and route reside.

  2. Create a rolebinding object in the same namespace as the secret and bind the router service account to the newly created role by running the following command: 

    $ oc create rolebinding secret-reader-binding --role=secret-reader --serviceaccount=openshift-ingress:router --namespace=<current-namespace>
    Copy to Clipboard Toggle word wrap
    • <current-namespace>: Specify the namespace where both your secret and route reside.

  3. Create a YAML file that defines the route and specifies the secret containing your certificate using the following example.

    YAML definition of the secure route

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: myedge
  namespace: test
spec:
  host: myedge-test.apps.example.com
  tls:
    externalCertificate:
      name: <secret-name>
    termination: edge
    [...]
[...]
    Copy to Clipboard Toggle word wrap

    • <secret-name>: Specify the actual name of your secret.

  4. Create a route resource by running the following command: 

    $ oc apply -f <route.yaml>
    Copy to Clipboard Toggle word wrap

    • <route.yaml>: Specify the generated YAML filename.

      If the secret exists and has a certificate/key pair, the router will serve the generated certificate if all prerequisites are met.

      Note

      If .spec.tls.externalCertificate is not provided, the router uses default generated certificates.

      You cannot provide the .spec.tls.certificate field or the .spec.tls.key field when using the .spec.tls.externalCertificate field.

To generate a secure, edge-terminated route that uses the default ingress certificate, specify an empty TLS configuration in the Ingress object. This configuration overrides the default behavior, preventing the creation of an insecure route.

Prerequisites

  • You have a service that you want to expose.
  • You have access to the OpenShift CLI (oc).

Procedure

  1. Create a YAML file for the Ingress object. In the following example, the file is called example-ingress.yaml:

    YAML definition of an Ingress object

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frontend
  ...
spec:
  rules:
    ...
  tls:
  - {}
    Copy to Clipboard Toggle word wrap

    where:

    spec.tls
    Specifies the TLS configuration. Use the exact syntax shown to specify TLS without specifying a custom certificate.

  2. Create the Ingress object by running the following command: 

    $ oc create -f example-ingress.yaml
    Copy to Clipboard Toggle word wrap

Verification

  • Verify that OpenShift Dedicated has created the expected route for the Ingress object by running the following command: 

    $ oc get routes -o yaml
    Copy to Clipboard Toggle word wrap

    Example output

    apiVersion: v1
items:
- apiVersion: route.openshift.io/v1
  kind: Route
  metadata:
    name: frontend-j9sdd
# ...
  spec:
  ...
    tls:
      insecureEdgeTerminationPolicy: Redirect
      termination: edge
# ...
    Copy to Clipboard Toggle word wrap

    where:

    metadata.name
    Specifies the name of the route, which includes the name of the Ingress object followed by a random suffix.
    spec.tls
    To use the default certificate, the route should not specify spec.certificate.
    tls.termination
    Specifies the termination policy for the route. The route should specify the edge termination policy.

Legal Notice
Copy link

Copyright © 2025 Red Hat

OpenShift documentation is licensed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).

Modified versions must remove all Red Hat trademarks.

Portions adapted from https://github.com/kubernetes-incubator/service-catalog/ with modifications by Red Hat.

Red Hat, Red Hat Enterprise Linux, the Red Hat logo, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.

Linux® is the registered trademark of Linus Torvalds in the United States and other countries.

Java® is a registered trademark of Oracle and/or its affiliates.

XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.

MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.

Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.

The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.

All other trademarks are the property of their respective owners.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2026 Red HatBack to top