Chapter 9. External and Ingress routing
9.1. Routing overview Copy linkLink copied to clipboard!
Knative leverages OpenShift Container Platform TLS termination to provide routing for Knative services. When a Knative service is created, an OpenShift Container Platform route is automatically created for the service. This route is managed by the OpenShift Serverless Operator. The OpenShift Container Platform route exposes the Knative service through the same domain as the OpenShift Container Platform cluster.
You can disable Operator control of OpenShift Container Platform routing so that you can configure a Knative route to directly use your TLS certificates instead.
Knative routes can also be used alongside the OpenShift Container Platform route to provide additional fine-grained routing capabilities, such as traffic splitting.
9.2. Customizing labels and annotations Copy linkLink copied to clipboard!
OpenShift Container Platform routes support the use of custom labels and annotations, which you can configure by modifying the metadata
spec of a Knative service. Custom labels and annotations are propagated from the service to the Knative route, then to the Knative ingress, and finally to the OpenShift Container Platform route.
9.2.1. Customizing labels and annotations for OpenShift Container Platform routes Copy linkLink copied to clipboard!
Prerequisites
- You must have the OpenShift Serverless Operator and Knative Serving installed on your OpenShift Container Platform cluster.
-
Install the OpenShift CLI (
oc
).
Procedure
Create a Knative service that contains the label or annotation that you want to propagate to the OpenShift Container Platform route:
To create a service by using YAML:
Example service created by using YAML
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To create a service by using the Knative (
kn
) CLI, enter:Example service created by using a
kn
commandkn service create <service_name> \ --image=<image> \ --annotation <annotation_name>=<annotation_value> \ --label <label_value>=<label_value>
$ kn service create <service_name> \ --image=<image> \ --annotation <annotation_name>=<annotation_value> \ --label <label_value>=<label_value>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verify that the OpenShift Container Platform route has been created with the annotation or label that you added by inspecting the output from the following command:
Example command for verification
oc get routes.route.openshift.io \ -l serving.knative.openshift.io/ingressName=<service_name> \ -l serving.knative.openshift.io/ingressNamespace=<service_namespace> \ -n knative-serving-ingress -o yaml \ | grep -e "<label_name>: \"<label_value>\"" -e "<annotation_name>: <annotation_value>"
$ oc get routes.route.openshift.io \ -l serving.knative.openshift.io/ingressName=<service_name> \
1 -l serving.knative.openshift.io/ingressNamespace=<service_namespace> \
2 -n knative-serving-ingress -o yaml \ | grep -e "<label_name>: \"<label_value>\"" -e "<annotation_name>: <annotation_value>"
3 Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.3. Configuring routes for Knative services Copy linkLink copied to clipboard!
If you want to configure a Knative service to use your TLS certificate on OpenShift Container Platform, you must disable the automatic creation of a route for the service by the OpenShift Serverless Operator and instead manually create a route for the service.
When you complete the following procedure, the default OpenShift Container Platform route in the knative-serving-ingress
namespace is not created. However, the Knative route for the application is still created in this namespace.
9.3.1. Configuring OpenShift Container Platform routes for Knative services Copy linkLink copied to clipboard!
Prerequisites
- The OpenShift Serverless Operator and Knative Serving component must be installed on your OpenShift Container Platform cluster.
-
Install the OpenShift CLI (
oc
).
Procedure
Create a Knative service that includes the
serving.knative.openshift.io/disableRoute=true
annotation:ImportantThe
serving.knative.openshift.io/disableRoute=true
annotation instructs OpenShift Serverless to not automatically create a route for you. However, the service still shows a URL and reaches a status ofReady
. This URL does not work externally until you create your own route with the same hostname as the hostname in the URL.Create a Knative
Service
resource:Example resource
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Apply the
Service
resource:oc apply -f <filename>
$ oc apply -f <filename>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional. Create a Knative service by using the
kn service create
command:Example
kn
commandkn service create <service_name> \ --image=gcr.io/knative-samples/helloworld-go \ --annotation serving.knative.openshift.io/disableRoute=true
$ kn service create <service_name> \ --image=gcr.io/knative-samples/helloworld-go \ --annotation serving.knative.openshift.io/disableRoute=true
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verify that no OpenShift Container Platform route has been created for the service:
Example command
$ oc get routes.route.openshift.io \ -l serving.knative.openshift.io/ingressName=$KSERVICE_NAME \ -l serving.knative.openshift.io/ingressNamespace=$KSERVICE_NAMESPACE \ -n knative-serving-ingress
$ $ oc get routes.route.openshift.io \ -l serving.knative.openshift.io/ingressName=$KSERVICE_NAME \ -l serving.knative.openshift.io/ingressNamespace=$KSERVICE_NAMESPACE \ -n knative-serving-ingress
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You will see the following output:
No resources found in knative-serving-ingress namespace.
No resources found in knative-serving-ingress namespace.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a
Route
resource in theknative-serving-ingress
namespace:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- The timeout value for the OpenShift Container Platform route. You must set the same value as the
max-revision-timeout-seconds
setting (600s
by default). - 2
- The name of the OpenShift Container Platform route.
- 3
- The namespace for the OpenShift Container Platform route. This must be
knative-serving-ingress
. - 4
- The hostname for external access. You can set this to
<service_name>-<service_namespace>.<domain>
. - 5
- The certificates you want to use. Currently, only
edge
termination is supported.
Apply the
Route
resource:oc apply -f <filename>
$ oc apply -f <filename>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.4. URL scheme for external routes Copy linkLink copied to clipboard!
The URL scheme of external routes defaults to HTTPS for enhanced security. This scheme is determined by the default-external-scheme
key in the KnativeServing
custom resource (CR) spec.
9.4.1. Setting the URL scheme for external routes Copy linkLink copied to clipboard!
Default spec
You can override the default spec to use HTTP by modifying the default-external-scheme
key:
HTTP override spec
9.5. Cluster local availability Copy linkLink copied to clipboard!
By default, Knative services are published to a public IP address. Being published to a public IP address means that Knative services are public applications, and have a publicly accessible URL.
Publicly accessible URLs are accessible from outside of the cluster. However, developers may need to build back-end services that are only be accessible from inside the cluster, known as private services. Developers can label individual services in the cluster with the networking.knative.dev/visibility=cluster-local
label to make them private.
For OpenShift Serverless 1.15.0 and newer versions, the serving.knative.dev/visibility
label is no longer available. You must update existing services to use the networking.knative.dev/visibility
label instead.
9.5.1. Setting cluster availability to cluster local Copy linkLink copied to clipboard!
Prerequisites
- The OpenShift Serverless Operator and Knative Serving are installed on the cluster.
- You have created a Knative service.
Procedure
Set the visibility for your service by adding the
networking.knative.dev/visibility=cluster-local
label:oc label ksvc <service_name> networking.knative.dev/visibility=cluster-local
$ oc label ksvc <service_name> networking.knative.dev/visibility=cluster-local
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Check that the URL for your service is now in the format
http://<service_name>.<namespace>.svc.cluster.local
, by entering the following command and reviewing the output:oc get ksvc
$ oc get ksvc
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME URL LATESTCREATED LATESTREADY READY REASON hello http://hello.default.svc.cluster.local hello-tx2g7 hello-tx2g7 True
NAME URL LATESTCREATED LATESTREADY READY REASON hello http://hello.default.svc.cluster.local hello-tx2g7 hello-tx2g7 True
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.5.2. Enabling TLS authentication for cluster local services Copy linkLink copied to clipboard!
For cluster local services, the Kourier local gateway kourier-internal
is used. If you want to use TLS traffic against the Kourier local gateway, you must configure your own server certificates in the local gateway.
Prerequisites
- You have installed the OpenShift Serverless Operator and Knative Serving.
- You have administrator permissions.
-
You have installed the OpenShift (
oc
) CLI.
Procedure
Deploy server certificates in the
knative-serving-ingress
namespace:export san="knative"
$ export san="knative"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteSubject Alternative Name (SAN) validation is required so that these certificates can serve the request to
<app_name>.<namespace>.svc.cluster.local
.Generate a root key and certificate:
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \ -subj '/O=Example/CN=Example' \ -keyout ca.key \ -out ca.crt
$ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \ -subj '/O=Example/CN=Example' \ -keyout ca.key \ -out ca.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Generate a server key that uses SAN validation:
openssl req -out tls.csr -newkey rsa:2048 -nodes -keyout tls.key \ -subj "/CN=Example/O=Example" \ -addext "subjectAltName = DNS:$san"
$ openssl req -out tls.csr -newkey rsa:2048 -nodes -keyout tls.key \ -subj "/CN=Example/O=Example" \ -addext "subjectAltName = DNS:$san"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create server certificates:
openssl x509 -req -extfile <(printf "subjectAltName=DNS:$san") \ -days 365 -in tls.csr \ -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt
$ openssl x509 -req -extfile <(printf "subjectAltName=DNS:$san") \ -days 365 -in tls.csr \ -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure a secret for the Kourier local gateway:
Deploy a secret in
knative-serving-ingress
namespace from the certificates created by the previous steps:oc create -n knative-serving-ingress secret tls server-certs \ --key=tls.key \ --cert=tls.crt --dry-run=client -o yaml | oc apply -f -
$ oc create -n knative-serving-ingress secret tls server-certs \ --key=tls.key \ --cert=tls.crt --dry-run=client -o yaml | oc apply -f -
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Update the
KnativeServing
custom resource (CR) spec to use the secret that was created by the Kourier gateway:Example KnativeServing CR
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
The Kourier controller sets the certificate without restarting the service, so that you do not need to restart the pod.
You can access the Kourier internal service with TLS through port 443
by mounting and using the ca.crt
from the client.
9.6. Kourier Gateway service type Copy linkLink copied to clipboard!
The Kourier Gateway is exposed by default as the ClusterIP
service type. This service type is determined by the service-type
ingress spec in the KnativeServing
custom resource (CR).
Default spec
9.6.1. Setting the Kourier Gateway service type Copy linkLink copied to clipboard!
You can override the default service type to use a load balancer service type instead by modifying the service-type
spec:
LoadBalancer override spec
9.7. Using HTTP2 and gRPC Copy linkLink copied to clipboard!
OpenShift Serverless supports only insecure or edge-terminated routes. Insecure or edge-terminated routes do not support HTTP2 on OpenShift Container Platform. These routes also do not support gRPC because gRPC is transported by HTTP2. If you use these protocols in your application, you must call the application using the ingress gateway directly. To do this you must find the ingress gateway’s public address and the application’s specific host.
9.7.1. Interacting with a serverless application using HTTP2 and gRPC Copy linkLink copied to clipboard!
This method applies to OpenShift Container Platform 4.10 and later. For older versions, see the following section.
Prerequisites
- Install OpenShift Serverless Operator and Knative Serving on your cluster.
-
Install the OpenShift CLI (
oc
). - Create a Knative service.
- Upgrade OpenShift Container Platform 4.10 or later.
- Enable HTTP/2 on OpenShift Ingress controller.
Procedure
Add the
serverless.openshift.io/default-enable-http2=true
annotation to theKnativeServing
Custom Resource:oc annotate knativeserving <your_knative_CR> -n knative-serving serverless.openshift.io/default-enable-http2=true
$ oc annotate knativeserving <your_knative_CR> -n knative-serving serverless.openshift.io/default-enable-http2=true
Copy to Clipboard Copied! Toggle word wrap Toggle overflow After the annotation is added, you can verify that the
appProtocol
value of the Kourier service ish2c
:oc get svc -n knative-serving-ingress kourier -o jsonpath="{.spec.ports[0].appProtocol}"
$ oc get svc -n knative-serving-ingress kourier -o jsonpath="{.spec.ports[0].appProtocol}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
h2c
h2c
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Now you can use the gRPC framework over the HTTP/2 protocol for external traffic, for example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
9.8. Using Serving with OpenShift ingress sharding Copy linkLink copied to clipboard!
You can use Knative Serving with OpenShift ingress sharding to split ingress traffic based on domains. This allows you to manage and route network traffic to different parts of a cluster more efficiently.
Even with OpenShift ingress sharding in place, OpenShift Serverless traffic is still routed through a single Knative Ingress Gateway and the activator component in the knative-serving
project.
For more information about isolating the network traffic, see Using Service Mesh to isolate network traffic with OpenShift Serverless.
Prerequisites
- You have installed the OpenShift Serverless Operator and Knative Serving.
- You have cluster administrator permissions on OpenShift Container Platform, or you have cluster or dedicated administrator permissions on Red Hat OpenShift Service on AWS or OpenShift Dedicated.
9.8.1. Configuring OpenShift ingress shards Copy linkLink copied to clipboard!
Before configuring Knative Serving, you must configure OpenShift ingress shards.
Procedure
Use a label selector in the
IngressController
CR to configure OpenShift Serverless to match specific ingress shards with different domains:Example
IngressController
CRCopy to Clipboard Copied! Toggle word wrap Toggle overflow
9.8.2. Configuring custom domains in the KnativeServing CR Copy linkLink copied to clipboard!
After configuring OpenShift ingress shards, you must configure Knative Serving to match them.
Procedure
In the
KnativeServing
CR, configure Serving to use the same domains and labels as your ingress shards by adding thespec.config.domain
field:Example
KnativeServing
CRCopy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- These values need to match the values in the ingress shard configuration.
9.8.3. Targeting a specific ingress shard in the Knative Service Copy linkLink copied to clipboard!
After configuring ingress sharding and Knative Serving, you can target a specific ingress shard in your Knative Service resources using a label.
9.8.4. Verifying Serving with OpenShift ingress sharding configuration Copy linkLink copied to clipboard!
After configuring ingress sharding, Knative Serving, and your service, you can verify that your service uses the correct route and the selected ingress shard.
Procedure
Print information about the services in the cluster by running the following command:
oc get ksvc
$ oc get ksvc
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
NAME URL LATESTCREATED LATESTREADY READY REASON hello-dev https://hello-dev-default.dev.serverless.cluster.example.com hello-dev-00001 hello-dev-00001 True hello-prod https://hello-prod-default.prod.serverless.cluster.example.com hello-prod-00001 hello-prod-00001 True
NAME URL LATESTCREATED LATESTREADY READY REASON hello-dev https://hello-dev-default.dev.serverless.cluster.example.com hello-dev-00001 hello-dev-00001 True hello-prod https://hello-prod-default.prod.serverless.cluster.example.com hello-prod-00001 hello-prod-00001 True
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Verify that your service uses the correct route and the selected ingress shard by running the following command:
oc get route -n knative-serving-ingress -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.host}{" "}{@.status.ingress[*].routerName}{"\n"}{end}'
$ oc get route -n knative-serving-ingress -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.host}{" "}{@.status.ingress[*].routerName}{"\n"}{end}'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
route-19e6628b-77af-4da0-9b4c-1224934b2250-323461616533 hello-prod-default.prod.serverless.cluster.example.com ingress-prod route-cb5085d9-b7da-4741-9a56-96c88c6adaaa-373065343266 hello-dev-default.dev.serverless.cluster.example.com ingress-dev
route-19e6628b-77af-4da0-9b4c-1224934b2250-323461616533 hello-prod-default.prod.serverless.cluster.example.com ingress-prod route-cb5085d9-b7da-4741-9a56-96c88c6adaaa-373065343266 hello-dev-default.dev.serverless.cluster.example.com ingress-dev
Copy to Clipboard Copied! Toggle word wrap Toggle overflow