Chapter 9. External and Ingress routing


9.1. Routing overview

Knative leverages OpenShift Container Platform TLS termination to provide routing for Knative services. When a Knative service is created, an OpenShift Container Platform route is automatically created for the service. This route is managed by the OpenShift Serverless Operator. The OpenShift Container Platform route exposes the Knative service through the same domain as the OpenShift Container Platform cluster.

You can disable Operator control of OpenShift Container Platform routing so that you can configure a Knative route to directly use your TLS certificates instead.

Knative routes can also be used alongside the OpenShift Container Platform route to provide additional fine-grained routing capabilities, such as traffic splitting.

9.1.1. Additional resources for OpenShift Container Platform

9.2. Customizing labels and annotations

OpenShift Container Platform routes support the use of custom labels and annotations, which you can configure by modifying the metadata spec of a Knative service. Custom labels and annotations are propagated from the service to the Knative route, then to the Knative ingress, and finally to the OpenShift Container Platform route.

9.2.1. Customizing labels and annotations for OpenShift Container Platform routes

Prerequisites

  • You must have the OpenShift Serverless Operator and Knative Serving installed on your OpenShift Container Platform cluster.
  • Install the OpenShift CLI (oc).

Procedure

  1. Create a Knative service that contains the label or annotation that you want to propagate to the OpenShift Container Platform route:

    • To create a service by using YAML:

      Example service created by using YAML

      apiVersion: serving.knative.dev/v1
      kind: Service
      metadata:
        name: <service_name>
        labels:
          <label_name>: <label_value>
        annotations:
          <annotation_name>: <annotation_value>
      ...

    • To create a service by using the Knative (kn) CLI, enter:

      Example service created by using a kn command

      $ kn service create <service_name> \
        --image=<image> \
        --annotation <annotation_name>=<annotation_value> \
        --label <label_value>=<label_value>

  2. Verify that the OpenShift Container Platform route has been created with the annotation or label that you added by inspecting the output from the following command:

    Example command for verification

    $ oc get routes.route.openshift.io \
         -l serving.knative.openshift.io/ingressName=<service_name> \ 1
         -l serving.knative.openshift.io/ingressNamespace=<service_namespace> \ 2
         -n knative-serving-ingress -o yaml \
             | grep -e "<label_name>: \"<label_value>\""  -e "<annotation_name>: <annotation_value>" 3

    1
    Use the name of your service.
    2
    Use the namespace where your service was created.
    3
    Use your values for the label and annotation names and values.

9.3. Configuring routes for Knative services

If you want to configure a Knative service to use your TLS certificate on OpenShift Container Platform, you must disable the automatic creation of a route for the service by the OpenShift Serverless Operator and instead manually create a route for the service.

Note

When you complete the following procedure, the default OpenShift Container Platform route in the knative-serving-ingress namespace is not created. However, the Knative route for the application is still created in this namespace.

9.3.1. Configuring OpenShift Container Platform routes for Knative services

Prerequisites

  • The OpenShift Serverless Operator and Knative Serving component must be installed on your OpenShift Container Platform cluster.
  • Install the OpenShift CLI (oc).

Procedure

  1. Create a Knative service that includes the serving.knative.openshift.io/disableRoute=true annotation:

    Important

    The serving.knative.openshift.io/disableRoute=true annotation instructs OpenShift Serverless to not automatically create a route for you. However, the service still shows a URL and reaches a status of Ready. This URL does not work externally until you create your own route with the same hostname as the hostname in the URL.

    1. Create a Knative Service resource:

      Example resource

      apiVersion: serving.knative.dev/v1
      kind: Service
      metadata:
        name: <service_name>
        annotations:
          serving.knative.openshift.io/disableRoute: "true"
      spec:
        template:
          spec:
            containers:
            - image: <image>
      ...

    2. Apply the Service resource:

      $ oc apply -f <filename>
    3. Optional. Create a Knative service by using the kn service create command:

      Example kn command

      $ kn service create <service_name> \
        --image=gcr.io/knative-samples/helloworld-go \
        --annotation serving.knative.openshift.io/disableRoute=true

  2. Verify that no OpenShift Container Platform route has been created for the service:

    Example command

    $ $ oc get routes.route.openshift.io \
      -l serving.knative.openshift.io/ingressName=$KSERVICE_NAME \
      -l serving.knative.openshift.io/ingressNamespace=$KSERVICE_NAMESPACE \
      -n knative-serving-ingress

    You will see the following output:

    No resources found in knative-serving-ingress namespace.
  3. Create a Route resource in the knative-serving-ingress namespace:

    apiVersion: route.openshift.io/v1
    kind: Route
    metadata:
      annotations:
        haproxy.router.openshift.io/timeout: 600s 1
      name: <route_name> 2
      namespace: knative-serving-ingress 3
    spec:
      host: <service_host> 4
      port:
        targetPort: http2
      to:
        kind: Service
        name: kourier
        weight: 100
      tls:
        insecureEdgeTerminationPolicy: Allow
        termination: edge 5
        key: |-
          -----BEGIN PRIVATE KEY-----
          [...]
          -----END PRIVATE KEY-----
        certificate: |-
          -----BEGIN CERTIFICATE-----
          [...]
          -----END CERTIFICATE-----
        caCertificate: |-
          -----BEGIN CERTIFICATE-----
          [...]
          -----END CERTIFICATE----
      wildcardPolicy: None
    1
    The timeout value for the OpenShift Container Platform route. You must set the same value as the max-revision-timeout-seconds setting (600s by default).
    2
    The name of the OpenShift Container Platform route.
    3
    The namespace for the OpenShift Container Platform route. This must be knative-serving-ingress.
    4
    The hostname for external access. You can set this to <service_name>-<service_namespace>.<domain>.
    5
    The certificates you want to use. Currently, only edge termination is supported.
  4. Apply the Route resource:

    $ oc apply -f <filename>

9.4. URL scheme for external routes

The URL scheme of external routes defaults to HTTPS for enhanced security. This scheme is determined by the default-external-scheme key in the KnativeServing custom resource (CR) spec.

9.4.1. Setting the URL scheme for external routes

Default spec

...
spec:
  config:
    network:
      default-external-scheme: "https"
...

You can override the default spec to use HTTP by modifying the default-external-scheme key:

HTTP override spec

...
spec:
  config:
    network:
      default-external-scheme: "http"
...

9.5. Cluster local availability

By default, Knative services are published to a public IP address. Being published to a public IP address means that Knative services are public applications, and have a publicly accessible URL.

Publicly accessible URLs are accessible from outside of the cluster. However, developers may need to build back-end services that are only be accessible from inside the cluster, known as private services. Developers can label individual services in the cluster with the networking.knative.dev/visibility=cluster-local label to make them private.

Important

For OpenShift Serverless 1.15.0 and newer versions, the serving.knative.dev/visibility label is no longer available. You must update existing services to use the networking.knative.dev/visibility label instead.

9.5.1. Setting cluster availability to cluster local

Prerequisites

  • The OpenShift Serverless Operator and Knative Serving are installed on the cluster.
  • You have created a Knative service.

Procedure

  • Set the visibility for your service by adding the networking.knative.dev/visibility=cluster-local label:

    $ oc label ksvc <service_name> networking.knative.dev/visibility=cluster-local

Verification

  • Check that the URL for your service is now in the format http://<service_name>.<namespace>.svc.cluster.local, by entering the following command and reviewing the output:

    $ oc get ksvc

    Example output

    NAME            URL                                                                         LATESTCREATED     LATESTREADY       READY   REASON
    hello           http://hello.default.svc.cluster.local                                      hello-tx2g7       hello-tx2g7       True

9.5.2. Enabling TLS authentication for cluster local services

For cluster local services, the Kourier local gateway kourier-internal is used. If you want to use TLS traffic against the Kourier local gateway, you must configure your own server certificates in the local gateway.

Prerequisites

  • You have installed the OpenShift Serverless Operator and Knative Serving.
  • You have administrator permissions.
  • You have installed the OpenShift (oc) CLI.

Procedure

  1. Deploy server certificates in the knative-serving-ingress namespace:

    $ export san="knative"
    Note

    Subject Alternative Name (SAN) validation is required so that these certificates can serve the request to <app_name>.<namespace>.svc.cluster.local.

  2. Generate a root key and certificate:

    $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
        -subj '/O=Example/CN=Example' \
        -keyout ca.key \
        -out ca.crt
  3. Generate a server key that uses SAN validation:

    $ openssl req -out tls.csr -newkey rsa:2048 -nodes -keyout tls.key \
      -subj "/CN=Example/O=Example" \
      -addext "subjectAltName = DNS:$san"
  4. Create server certificates:

    $ openssl x509 -req -extfile <(printf "subjectAltName=DNS:$san") \
      -days 365 -in tls.csr \
      -CA ca.crt -CAkey ca.key -CAcreateserial -out tls.crt
  5. Configure a secret for the Kourier local gateway:

    1. Deploy a secret in knative-serving-ingress namespace from the certificates created by the previous steps:

      $ oc create -n knative-serving-ingress secret tls server-certs \
          --key=tls.key \
          --cert=tls.crt --dry-run=client -o yaml | oc apply -f -
    2. Update the KnativeServing custom resource (CR) spec to use the secret that was created by the Kourier gateway:

      Example KnativeServing CR

      ...
      spec:
        config:
          kourier:
            cluster-cert-secret: server-certs
      ...

The Kourier controller sets the certificate without restarting the service, so that you do not need to restart the pod.

You can access the Kourier internal service with TLS through port 443 by mounting and using the ca.crt from the client.

9.6. Kourier Gateway service type

The Kourier Gateway is exposed by default as the ClusterIP service type. This service type is determined by the service-type ingress spec in the KnativeServing custom resource (CR).

Default spec

...
spec:
  ingress:
    kourier:
      service-type: ClusterIP
...

9.6.1. Setting the Kourier Gateway service type

You can override the default service type to use a load balancer service type instead by modifying the service-type spec:

LoadBalancer override spec

...
spec:
  ingress:
    kourier:
      service-type: LoadBalancer
...

9.7. Using HTTP2 and gRPC

OpenShift Serverless supports only insecure or edge-terminated routes. Insecure or edge-terminated routes do not support HTTP2 on OpenShift Container Platform. These routes also do not support gRPC because gRPC is transported by HTTP2. If you use these protocols in your application, you must call the application using the ingress gateway directly. To do this you must find the ingress gateway’s public address and the application’s specific host.

9.7.1. Interacting with a serverless application using HTTP2 and gRPC

Important

This method applies to OpenShift Container Platform 4.10 and later. For older versions, see the following section.

Prerequisites

  • Install OpenShift Serverless Operator and Knative Serving on your cluster.
  • Install the OpenShift CLI (oc).
  • Create a Knative service.
  • Upgrade OpenShift Container Platform 4.10 or later.
  • Enable HTTP/2 on OpenShift Ingress controller.

Procedure

  1. Add the serverless.openshift.io/default-enable-http2=true annotation to the KnativeServing Custom Resource:

    $ oc annotate knativeserving <your_knative_CR> -n knative-serving serverless.openshift.io/default-enable-http2=true
  2. After the annotation is added, you can verify that the appProtocol value of the Kourier service is h2c:

    $ oc get svc -n knative-serving-ingress kourier -o jsonpath="{.spec.ports[0].appProtocol}"

    Example output

    h2c

  3. Now you can use the gRPC framework over the HTTP/2 protocol for external traffic, for example:

    import "google.golang.org/grpc"
    
    grpc.Dial(
       YOUR_URL, 1
       grpc.WithTransportCredentials(insecure.NewCredentials())), 2
    )
    1
    Your ksvc URL.
    2
    Your certificate.

9.8. Using Serving with OpenShift ingress sharding

You can use Knative Serving with OpenShift ingress sharding to split ingress traffic based on domains. This allows you to manage and route network traffic to different parts of a cluster more efficiently.

Note

Even with OpenShift ingress sharding in place, OpenShift Serverless traffic is still routed through a single Knative Ingress Gateway and the activator component in the knative-serving project.

For more information about isolating the network traffic, see Using Service Mesh to isolate network traffic with OpenShift Serverless.

Prerequisites

  • You have installed the OpenShift Serverless Operator and Knative Serving.
  • You have cluster administrator permissions on OpenShift Container Platform, or you have cluster or dedicated administrator permissions on Red Hat OpenShift Service on AWS or OpenShift Dedicated.

9.8.1. Configuring OpenShift ingress shards

Before configuring Knative Serving, you must configure OpenShift ingress shards.

Procedure

  • Use a label selector in the IngressController CR to configure OpenShift Serverless to match specific ingress shards with different domains:

    Example IngressController CR

    apiVersion: operator.openshift.io/v1
    kind: IngressController
    metadata:
      name: ingress-dev 1
      namespace: openshift-ingress-operator
    spec:
      routeSelector:
        matchLabels:
          router: dev 2
      domain: "dev.serverless.cluster.example.com" 3
      # ...
    ---
    apiVersion: operator.openshift.io/v1
    kind: IngressController
    metadata:
      name: ingress-prod 4
      namespace: openshift-ingress-operator
    spec:
      routeSelector:
        matchLabels:
          router: prod 5
      domain: "prod.serverless.cluster.example.com" 6
      # ...

    1
    Name of the first ingress shard.
    2
    A label selector to match the ingress-dev shard.
    3
    A custom domain for the ingress-dev shard.
    4
    Name of the second ingress shard.
    5
    A label selector to match the ingress-prod shard.
    6
    A custom domain for the ingress-prod shard.

9.8.2. Configuring custom domains in the KnativeServing CR

After configuring OpenShift ingress shards, you must configure Knative Serving to match them.

Procedure

  • In the KnativeServing CR, configure Serving to use the same domains and labels as your ingress shards by adding the spec.config.domain field:

    Example KnativeServing CR

    spec:
      config:
        domain: 1
          dev.serverless.cluster.example.com: |
            selector:
              router: dev
          prod.serverless.cluster.example.com: |
            selector:
              router: prod
      # ...

    1
    These values need to match the values in the ingress shard configuration.

9.8.3. Targeting a specific ingress shard in the Knative Service

After configuring ingress sharding and Knative Serving, you can target a specific ingress shard in your Knative Service resources using a label.

Procedure

  • In your Service CR, add the label selector that matches a specific shard:

    Example Service CR

    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: hello-dev
      labels:
        router: dev 1
    spec:
      template:
        spec:
          containers:
          - image: docker.io/openshift/hello-openshift
    ---
    apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
      name: hello-prod
      labels:
        router: prod 2
    spec:
      template:
        spec:
          containers:
          - image: docker.io/openshift/hello-openshift
      # ...

    1 2
    The labels must match the configuration in the KnativeServing CR.

9.8.4. Verifying Serving with OpenShift ingress sharding configuration

After configuring ingress sharding, Knative Serving, and your service, you can verify that your service uses the correct route and the selected ingress shard.

Procedure

  1. Print information about the services in the cluster by running the following command:

    $ oc get ksvc

    Example output

    NAME         URL                                                             LATESTCREATED      LATESTREADY        READY   REASON
    hello-dev    https://hello-dev-default.dev.serverless.cluster.example.com    hello-dev-00001    hello-dev-00001    True
    hello-prod   https://hello-prod-default.prod.serverless.cluster.example.com  hello-prod-00001   hello-prod-00001   True

  2. Verify that your service uses the correct route and the selected ingress shard by running the following command:

    $ oc get route -n knative-serving-ingress -o jsonpath='{range .items[*]}{@.metadata.name}{" "}{@.spec.host}{" "}{@.status.ingress[*].routerName}{"\n"}{end}'

    Example output

    route-19e6628b-77af-4da0-9b4c-1224934b2250-323461616533 hello-prod-default.prod.serverless.cluster.example.com ingress-prod
    route-cb5085d9-b7da-4741-9a56-96c88c6adaaa-373065343266 hello-dev-default.dev.serverless.cluster.example.com ingress-dev

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.