Chapter 1. Configuring and deploying gateway policies
As a platform engineer or application developer, you can secure, protect, and connect an API exposed by a gateway that uses Gateway API by using Connectivity Link.
1.1. Secure, protect, and connect APIs on OpenShift Container Platform with Connectivity Link Copy linkLink copied to clipboard!
This guide shows how you can use Connectivity Link on OpenShift Container Platform to secure, protect, and connect an API exposed by a Gateway that uses Kubernetes Gateway API. This guide applies to the platform engineer and application developer user roles in Connectivity Link.
In multi-cluster environments, you must perform the following steps in each cluster individually, unless specifically excluded.
1.1.1. Connectivity Link capabilities in multicluster environments Copy linkLink copied to clipboard!
You can use Connectivity Link capabilities in single or multiple OpenShift Container Platform clusters. The following features are designed to work across multiple clusters and in a single-cluster environment:
-
Multicluster ingress: Connectivity Link provides multicluster ingress connectivity using DNS to bring traffic to your gateways by using a strategy defined in a
DNSPolicy. -
Global rate limiting: Connectivity Link can enable global rate limiting use cases when configured to use a shared Redis-based store for counters based on limits defined by a
RateLimitPolicy. -
Global auth: You can configure a Connectivity Link
AuthPolicyto use external auth providers to ensure that different clusters exposing the same API can authenticate and allow in the same way. -
Automatic TLS certificate generation: You can configure a
TLSPolicyto automatically provision TLS certificates based on Gateway listener hosts by using integration with cert-manager and ACME providers such as Let’s Encrypt. - Integration with federated metrics stores: Connectivity Link has example dashboards and metrics for visualizing your gateways and observing traffic hitting those gateways across multiple clusters.
1.1.2. Connectivity Link user role workflows Copy linkLink copied to clipboard!
Platform engineer: This guide shows how platform engineers can deploy gateways that provide secure communication and are protected and ready for use by application development teams to deploy APIs.
Platform engineers can use Connectivity Link in clusters in different geographic regions to bring specific traffic to geo-located gateways. This approach reduces latency, distributes load, and protects and secures with global rate limiting and auth policies.
- Application developer: This guide shows how application developers can override the Gateway-level global auth and rate limiting policies to configure application-level auth and rate limiting requirements for specific users.
1.2. Set up your environment Copy linkLink copied to clipboard!
This section shows how you can set up your environment variables and deploy the example Toystore application on your OpenShift Container Platform cluster.
Prerequisites
- Connectivity Link is installed on the OpenShift Container Platform cluster you are working with.
-
The OpenShift CLI (
oc) is installed. - You are logged in to the OpenShift Container Platform cluster with write access to the namespaces you need to use.
- You have a DNS zone available for use.
- Optional. For rate limiting in a multicluster environment, you have installed Connectivity Link on more than one cluster. You also have a shared and accessible Redis-based datastore.
- Optional. For observability, OpenShift Container Platform user workload monitoring is configured to remote-write to a central storage system.
Procedure
Optional: Set the following environment variables:
$ export KUADRANT_GATEWAY_NS=api-gateway \ export KUADRANT_GATEWAY_NAME=ingress-gateway \ export KUADRANT_DEVELOPER_NS=toystore \ export KUADRANT_AWS_ACCESS_KEY_ID=xxxx \ export KUADRANT_AWS_SECRET_ACCESS_KEY=xxxx \ export KUADRANT_AWS_DNS_PUBLIC_ZONE_ID=xxxx \ export KUADRANT_ZONE_ROOT_DOMAIN=example.com export KUADRANT_CLUSTER_ISSUER_NAME=self-signedThese environment variables are described as follows:
-
KUADRANT_GATEWAY_NS: Namespace for your gateway in OpenShift Container Platform. -
KUADRANT_GATEWAY_NAME: Name of your ingress gateway in OpenShift Container Platform. -
KUADRANT_DEVELOPER_NS: Namespace for the example Toystore app in OpenShift Container Platform. -
KUADRANT_AWS_ACCESS_KEY_ID: AWS key ID with access to manage your DNS zone. -
KUADRANT_AWS_SECRET_ACCESS_KEY: AWS secret access key with permissions to manage your DNS zone. -
KUADRANT_AWS_DNS_PUBLIC_ZONE_ID: AWS Route 53 zone ID for the Gateway. This is the ID of the hosted zone that is displayed in the AWS Route 53 console. -
KUADRANT_ZONE_ROOT_DOMAIN: Root domain in AWS Route 53 associated with your DNS zone ID. KUADRANT_CLUSTER_ISSUER_NAME: Name of the certificate authority or issuer TLS certificates.NoteIf you know your environment variable values, you can set up the required YAML files to suit your environment.
-
Create the namespace for the Toystore app as follows:
$ oc create ns ${KUADRANT_DEVELOPER_NS}
Deploy the Toystore app to the developer namespace:
$ oc apply -f https://raw.githubusercontent.com/Kuadrant/Kuadrant-operator/main/examples/toystore/toystore.yaml -n ${KUADRANT_DEVELOPER_NS}
1.3. Set up a DNS provider secret Copy linkLink copied to clipboard!
Your DNS provider supplies credentials to access the DNS zones that Connectivity Link can use to set up your DNS configuration. You must ensure that these credentials have access to only the DNS zones that you want Connectivity Link to manage with your DNSPolicy.
You must apply the following Secret resource to each cluster. If you are adding an additional cluster, add it to the new cluster.
Prerequisites
- You installed Connectivity Link on one or more clusters.
- If you plan to use rate-limiting in a multicluster environment, you have a shared Redis-based datastore.
-
You installed the OpenShift CLI (
oc). - You have write access to the OpenShift Container Platform namespaces you need to work with.
- You have access to external or on-premise DNS.
- You created a gateway.
-
You configured your gateway policies and
HTTProutes.
Procedure
Create the namespace that the Gateway will be deployed in as follows:
$ oc create ns ${KUADRANT_GATEWAY_NS}Create the secret credentials in the same namespace as the Gateway as follows:
$ oc -n ${KUADRANT_GATEWAY_NS} create secret generic aws-credentials \ --type=kuadrant.io/aws \ --from-literal=AWS_ACCESS_KEY_ID=$KUADRANT_AWS_ACCESS_KEY_ID \ --from-literal=AWS_SECRET_ACCESS_KEY=$KUADRANT_AWS_SECRET_ACCESS_KEYBefore adding a TLS certificate issuer, create the secret credentials in the
cert-managernamespace as follows:$ oc -n cert-manager create secret generic aws-credentials \ --type=kuadrant.io/aws \ --from-literal=AWS_ACCESS_KEY_ID=$KUADRANT_AWS_ACCESS_KEY_ID \ --from-literal=AWS_SECRET_ACCESS_KEY=$KUADRANT_AWS_SECRET_ACCESS_KEY
1.4. Add a TLS certificate issuer Copy linkLink copied to clipboard!
To secure communication to your Gateways, you must define a certification authority as an issuer for TLS certificates.
This example uses the Let’s Encrypt TLS certificate issuer for simplicity, but you can use any certificate issuer supported by cert-manager. In multicluster environments, you must add your TLS issuer in each OpenShift Container Platform cluster.
Prerequisites
- You installed Connectivity Link on one or more clusters.
- If you plan to use rate-limiting in a multicluster environment, you have a shared Redis-based datastore.
-
You installed the OpenShift CLI (
oc). - You have write access to the OpenShift Container Platform namespaces you need to work with.
- You have access to external or on-premise DNS.
- You created a gateway.
-
You configured your gateway policies and
HTTProutes.
Procedure
Enter the following command to define a TLS certificate issuer:
$ oc apply -f - <<EOF apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: ${KUADRANT_CLUSTER_ISSUER_NAME} spec: selfSigned: {} EOFWait for the
ClusterIssuerto become ready as follows:$ oc wait clusterissuer/${KUADRANT_CLUSTER_ISSUER_NAME} --for=condition=ready=true
1.5. Creating a gateway Copy linkLink copied to clipboard!
As a platform engineer, create a gateway to your OpenShift Container Platform cluster to set up the infrastructure used by application developers. The following example assumes that you are using the OpenShift Container Platform Cluster Ingress Operator (CIO).
When using the Gateway API custom resource definitions (CRDs) provided in OpenShift Container Platform 4.19 or newer, you must create a GatewayClass named openshift-default and specify a controllerName of openshift.io/gateway-controller/v1. For more details, see the Getting started with Gateway API for the Ingress Operator (OpenShift Container Platform documentation).
If you are using OpenShift Service Mesh on OpenShift Container Platform 4.19 and newer and you set the ISTIO_GATEWAY_CONTROLLER_NAMES variable to istio.io/gateway-controller during your Connectivity Link installation, then you can use the GatewayClass custom resource (CR) created by default by OpenShift Service Mesh. Make sure you use the corresponding spec.gatewayClassName value in your Gateway CR.
Prerequisites
- Connectivity Link is installed on the OpenShift Container Platform cluster you are working with.
-
You set the
ISTIO_GATEWAY_CONTROLLER_NAMESenvironment variable value toopenshift.io/gateway-controller/v1during your Connectivity Link installation. -
You created a
GatewayClassnamedopenshift-defaultand specified acontrollerNameofopenshift.io/gateway-controller/v1. -
The OpenShift CLI (
oc) is installed. - You are logged in to the OpenShift Container Platform cluster with write access to the namespaces you need to use.
- You have a DNS zone available for use.
Procedure
Create a gateway that uses the OpenShift Container Platform CIO by running the following command:
$ oc apply -f - <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: ${KUADRANT_GATEWAY_NAME} namespace: ${KUADRANT_GATEWAY_NS} labels: kuadrant.io/gateway: "true" spec: gatewayClassName: openshift-default listeners: - allowedRoutes: namespaces: from: All hostname: "api.${KUADRANT_ZONE_ROOT_DOMAIN}" name: api port: 443 protocol: HTTPS tls: certificateRefs: - group: "" kind: Secret name: api-${KUADRANT_GATEWAY_NAME}-tls mode: Terminate EOFImportantIn a multicluster environment, for Connectivity Link to balance traffic by using DNS across clusters, you must specify a gateway with a shared hostname. You can define this by using an HTTPS listener with a wildcard hostname based on the root domain.
Verification
Check the status of your gateway by running the following command:
$ oc get gateway ${KUADRANT_GATEWAY_NAME} -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Programmed")].message}'The statuses
AcceptedandProgrammedmean that your gateway is valid and assigned an external address.Check the status of your HTTPS listener by running the following command:
$ oc get gateway ${KUADRANT_GATEWAY_NAME} -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.listeners[0].conditions[?(@.type=="Programmed")].message}'
Next steps
- Configure a TLS policy so that the The HTTPS listener can accept traffic.
1.6. Configure your Gateway policies and HTTP route Copy linkLink copied to clipboard!
While your Gateway is now deployed, it has no exposed endpoints and your HTTPS listener is not programmed. Next, you can do take the following steps:
-
Define a
TLSPolicythat leverages yourCertificateIssuerto set up your HTTPS listener certificates. -
Define an
HTTPRoutefor your Gateway to communicate with your backend application API. -
Define an
AuthPolicyto set up a default HTTP403response for any unprotected endpoints -
Define and a
RateLimitPolicyto set up a default artificially low global limit to further protect any endpoints exposed by the Gateway. -
Define a
DNSPolicywith a load balancing strategy for your Gateway.
In multicluster environments, you must perform the following steps in each cluster individually, unless specifically excluded.
Prerequisites
- You installed Connectivity Link on one or more clusters.
- If plan to use rate-limiting in a multicluster environment, you have a shared Redis-based datastore.
-
You installed the OpenShift CLI (
oc). - You have write access to the OpenShift Container Platform namespaces you need to work with.
- You have access to external or on-premise DNS.
- You created a gateway.
Procedure
Set the
TLSPolicyfor your Gateway as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: TLSPolicy metadata: name: ${KUADRANT_GATEWAY_NAME}-tls namespace: ${KUADRANT_GATEWAY_NS} spec: targetRef: name: ${KUADRANT_GATEWAY_NAME} group: gateway.networking.k8s.io kind: Gateway issuerRef: group: cert-manager.io kind: ClusterIssuer name: ${KUADRANT_CLUSTER_ISSUER_NAME} EOFCheck that your TLS policy has an
AcceptedandEnforcedstatus as follows:$ oc get tlspolicy ${KUADRANT_GATEWAY_NAME}-tls -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'This may take a few minutes depending on the TLS provider, for example, Let’s Encrypt.
1.6.1. Create an HTTP route for your application Copy linkLink copied to clipboard!
Procedure
Create an
HTTPRoutefor the example Toystore application as follows:$ oc apply -f - <<EOF apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: toystore namespace: ${KUADRANT_DEVELOPER_NS} labels: deployment: toystore service: toystore spec: parentRefs: - name: ${KUADRANT_GATEWAY_NAME} namespace: ${KUADRANT_GATEWAY_NS} hostnames: - "api.${KUADRANT_ZONE_ROOT_DOMAIN}" rules: - matches: - method: GET path: type: PathPrefix value: "/cars" - method: GET path: type: PathPrefix value: "/health" backendRefs: - name: toystore port: 80 EOF
1.6.2. Set the default AuthPolicy Copy linkLink copied to clipboard!
Procedure
Set a default
AuthPolicywith adeny-allsetting for your Gateway as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: AuthPolicy metadata: name: ${KUADRANT_GATEWAY_NAME}-auth namespace: ${KUADRANT_GATEWAY_NS} spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: ${KUADRANT_GATEWAY_NAME} defaults: when: - predicate: "request.path != '/health'" rules: authorization: deny-all: opa: rego: "allow = false" response: unauthorized: headers: "content-type": value: application/json body: value: | { "error": "Forbidden", "message": "Access denied by default by the gateway operator. If you are the administrator of the service, create a specific auth policy for the route." } EOFCheck that your
AuthPolicyhasAcceptedandEnforcedstatus as follows:$ oc get authpolicy ${KUADRANT_GATEWAY_NAME}-auth -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'
1.6.3. Set the default RateLimitPolicy Copy linkLink copied to clipboard!
Procedure
Set the default
RateLimitPolicywith alow-limitsetting for your Gateway as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: RateLimitPolicy metadata: name: ${KUADRANT_GATEWAY_NAME}-rlp namespace: ${KUADRANT_GATEWAY_NS} spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: ${KUADRANT_GATEWAY_NAME} defaults: limits: "low-limit": rates: - limit: 1 window: 10s EOFIt might take a few minutes for the
RateLimitPolicyto be applied depending on your cluster. The limit in this example is artificially low to show it working easily.Check that your
RateLimitPolicyhasAcceptedandEnforcedstatus as follows:$ oc get ratelimitpolicy ${KUADRANT_GATEWAY_NAME}-rlp -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'
1.6.4. Set the DNS policy Copy linkLink copied to clipboard!
Procedure
Set the
DNSPolicyfor your Gateway as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: DNSPolicy metadata: name: ${KUADRANT_GATEWAY_NAME}-dnspolicy namespace: ${KUADRANT_GATEWAY_NS} spec: healthCheck: failureThreshold: 3 interval: 1m path: /health loadBalancing: defaultGeo: true geo: GEO-NA weight: 120 targetRef: name: ${KUADRANT_GATEWAY_NAME} group: gateway.networking.k8s.io kind: Gateway providerRefs: - name: aws-credentials # Secret created earlier EOFThe
DNSPolicyuses the DNS ProviderSecretthat you defined earlier. Thegeoin this example isGEO-NA, but you can change this to suit your requirements.Check that your
DNSPolicyhas status ofAcceptedandEnforcedas follows:$ oc get dnspolicy ${KUADRANT_GATEWAY_NAME}-dnspolicy -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'This might take a few minutes.
Check the status of the DNS health checks that are enabled on your DNSPolicy as follows:
$ oc get dnspolicy ${KUADRANT_GATEWAY_NAME}-dnspolicy -n ${KUADRANT_GATEWAY_NS} -These health checks flag a published endpoint as healthy or unhealthy based on defined configuration. When unhealthy, an endpoint will not be published if it has not already been published to the DNS provider. An endpoint will only be unpublished if it is part of a multi-value A record, and in all cases can be observed in the DNSPolicy status.
1.6.5. Test your default rate limit and auth policies Copy linkLink copied to clipboard!
You can use a curl command to test the default low-limit and deny-all policies for your Gateway.
Procedure
Enter the following
curlcommand:$ while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; doneYou should see a HTTP
403responses.
1.7. About token-based rate limiting with TokenRateLimitPolicy Copy linkLink copied to clipboard!
Red Hat Connectivity Link provides the TokenRateLimitPolicy custom resource to enforce rate limits based on token consumption rather than the number of requests. This policy extends the Envoy Rate Limit Service (RLS) protocol with automatic token usage extraction. It is particularly useful for protecting Large Language Model (LLM) APIs, where the cost and resource usage correlate more closely with the number of tokens processed.
Unlike the standard RateLimitPolicy which counts requests, TokenRateLimitPolicy counts tokens by extracting usage metrics in the body of the AI inference API call, allowing for finer-grained control over API usage based on actual workload.
1.7.1. How token rate limiting works Copy linkLink copied to clipboard!
The TokenRateLimitPolicy tracks cumulative token usage per client. Before forwarding a request, it checks if the client has already exceeded their limit from previous usage. After the upstream responds, it extracts the actual token cost and updates the client’s counter.
The flow is as follows:
-
On an incoming request, the gateway evaluates the matching rules and predicates from the
TokenRateLimitPolicyresources. - If the request matches, the gateway prepares the necessary rate limit descriptors and monitors the response.
-
After receiving the response, the gateway extracts the
usage.total_tokensfield from the JSON response body. -
The gateway then sends a
RateLimitRequestto Limitador, including the actual token count as ahits_addend. -
Limitador tracks the cumulative token usage and responds to the gateway with
OKorOVER_LIMIT.
1.7.2. Key features and use cases Copy linkLink copied to clipboard!
-
Enforces limits based on token usage by extracting the
usage.total_tokensfield from an OpenAI-style inference JSON response body. - Suitable for consumption-based APIs such as LLMs where the cost is tied to token counts.
- Allows defining different limits based on criteria such as user identity, API endpoints, or HTTP methods.
-
Works with
AuthPolicyto apply specific limits to authenticated users or groups. -
Inherits functionalities from
RateLimitPolicy, including defining multiple limits with different durations and using Redis for shared counters in multi-cluster environments.
1.7.3. Integrating with AuthPolicy Copy linkLink copied to clipboard!
You can combine TokenRateLimitPolicy with AuthPolicy to apply token limits based on authenticated user identity. When an AuthPolicy successfully authenticates a request, it injects identity information that is used by the TokenRateLimitPolicy to select the appropriate limit.
For example, you can define different token limits for users belonging to 'free-tier' compared to 'premium-tier' groups, identified using claims in a JWT validated by AuthPolicy.
1.8. Configure token-based rate limiting with TokenRateLimitPolicy Copy linkLink copied to clipboard!
Red Hat Connectivity Link provides the TokenRateLimitPolicy custom resource to enforce rate limits based on token consumption rather than the number of requests. This policy extends the Envoy Rate Limit Service (RLS) protocol with automatic token usage extraction. It is particularly useful for protecting Large Language Model (LLM) APIs, where the cost and resource usage correlate more closely with the number of tokens processed.
Unlike the standard RateLimitPolicy which counts requests, TokenRateLimitPolicy counts tokens by extracting usage metrics in the body of the AI inference API call, allowing for finer-grained control over API usage based on actual workload.
1.8.1. How token rate limiting works Copy linkLink copied to clipboard!
The TokenRateLimitPolicy tracks cumulative token usage per client. Before forwarding a request, it checks if the client has already exceeded their limit from previous usage. After the upstream responds, it extracts the actual token cost and updates the client’s counter.
The flow is as follows:
-
On an incoming request, the gateway evaluates the matching rules and predicates from the
TokenRateLimitPolicyresources. - If the request matches, the gateway prepares the necessary rate limit descriptors and monitors the response.
-
After receiving the response, the gateway extracts the
usage.total_tokensfield from the JSON response body. -
The gateway then sends a
RateLimitRequestto Limitador, including the actual token count as ahits_addend. -
Limitador tracks the cumulative token usage and responds to the gateway with
OKorOVER_LIMIT.
1.8.2. Key features and use cases Copy linkLink copied to clipboard!
Token-based rate limiting means you complete the following tasks:
-
Enforces limits based on token usage by extracting the
usage.total_tokensfield from an OpenAI-style inference JSON response body. - Suitable for consumption-based APIs such as LLMs where the cost is tied to token counts.
- Allows defining different limits based on criteria such as user identity, API endpoints, or HTTP methods.
-
Works with
AuthPolicyto apply specific limits to authenticated users or groups. -
Inherits functionalities from
RateLimitPolicy, including defining multiple limits with different durations and using Redis for shared counters in multi-cluster environments.
1.8.3. Integrating with AuthPolicy Copy linkLink copied to clipboard!
You can combine TokenRateLimitPolicy with AuthPolicy to apply token limits based on authenticated user identity. When an AuthPolicy successfully authenticates a request, it injects identity information which can then be used by the TokenRateLimitPolicy to select the appropriate limit.
For example, you can define different token limits for users belonging to 'free-tier' versus 'premium-tier' groups, identified using claims in a JWT validated by AuthPolicy.
1.8.4. Configure token-based rate limiting for LLM APIs Copy linkLink copied to clipboard!
This guide shows how to configure TokenRateLimitPolicy to You can protect a hypothetical LLM API deployed on OpenShift Container Platform, integrated with AuthPolicy for user-specific limits.
Prerequisites
- Connectivity Link is installed on your OpenShift Container Platform cluster.
-
A Gateway and an
HTTPRouteare configured to expose your service. -
An
AuthPolicyis configured for authentication (for example, using API keys or OIDC). - Redis is configured for Limitador if running in a multi-cluster setup or requiring persistent counters.
-
Your upstream service is configured to return an OpenAI-compatible JSON response containing a
usage.total_tokensfield in the response body.
Procedure
Create a
TokenRateLimitPolicyresource. This example defines two limits: one for free users on a 10,000 tokens per day request limit, and one for pro users with a 100,000 tokens per day request limit.apiVersion: kuadrant.io/v1alpha1 kind: TokenRateLimitPolicy metadata: name: llm-protection spec: targetRef: group: gateway.networking.k8s.io kind: Gateway name: ai-gateway limits: free-users: rates: - limit: 10000 # 10k tokens per day for free tier window: 24h when: - predicate: request.path == "/v1/chat/completions" # Inference traffic only - predicate: | auth.identity.groups.split(",").exists(g, g == "free") counters: - expression: auth.identity.userid pro-users: rates: - limit: 100000 # 200 tokens per minute for pro users window: 24h when: - predicate: request.path == "/v1/chat/completions" # Inference traffic only - predicate: | auth.identity.groups.split(",").exists(g, g == "pro") counters: - expression: auth.identity.useridApply the policy:
$ oc apply -f your-tokenratelimitpolicy.yaml -n my-api-namespaceCheck the status of the policy to ensure it has been accepted and enforced on the target
HTTPRoute. Look for conditions withtype: Acceptedandtype: Enforcedwithstatus: "True".$ oc get tokenratelimitpolicy llm-protection -n my-api-namespace -o jsonpath='{.status.conditions}'Send requests to your API endpoint, including the required authentication details.
$ curl -H "Authorization: <auth-details>" \ -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}' \ <your-api-endpoint>
Verification
-
Ensure that your upstream service responds with an OpenAI-compatible JSON body containing the
usage.total_tokensfield. -
Requests made when the client is within their token limits should receive a
200 OKresponse or other success status and their token counter is updated. -
Requests made when the client has already exceeded their token limits should receive a
429 Too Many Requestsresponse.
1.9. Override your gateway policies for auth and rate limiting Copy linkLink copied to clipboard!
As an application developer, you can override your existing gateway-level policies to configure your application-level auth and rate limiting requirements.
You can allow authenticated access to the Toystore API by defining a new AuthPolicy that targets the HTTPRoute resource created in the previous section.
Any new HTTPRoutes are affected by the existing gateway-level policy. Because you want users to now access this API, you must override that gateway policy. For simplicity, you can use API keys to authenticate the requests, but other options such as OpenID Connect are also available.
Prerequisites
- Connectivity Link is installed.
- You configured Connectivity Link policies.
-
You installed the OpenShift CLI (
oc). - You are logged into OpenShift Container Platform as a cluster administrator.
Procedure
Ensure that your Connectivity Link system namespace is set correctly by running the following command:
$ export KUADRANT_SYSTEM_NS=$(oc get kuadrant -A -o jsonpath="{.items[0].metadata.namespace}")Define API keys for bob and alice users as follows:
$ oc apply -f - <<EOF apiVersion: v1 kind: Secret metadata: name: bob-key namespace: ${KUADRANT_SYSTEM_NS} labels: authorino.kuadrant.io/managed-by: authorino app: toystore annotations: secret.kuadrant.io/user-id: bob stringData: api_key: IAMBOB type: Opaque --- apiVersion: v1 kind: Secret metadata: name: alice-key namespace: ${KUADRANT_SYSTEM_NS} labels: authorino.kuadrant.io/managed-by: authorino app: toystore annotations: secret.kuadrant.io/user-id: alice stringData: api_key: IAMALICE type: Opaque EOFCreate a new
AuthPolicyin a different namespace that overrides thedeny-allpolicy created earlier and accepts the API keys as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: AuthPolicy metadata: name: toystore-auth namespace: ${KUADRANT_DEVELOPER_NS} spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore defaults: when: - predicate: "request.path != '/health'" rules: authentication: "api-key-users": apiKey: selector: matchLabels: app: toystore credentials: authorizationHeader: prefix: APIKEY response: success: filters: "identity": json: properties: "userid": selector: auth.identity.metadata.annotations.secret\.kuadrant\.io/user-id EOF
1.10. Overriding the low-limit RateLimitPolicy for specific users Copy linkLink copied to clipboard!
The configured Gateway limits provide a good set of limits for the general case. However, as the developer of the Toystore API, you might want to only allow a certain number of requests for specific users, and a general limit for all other users.
Any new HTTPRoutes are affected by the existing Gateway-level policy. Because you want users to now access this API, you must override that Gateway policy. For simplicity, you can use API keys to authenticate the requests, but other options such as OpenID Connect are also available.
Prerequisites
- You installed Connectivity Link on one or more clusters.
- If you plan to use rate-limiting in a multicluster environment, you have a shared Redis-based datastore.
-
You installed the OpenShift CLI (
oc). - You have write access to the OpenShift Container Platform namespaces you need to work with.
- You have access to external or on-premise DNS.
- You created a gateway.
-
You configured your gateway policies and
HTTProutes.
Procedure
Create a new
RateLimitPolicyin a different namespace to override the defaultlow-limitpolicy created previously and set rate limits for specific users as follows:$ oc apply -f - <<EOF apiVersion: kuadrant.io/v1 kind: RateLimitPolicy metadata: name: toystore-rlp namespace: ${KUADRANT_DEVELOPER_NS} spec: targetRef: group: gateway.networking.k8s.io kind: HTTPRoute name: toystore limits: "general-user": rates: - limit: 5 window: 10s counters: - expression: auth.identity.userid when: - predicate: "auth.identity.userid != 'bob'" "bob-limit": rates: - limit: 2 window: 10s when: - predicate: "auth.identity.userid == 'bob'" EOFIt might take a few minutes for the
RateLimitPolicyto be applied, depending on your cluster.Check that the
RateLimitPolicyhas a status ofAcceptedandEnforcedas follows:$ oc get ratelimitpolicy -n ${KUADRANT_DEVELOPER_NS} toystore-rlp -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'Check that the status of the
HTTPRouteis now affected by theRateLimitPolicyin the same namespace:$ oc get httproute toystore -n ${KUADRANT_DEVELOPER_NS} -o=jsonpath='{.status.parents[0].conditions[?(@.type=="kuadrant.io/RateLimitPolicyAffected")].message}'
Verification
Send requests as user alice as follows:
$ while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMALICE' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; doneYou should see HTTP status
200every second for 5 seconds, followed by HTTP status429every second for 5 seconds.Send requests as user bob as follows:
$ while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMBOB' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; doneYou should see HTTP status
200every second for 2 seconds, followed by HTTP status429every second for 8 seconds.