Configuring and deploying Gateway policies with Connectivity Link

Procedure

1. Set the TLSPolicy for your Gateway as follows:

   kubectl apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: TLSPolicy
   metadata:
     name: ${KUADRANT_GATEWAY_NAME}-tls
     namespace: ${KUADRANT_GATEWAY_NS}
   spec:
     targetRef:
       name: ${KUADRANT_GATEWAY_NAME}
       group: gateway.networking.k8s.io
       kind: Gateway
     issuerRef:
       group: cert-manager.io
       kind: ClusterIssuer
       name: ${KUADRANT_CLUSTER_ISSUER_NAME}
   EOF

   Copy to Clipboard Toggle word wrap

2. Check that your TLS policy has an Accepted and Enforced status as follows:

   kubectl get tlspolicy ${KUADRANT_GATEWAY_NAME}-tls -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy to Clipboard Toggle word wrap

   This may take a few minutes depending on the TLS provider, for example, Let’s Encrypt.

Procedure

1. Create an HTTPRoute for the example Toystore application as follows:

   kubectl apply -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: toystore
     namespace: ${KUADRANT_DEVELOPER_NS}
     labels:
       deployment: toystore
       service: toystore
   spec:
     parentRefs:
     - name: ${KUADRANT_GATEWAY_NAME}
       namespace: ${KUADRANT_GATEWAY_NS}
     hostnames:
     - "api.${KUADRANT_ZONE_ROOT_DOMAIN}"
     rules:
     - matches:
       - method: GET
         path:
           type: PathPrefix
           value: "/cars"
       - method: GET
         path:
           type: PathPrefix
           value: "/health"
       backendRefs:
       - name: toystore
         port: 80
   EOF

   Copy to Clipboard Toggle word wrap

   While the Gateway is deployed, it currently has exposed endpoints. The next steps are defining an AuthPolicy to set up a default HTTP 403 response for any unprotected endpoints, and a RateLimitPolicy to set up a default unrealistic low global limit to further protect any exposed endpoints.

Procedure

1. Set a default AuthPolicy with a deny-all setting for your Gateway as follows:

   kubectl apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: AuthPolicy
   metadata:
     name: ${KUADRANT_GATEWAY_NAME}-auth
     namespace: ${KUADRANT_GATEWAY_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: Gateway
       name: ${KUADRANT_GATEWAY_NAME}
     defaults:
      when:
        - predicate: "request.path != '/health'"
      rules:
       authorization:
         deny-all:
           opa:
             rego: "allow = false"
       response:
         unauthorized:
           headers:
             "content-type":
               value: application/json
           body:
             value: |
               {
                 "error": "Forbidden",
                 "message": "Access denied by default by the gateway operator. If you are the administrator of the service, create a specific auth policy for the route."
               }
   EOF

   Copy to Clipboard Toggle word wrap

2. Check that your AuthPolicy has Accepted and Enforced status as follows:

   kubectl get authpolicy ${KUADRANT_GATEWAY_NAME}-auth -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy to Clipboard Toggle word wrap

Procedure

1. Set the default RateLimitPolicy with a low-limit setting for your Gateway as follows:

   kubectl apply -f  - <<EOF
   apiVersion: kuadrant.io/v1
   kind: RateLimitPolicy
   metadata:
     name: ${KUADRANT_GATEWAY_NAME}-rlp
     namespace: ${KUADRANT_GATEWAY_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: Gateway
       name: ${KUADRANT_GATEWAY_NAME}
     defaults:
       limits:
         "low-limit":
           rates:
           - limit: 1
             window: 10s
   EOF

   Copy to Clipboard Toggle word wrap
   Note

   It might take a few minutes for the RateLimitPolicy to be applied depending on your cluster. The limit in this example is artificially low to show it working easily.

2. Check that your RateLimitPolicy has Accepted and Enforced status as follows:

   kubectl get ratelimitpolicy ${KUADRANT_GATEWAY_NAME}-rlp -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy to Clipboard Toggle word wrap

Procedure

1. Set the DNSPolicy for your Gateway as follows:

   kubectl apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: DNSPolicy
   metadata:
     name: ${KUADRANT_GATEWAY_NAME}-dnspolicy
     namespace: ${KUADRANT_GATEWAY_NS}
   spec:
     healthCheck:
       failureThreshold: 3
       interval: 1m
       path: /health
     loadBalancing:
       defaultGeo: true
       geo: GEO-NA
       weight: 120
     targetRef:
       name: ${KUADRANT_GATEWAY_NAME}
       group: gateway.networking.k8s.io
       kind: Gateway
     providerRefs:
     - name: aws-credentials # Secret created earlier
   EOF

   Copy to Clipboard Toggle word wrap
   Note

   The DNSPolicy will use the DNS Provider Secret that you defined earlier. The geo in this example is GEO-NA, but you can change this to suit your requirements.

2. Check that your DNSPolicy has status of Accepted and Enforced as follows:

   kubectl get dnspolicy ${KUADRANT_GATEWAY_NAME}-dnspolicy -n ${KUADRANT_GATEWAY_NS} -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy to Clipboard Toggle word wrap

   This might take a few minutes.

3. Check the status of the DNS health checks that are enabled on your DNSPolicy as follows:

   kubectl get dnspolicy ${KUADRANT_GATEWAY_NAME}-dnspolicy -n ${KUADRANT_GATEWAY_NS} -

   Copy to Clipboard Toggle word wrap

   These health checks flag a published endpoint as healthy or unhealthy based on defined configuration. When unhealthy, an endpoint will not be published if it has not already been published to the DNS provider. An endpoint will only be unpublished if it is part of a multi-value A record, and in all cases can be observed in the DNSPolicy status.

1. On an incoming request, the gateway evaluates the matching rules and predicates from the TokenRateLimitPolicy resources.
2. If the request matches, the gateway prepares the necessary rate limit descriptors and monitors the response.
3. After receiving the response, the gateway extracts the usage.total_tokens field from the JSON response body.
4. The gateway then sends a RateLimitRequest to Limitador, including the actual token count as a hits_addend.
5. Limitador tracks the cumulative token usage and responds to the gateway with OK or OVER_LIMIT.

Procedure

1. Create a TokenRateLimitPolicy resource. This example defines two limits: one for free users on a 10,000 tokens per day request limit, and one for pro users with a 100,000 tokens per day request limit.

   apiVersion: kuadrant.io/v1alpha1
   kind: TokenRateLimitPolicy
   metadata:
     name: llm-protection
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: Gateway
       name: ai-gateway
     limits:
       free-users:
         rates:
           - limit: 10000 # 10k tokens per day for free tier
             window: 24h
         when:
           - predicate: request.path == "/v1/chat/completions" # Inference traffic only
           - predicate: |
               auth.identity.groups.split(",").exists(g, g == "free")
         counters:
           - expression: auth.identity.userid
       pro-users:
         rates:
           - limit: 100000 # 200 tokens per minute for pro users
             window: 24h
         when:
           - predicate: request.path == "/v1/chat/completions" # Inference traffic only
           - predicate: |
               auth.identity.groups.split(",").exists(g, g == "pro")
         counters:
           - expression: auth.identity.userid

   Copy to Clipboard Toggle word wrap

2. Apply the policy:

   oc apply -f your-tokenratelimitpolicy.yaml -n my-api-namespace

   Copy to Clipboard Toggle word wrap

3. Check the status of the policy to ensure it has been accepted and enforced on the target HTTPRoute. Look for conditions with type: Accepted and type: Enforced with status: "True".

   oc get tokenratelimitpolicy llm-protection -n my-api-namespace -o jsonpath='{.status.conditions}'

   Copy to Clipboard Toggle word wrap

4. Send requests to your API endpoint, including the required authentication details.

   curl -H "Authorization: <auth-details>" \
        -d '{"model": "gpt-4", "messages": [{"role": "user", "content": "Hello"}]}' \
        <your-api-endpoint>

   Copy to Clipboard Toggle word wrap

Procedure

1. Ensure that your Connectivity Link system namespace is set correctly as follows:

   export KUADRANT_SYSTEM_NS=$(kubectl get kuadrant -A -o jsonpath="{.items[0].metadata.namespace}")

   Copy to Clipboard Toggle word wrap

2. Define API keys for bob and alice users as follows:

   kubectl apply -f - <<EOF
   apiVersion: v1
   kind: Secret
   metadata:
     name: bob-key
     namespace: ${KUADRANT_SYSTEM_NS}
     labels:
       authorino.kuadrant.io/managed-by: authorino
       app: toystore
     annotations:
       secret.kuadrant.io/user-id: bob
   stringData:
     api_key: IAMBOB
   type: Opaque
   ---
   apiVersion: v1
   kind: Secret
   metadata:
     name: alice-key
     namespace: ${KUADRANT_SYSTEM_NS}
     labels:
       authorino.kuadrant.io/managed-by: authorino
       app: toystore
     annotations:
       secret.kuadrant.io/user-id: alice
   stringData:
     api_key: IAMALICE
   type: Opaque
   EOF

   Copy to Clipboard Toggle word wrap

3. Create a new AuthPolicy in a different namespace that overrides the deny-all policy created earlier and accepts the API keys as follows:

   kubectl apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: AuthPolicy
   metadata:
     name: toystore-auth
     namespace: ${KUADRANT_DEVELOPER_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: toystore
     defaults:
      when:
        - predicate: "request.path != '/health'"
      rules:
       authentication:
         "api-key-users":
           apiKey:
             selector:
               matchLabels:
                 app: toystore
           credentials:
             authorizationHeader:
               prefix: APIKEY
       response:
         success:
           filters:
             "identity":
               json:
                 properties:
                   "userid":
                     selector: auth.identity.metadata.annotations.secret\.kuadrant\.io/user-id
   EOF

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a new RateLimitPolicy in a different namespace to override the default low-limit policy created previously and set rate limits for specific users as follows:

   kubectl apply -f - <<EOF
   apiVersion: kuadrant.io/v1
   kind: RateLimitPolicy
   metadata:
     name: toystore-rlp
     namespace: ${KUADRANT_DEVELOPER_NS}
   spec:
     targetRef:
       group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: toystore
     limits:
       "general-user":
         rates:
   
         - limit: 5
           window: 10s
         counters:
         - expression: auth.identity.userid
         when:
         - predicate: "auth.identity.userid != 'bob'"
       "bob-limit":
         rates:
         - limit: 2
           window: 10s
         when:
         - predicate: "auth.identity.userid == 'bob'"
   EOF

   Copy to Clipboard Toggle word wrap
   Note

   It might take a few minutes for the RateLimitPolicy to be applied, depending on your cluster.

2. Check that the RateLimitPolicy has a status of Accepted and Enforced as follows:

   kubectl get ratelimitpolicy -n ${KUADRANT_DEVELOPER_NS} toystore-rlp -o=jsonpath='{.status.conditions[?(@.type=="Accepted")].message}{"\n"}{.status.conditions[?(@.type=="Enforced")].message}'

   Copy to Clipboard Toggle word wrap

3. Check that the status of the HTTPRoute is now affected by the RateLimitPolicy in the same namespace:

   kubectl get httproute toystore -n ${KUADRANT_DEVELOPER_NS} -o=jsonpath='{.status.parents[0].conditions[?(@.type=="kuadrant.io/RateLimitPolicyAffected")].message}'

   Copy to Clipboard Toggle word wrap

1. Send requests as user alice as follows:

   while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMALICE' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; done

   Copy to Clipboard Toggle word wrap

   You should see HTTP status 200 every second for 5 seconds, followed by HTTP status 429 every second for 5 seconds.

2. Send requests as user bob as follows:

   while :; do curl -k --write-out '%{http_code}\n' --silent --output /dev/null -H 'Authorization: APIKEY IAMBOB' "https://api.$KUADRANT_ZONE_ROOT_DOMAIN/cars" | grep -E --color "\b(429)\b|$"; sleep 1; done

   Copy to Clipboard Toggle word wrap

   You should see HTTP status 200 every second for 2 seconds, followed by HTTP status 429 every second for 8 seconds.