Chapter 2. Scalability and performance of OpenShift Serverless Serving

• 170 Knative Services
• 1-2 Revisions per Knative Service
• 89 test scenarios mainly focused on testing the control plane
• 48 re-creating scenarios where Knative Services are deleted and re-created
• 41 stable scenarios, in which requests are slowly but continuously sent to the system

Procedure

• You can configure Knative Serving for minimal workloads using the KnativeServing custom resource (CR):

  A minimal workload configuration in KnativeServing CR

  apiVersion: operator.knative.dev/v1beta1
  kind: KnativeServing
  metadata:
    name: knative-serving
    namespace: knative-serving
  spec:
    high-availability:
      replicas: 1 1
    workloads:
      - name: activator
        replicas: 2 2
        resources:
          - container: activator
            requests:
              cpu: 250m 3
              memory: 60Mi 4
            limits:
              cpu: 1000m
              memory: 600Mi
      - name: controller
        replicas: 1 5
        resources:
          - container: controller
            requests:
              cpu: 10m
              memory: 100Mi
            limits: 6
              cpu: 200m
              memory: 300Mi
      - name: webhook
        replicas: 2
        resources:
          - container: webhook
            requests:
              cpu: 100m 7
              memory: 60Mi
            limits:
              cpu: 200m
              memory: 200Mi
    podDisruptionBudgets: 8
      - name: activator-pdb
        minAvailable: 1
      - name: webhook-pdb
        minAvailable: 1

  1

  Setting this to 1 scales all system components to one replica.

  2

  Activator should always be scaled to a minimum of 2 instances to avoid downtime.

  3

  Activator CPU requests should not be set lower than 250m, as a HorizontalPodAutoscaler will use this as a reference to scale up and down.

  4

  Adjust memory requests to the idle values from the previous table. Also adjust memory limits according to your expected load (this might need custom testing to find the best values).

  5

  One webhook and one controller are sufficient for a minimal-workload scenario

  6

  These limits are sufficient for a minimal-workload scenario, but they also might need adjustments depending on your concrete workload.

  7

  Webhook CPU requests should not be set lower than 100m, as a HorizontalPodAutoscaler will use this as a reference to scale up and down.

  8

  Adjust the PodDistruptionBudgets to a value lower than replicas, to avoid problems during node maintenance.

• All data-plane components are mostly increasing CPU usage on higher requests and payload scenarios, so the CPU requests and limits have to be tested and potentially increased.
• The activator component also might need more memory, when it has to buffer more or bigger request payloads, so the memory requests and limits might need to be increased as well.
• One activator pod can handle approximately 2500 requests per second before it starts to increase latency and, at some point, leads to errors.
• One 3scale-kourier-gateway or istio-ingressgateway pod can also handle approximately 2500 requests per second before it starts to increase latency and, at some point, leads to errors.
• Each of the data-plane components consumes up to 1 vCPU of CPU for handling 2500 requests per second. Note that this highly depends on the payload size and the response times of the Knative Service backend.

Procedure

• To fine-tune your installation, use the previous findings combined with your own test results to configure the KnativeServing custom resource:

  A high workload configuration in KnativeServing CR

  apiVersion: operator.knative.dev/v1beta1
  kind: KnativeServing
  metadata:
    name: knative-serving
    namespace: knative-serving
  spec:
    high-availability:
      replicas: 2 1
    workloads:
      - name: component-name 2
        replicas: 2 3
        resources:
          - container: container-name
            requests:
              cpu: 4
              memory:
            limits:
              cpu:
              memory:
    podDisruptionBudgets: 5
      - name: name-of-pod-disruption-budget
        minAvailable: 1

  1

  Set this parameter to at least 2 to make sure you always have at least two instances of every component running. You can also use workloads to override the replicas for certain components.

  2

  Use the workloads list to configure specific components. Use the deployment name of the component and set the replicas field.

  3

  For the activator, webhook, and 3scale-kourier-gateway components, which use horizontal pod autoscalers (HPAs), the replicas field sets the minimum number of replicas. The actual number of replicas depends on the CPU load and scaling done by the HPAs.

  4

  Set the requested and limited CPU and memory according to at least the idle consumption while also taking the previous findings and your own test results into consideration.

  5

  Adjust the PodDistruptionBudgets to a value lower than replicas to avoid problems during node maintenance. The default minAvailable is set to 1, so if you increase the required replicas, you must also increase minAvailable.