6.5. Configuring your cluster to place pods on overcommitted nodes

1. In the OpenShift Container Platform web console, navigate to Home Projects

   1. Click Create Project.
   2. Specify clusterresourceoverride-operator as the name of the project.
   3. Click Create.

2. Navigate to Operators OperatorHub.

   1. Choose ClusterResourceOverride Operator from the list of available Operators and click Install.
   2. On the Install Operator page, make sure A specific Namespace on the cluster is selected for Installation Mode.
   3. Make sure clusterresourceoverride-operator is selected for Installed Namespace.
   4. Select an Update Channel and Approval Strategy.
   5. Click Install.

3. On the Installed Operators page, click ClusterResourceOverride.

   1. On the ClusterResourceOverride Operator details page, click Create Instance.

   2. On the Create ClusterResourceOverride page, edit the YAML template to set the overcommit values as needed:

      apiVersion: operator.autoscaling.openshift.io/v1
      kind: ClusterResourceOverride
      metadata:
        name: cluster 1
      spec:
        podResourceOverride:
          spec:
            memoryRequestToLimitPercent: 50 2
            cpuRequestToLimitPercent: 25 3
            limitCPUToMemoryPercent: 200 4

      1

      The name must be cluster.

      2

      Optional. Specify the percentage to override the container memory limit, if used, between 1-100. The default is 50.

      3

      Optional. Specify the percentage to override the container CPU limit, if used, between 1-100. The default is 25.

      4

      Optional. Specify the percentage to override the container memory limit, if used. Scaling 1Gi of RAM at 100 percent is equal to 1 CPU core. This is processed prior to overriding the CPU request, if configured. The default is 200.

   3. Click Create.

4. Check the current state of the admission webhook by checking the status of the cluster custom resource:

   1. On the ClusterResourceOverride Operator page, click cluster.

   2. On the ClusterResourceOverride Details age, click YAML. The mutatingWebhookConfigurationRef section appears when the webhook is called.

      apiVersion: operator.autoscaling.openshift.io/v1
      kind: ClusterResourceOverride
      metadata:
        annotations:
          kubectl.kubernetes.io/last-applied-configuration: |
            {"apiVersion":"operator.autoscaling.openshift.io/v1","kind":"ClusterResourceOverride","metadata":{"annotations":{},"name":"cluster"},"spec":{"podResourceOverride":{"spec":{"cpuRequestToLimitPercent":25,"limitCPUToMemoryPercent":200,"memoryRequestToLimitPercent":50}}}}
        creationTimestamp: "2019-12-18T22:35:02Z"
        generation: 1
        name: cluster
        resourceVersion: "127622"
        selfLink: /apis/operator.autoscaling.openshift.io/v1/clusterresourceoverrides/cluster
        uid: 978fc959-1717-4bd1-97d0-ae00ee111e8d
      spec:
        podResourceOverride:
          spec:
            cpuRequestToLimitPercent: 25
            limitCPUToMemoryPercent: 200
            memoryRequestToLimitPercent: 50
      status:
      
      ....
      
          mutatingWebhookConfigurationRef: 1
            apiVersion: admissionregistration.k8s.io/v1beta1
            kind: MutatingWebhookConfiguration
            name: clusterresourceoverrides.admission.autoscaling.openshift.io
            resourceVersion: "127621"
            uid: 98b3b8ae-d5ce-462b-8ab5-a729ea8f38f3
      
      ....

      1

      Reference to the ClusterResourceOverride admission webhook.

1. Create a namespace for the Cluster Resource Override Operator:

   1. Create a Namespace object YAML file (for example, cro-namespace.yaml) for the Cluster Resource Override Operator:

      apiVersion: v1
      kind: Namespace
      metadata:
        name: clusterresourceoverride-operator

   2. Create the namespace:

      $ oc create -f <file-name>.yaml

      For example:

      $ oc create -f cro-namespace.yaml

2. Create an Operator group:

   1. Create an OperatorGroup object YAML file (for example, cro-og.yaml) for the Cluster Resource Override Operator:

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: clusterresourceoverride-operator
        namespace: clusterresourceoverride-operator
      spec:
        targetNamespaces:
          - clusterresourceoverride-operator

   2. Create the Operator Group:

      $ oc create -f <file-name>.yaml

      For example:

      $ oc create -f cro-og.yaml

3. Create a subscription:

   1. Create a Subscription object YAML file (for example, cro-sub.yaml) for the Cluster Resource Override Operator:

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: clusterresourceoverride
        namespace: clusterresourceoverride-operator
      spec:
        channel: "4.5"
        name: clusterresourceoverride
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   2. Create the subscription:

      $ oc create -f <file-name>.yaml

      For example:

      $ oc create -f cro-sub.yaml

4. Create a ClusterResourceOverride custom resource (CR) object in the clusterresourceoverride-operator namespace:

   1. Change to the clusterresourceoverride-operator namespace.

      $ oc project clusterresourceoverride-operator

   2. Create a ClusterResourceOverride object YAML file (for example, cro-cr.yaml) for the Cluster Resource Override Operator:

      apiVersion: operator.autoscaling.openshift.io/v1
      kind: ClusterResourceOverride
      metadata:
          name: cluster 1
      spec:
        podResourceOverride:
          spec:
            memoryRequestToLimitPercent: 50 2
            cpuRequestToLimitPercent: 25 3
            limitCPUToMemoryPercent: 200 4

      1

      The name must be cluster.

      2

      Optional. Specify the percentage to override the container memory limit, if used, between 1-100. The default is 50.

      3

      Optional. Specify the percentage to override the container CPU limit, if used, between 1-100. The default is 25.

      4

      Optional. Specify the percentage to override the container memory limit, if used. Scaling 1Gi of RAM at 100 percent is equal to 1 CPU core. This is processed prior to overriding the CPU request, if configured. The default is 200.

   3. Create the ClusterResourceOverride object:

      $ oc create -f <file-name>.yaml

      For example:

      $ oc create -f cro-cr.yaml

5. Verify the current state of the admission webhook by checking the status of the cluster custom resource.

   $ oc get clusterresourceoverride cluster -n clusterresourceoverride-operator -o yaml

   The mutatingWebhookConfigurationRef section appears when the webhook is called.

   Example output

   apiVersion: operator.autoscaling.openshift.io/v1
   kind: ClusterResourceOverride
   metadata:
     annotations:
       kubectl.kubernetes.io/last-applied-configuration: |
         {"apiVersion":"operator.autoscaling.openshift.io/v1","kind":"ClusterResourceOverride","metadata":{"annotations":{},"name":"cluster"},"spec":{"podResourceOverride":{"spec":{"cpuRequestToLimitPercent":25,"limitCPUToMemoryPercent":200,"memoryRequestToLimitPercent":50}}}}
     creationTimestamp: "2019-12-18T22:35:02Z"
     generation: 1
     name: cluster
     resourceVersion: "127622"
     selfLink: /apis/operator.autoscaling.openshift.io/v1/clusterresourceoverrides/cluster
     uid: 978fc959-1717-4bd1-97d0-ae00ee111e8d
   spec:
     podResourceOverride:
       spec:
         cpuRequestToLimitPercent: 25
         limitCPUToMemoryPercent: 200
         memoryRequestToLimitPercent: 50
   status:
   
   ....
   
       mutatingWebhookConfigurationRef: 1
         apiVersion: admissionregistration.k8s.io/v1beta1
         kind: MutatingWebhookConfiguration
         name: clusterresourceoverrides.admission.autoscaling.openshift.io
         resourceVersion: "127621"
         uid: 98b3b8ae-d5ce-462b-8ab5-a729ea8f38f3
   
   ....

   1

   Reference to the ClusterResourceOverride admission webhook.

1. Edit the ClusterResourceOverride CR:

   apiVersion: operator.autoscaling.openshift.io/v1
   kind: ClusterResourceOverride
   metadata:
   -   name: cluster
   spec:
      memoryRequestToLimitPercent: 50 1
      cpuRequestToLimitPercent: 25 2
      limitCPUToMemoryPercent: 200 3

   1

   Optional. Specify the percentage to override the container memory limit, if used, between 1-100. The default is 50.

   2

   Optional. Specify the percentage to override the container CPU limit, if used, between 1-100. The default is 25.

   3

   Optional. Specify the percentage to override the container memory limit, if used. Scaling 1Gi of RAM at 100 percent is equal to 1 CPU core. This is processed prior to overriding the CPU request, if configured. The default is 200.

2. Ensure the following label has been added to the Namespace object for each project where you want the Cluster Resource Override Operator to control overcommit:

   apiVersion: v1
   kind: Namespace
   metadata:
   
   ....
   
     labels:
       clusterresourceoverrides.admission.autoscaling.openshift.io/enabled: "true" 1
   
   ....

   1

   Add this label to each project.

Prerequisites

1. Obtain the label associated with the static MachineConfigPool CRD for the type of node you want to configure. Perform one of the following steps:

   1. View the machine config pool:

      $ oc describe machineconfigpool <name>

      For example:

      $ oc describe machineconfigpool worker

      Example output

      apiVersion: machineconfiguration.openshift.io/v1
      kind: MachineConfigPool
      metadata:
        creationTimestamp: 2019-02-08T14:52:39Z
        generation: 1
        labels:
          custom-kubelet: small-pods 1

      1

      If a label has been added it appears under labels.

   2. If the label is not present, add a key/value pair:

      $ oc label machineconfigpool worker custom-kubelet=small-pods

Procedure

1. Create a custom resource (CR) for your configuration change.

   Sample configuration for a disabling CPU limits

   apiVersion: machineconfiguration.openshift.io/v1
   kind: KubeletConfig
   metadata:
     name: disable-cpu-units 1
   spec:
     machineConfigPoolSelector:
       matchLabels:
         custom-kubelet: small-pods 2
     kubeletConfig:
       cpuCfsQuota: 3
         - "false"

   1

   Assign a name to CR.

   2

   Specify the label to apply the configuration change.

   3

   Set the cpuCfsQuota parameter to false.

1. Edit the project object file

2. Add the following annotation:

   quota.openshift.io/cluster-resource-override-enabled: "false"

3. Create the project object:

   $ oc create -f <file-name>.yaml