6.4. Configuring cluster memory to meet container memory and risk requirements

1. Overriding the JVM maximum heap size.
2. Encouraging the JVM to release unused memory to the operating system, if appropriate.
3. Ensuring all JVM processes within a container are appropriately configured.

Procedure

1. Configure the pod to add the MEMORY_REQUEST and MEMORY_LIMIT stanzas:

   apiVersion: v1
   kind: Pod
   metadata:
     name: test
   spec:
     containers:
     - name: test
       image: fedora:latest
       command:
       - sleep
       - "3600"
       env:
       - name: MEMORY_REQUEST 1
         valueFrom:
           resourceFieldRef:
             containerName: test
             resource: requests.memory
       - name: MEMORY_LIMIT 2
         valueFrom:
           resourceFieldRef:
             containerName: test
             resource: limits.memory
       resources:
         requests:
           memory: 384Mi
         limits:
           memory: 512Mi

   1

   Add this stanza to discover the application memory request value.

   2

   Add this stanza to discover the application memory limit value.

2. Create the pod:

   $ oc create -f <file-name>.yaml

3. Access the pod using a remote shell:

   $ oc rsh test

4. Check that the requested values were applied:

   $ env | grep MEMORY | sort

   Example output

   MEMORY_LIMIT=536870912
   MEMORY_REQUEST=402653184

1. Access the pod using a remote shell:

   # oc rsh test

2. Run the following command to see the current OOM kill count in /sys/fs/cgroup/memory/memory.oom_control:

   $ grep '^oom_kill ' /sys/fs/cgroup/memory/memory.oom_control
   oom_kill 0

3. Run the following command to provoke an OOM kill:

   $ sed -e '' </dev/zero

   Example output

   Killed

4. Run the following command to view the exit status of the sed command:

   $ echo $?

   Example output

   137

   The 137 code indicates the container process exited with code 137, indicating it received a SIGKILL signal.

5. Run the following command to see that the OOM kill counter in /sys/fs/cgroup/memory/memory.oom_control incremented:

   $ grep '^oom_kill ' /sys/fs/cgroup/memory/memory.oom_control
   oom_kill 1

   If one or more processes in a pod are OOM killed, when the pod subsequently exits, whether immediately or not, it will have phase Failed and reason OOMKilled. An OOM-killed pod might be restarted depending on the value of restartPolicy. If not restarted, controllers such as the replication controller will notice the pod’s failed status and create a new pod to replace the old one.

   Use the follwing command to get the pod status:

   $ oc get pod test

   Example output

   NAME      READY     STATUS      RESTARTS   AGE
   test      0/1       OOMKilled   0          1m

   • If the pod has not restarted, run the following command to view the pod:

     $ oc get pod test -o yaml

     Example output

     ...
     status:
       containerStatuses:
       - name: test
         ready: false
         restartCount: 0
         state:
           terminated:
             exitCode: 137
             reason: OOMKilled
       phase: Failed

   • If restarted, run the following command to view the pod:

     $ oc get pod test -o yaml

     Example output

     ...
     status:
       containerStatuses:
       - name: test
         ready: true
         restartCount: 1
         lastState:
           terminated:
             exitCode: 137
             reason: OOMKilled
         state:
           running:
       phase: Running