Chapter 13. Pruning objects to reclaim resources

• The <object_type> to perform the action on, such as groups, builds, deployments, or images.
• The <options> supported to prune that object type.

Prerequisites

• You have access to an OpenShift Container Platform cluster using an account with cluster administrator permissions.
• Install the oc CLI.

Procedure

• Verify that the object named imagepruners.imageregistry.operator.openshift.io/cluster contains the following spec and status fields:

• Running image pruning as a Job or CronJob on the cluster.
• Running the oc adm prune images command.

Prerequisites

• To prune images, you must first log in to the CLI as a user with an access token. The user must also have the system:image-pruner cluster role or greater (for example, cluster-admin).
• Expose the image registry.

• Run image pruning as a Job or CronJob on the cluster by creating a YAML file for the pruner service account, for example:

  $ oc create -f <filename>.yaml

  Example output

  kind: List
  apiVersion: v1
  items:
  - apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: pruner
      namespace: openshift-image-registry
  - apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: openshift-image-registry-pruner
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:image-pruner
    subjects:
    - kind: ServiceAccount
      name: pruner
      namespace: openshift-image-registry
  - apiVersion: batch/v1
    kind: CronJob
    metadata:
      name: image-pruner
      namespace: openshift-image-registry
    spec:
      schedule: "0 0 * * *"
      concurrencyPolicy: Forbid
      successfulJobsHistoryLimit: 1
      failedJobsHistoryLimit: 3
      jobTemplate:
        spec:
          template:
            spec:
              restartPolicy: OnFailure
              containers:
              - image: "quay.io/openshift/origin-cli:4.1"
                resources:
                  requests:
                    cpu: 1
                    memory: 1Gi
                terminationMessagePolicy: FallbackToLogsOnError
                command:
                - oc
                args:
                - adm
                - prune
                - images
                - --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt
                - --keep-tag-revisions=5
                - --keep-younger-than=96h
                - --confirm=true
                name: image-pruner
              serviceAccountName: pruner

• Run the oc adm prune images [<options>] command:

  $ oc adm prune images [<options>]

  Pruning images removes data from the integrated registry unless --prune-registry=false is used.

  Pruning images with the --namespace flag does not remove images, only image streams. Images are non-namespaced resources. Therefore, limiting pruning to a particular namespace makes it impossible to calculate its current usage.

  By default, the integrated registry caches metadata of blobs to reduce the number of requests to storage, and to increase the request-processing speed. Pruning does not update the integrated registry cache. Images that still contain pruned layers after pruning will be broken because the pruned layers that have metadata in the cache will not be pushed. Therefore, you must redeploy the registry to clear the cache after pruning:

  $ oc rollout restart deployment/image-registry -n openshift-image-registry

  If the integrated registry uses a Redis cache, you must clean the database manually.

  If redeploying the registry after pruning is not an option, then you must permanently disable the cache.

  oc adm prune images operations require a route for your registry. Registry routes are not created by default.

  The Prune images CLI configuration options table describes the options you can use with the oc adm prune images <options> command.

  Table 13.4. Prune images CLI configuration options

  Option	Description
  --all	Include images that were not pushed to the registry, but have been mirrored by pullthrough. This is on by default. To limit the pruning to images that were pushed to the integrated registry, pass --all=false.
  --certificate-authority	The path to a certificate authority file to use when communicating with the OpenShift Container Platform-managed registries. Defaults to the certificate authority data from the current user’s configuration file. If provided, a secure connection is initiated.
  --confirm	Indicate that pruning should occur, instead of performing a test-run. This requires a valid route to the integrated container image registry. If this command is run outside of the cluster network, the route must be provided using --registry-url.
  --force-insecure	Use caution with this option. Allow an insecure connection to the container registry that is hosted via HTTP or has an invalid HTTPS certificate.
  --keep-tag-revisions=<N>	For each imagestream, keep up to at most N image revisions per tag (default 3).
  --keep-younger-than=<duration>	Do not prune any image that is younger than <duration> relative to the current time. Alternately, do not prune any image that is referenced by any other object that is younger than <duration> relative to the current time (default 60m).
  --prune-over-size-limit	Prune each image that exceeds the smallest limit defined in the same project. This flag cannot be combined with --keep-tag-revisions nor --keep-younger-than.
  --registry-url	The address to use when contacting the registry. The command attempts to use a cluster-internal URL determined from managed images and image streams. In case it fails (the registry cannot be resolved or reached), an alternative route that works needs to be provided using this flag. The registry hostname can be prefixed by https:// or http://, which enforces particular connection protocol.
  --prune-registry	In conjunction with the conditions stipulated by the other options, this option controls whether the data in the registry corresponding to the OpenShift Container Platform image API object is pruned. By default, image pruning processes both the image API objects and corresponding data in the registry. This option is useful when you are only concerned with removing etcd content, to reduce the number of image objects but are not concerned with cleaning up registry storage, or if you intend to do that separately by hard pruning the registry during an appropriate maintenance window for the registry.

• Manually deleting an image with oc delete image <sha256:image-id> command, which only removes the image from etcd, but not from the registry’s storage.
• Pushing to the registry initiated by daemon failures, which causes some blobs to get uploaded, but the image manifest (which is uploaded as the very last component) does not. All unique image blobs become orphans.
• OpenShift Container Platform refusing an image because of quota restrictions.
• The standard image pruner deleting an image manifest, but is interrupted before it deletes the related blobs.
• A bug in the registry pruner, which fails to remove the intended blobs, causing the image objects referencing them to be removed and the blobs becoming orphans.