Chapter 1. OADP Application backup and restore


Use OpenShift API for Data Protection (OADP) to safeguard applications, application-related cluster resources, persistent volumes, and internal images on Red Hat OpenShift Service on AWS classic architecture. OADP backs up containerized applications and virtual machines (VMs). This helps you ensure disaster recovery.

Important

OADP support is applicable to customer workload namespaces and cluster scope resources.

Full cluster backup and restore are not supported.

1.1.1. OpenShift API for Data Protection APIs

OADP provides APIs that enable multiple approaches to customizing backups and preventing the inclusion of unnecessary or inappropriate resources.

OADP provides the following APIs. See the Additional resources section for more details.

  • Backup
  • Restore
  • Schedule
  • BackupStorageLocation
  • VolumeSnapshotLocation

Review the OADP support matrix for version compatibility with Red Hat OpenShift Service on AWS classic architecture releases and lifecycle policy information, including Extended Update Support (EUS) options.

Expand
Table 1.1. Supported versions of OADP

Version

Red Hat OpenShift Service on AWS classic architecture version

General availability

Full support ends

Maintenance ends

Extended Update Support (EUS)

Extended Update Support Term 2 (EUS Term 2)

1.6

  • 4.22

09 Jun 2026

Release of 1.7

Release of 1.8

30 Jun 2028

30 Jun 2029

1.5

  • 4.19
  • 4.20
  • 4.21

17 June 2025

Release of 1.6

Release of 1.7

30 Jun 2028

30 Jun 2029

EUS must be on Red Hat OpenShift Service on AWS classic architecture 4.21

EUS Term 2 must be on Red Hat OpenShift Service on AWS classic architecture 4.21

1.4

  • 4.14
  • 4.15
  • 4.16
  • 4.17
  • 4.18

10 Jul 2024

Release of 1.5

Release of 1.6

27 Jun 2026

EUS must be on Red Hat OpenShift Service on AWS classic architecture 4.16

27 Jun 2027

EUS Term 2 must be on Red Hat OpenShift Service on AWS classic architecture 4.16

1.3

  • 4.12
  • 4.13
  • 4.14
  • 4.15

29 Nov 2023

10 Jul 2024

Release of 1.5

31 Oct 2025

EUS must be on Red Hat OpenShift Service on AWS classic architecture 4.14

Expand
Table 1.2. Previous versions of the OADP Operator which are no longer supported

Version

General availability

Full support ended

Maintenance ended

1.2

14 Jun 2023

29 Nov 2023

10 Jul 2024

1.1

01 Sep 2022

14 Jun 2023

29 Nov 2023

1.0

09 Feb 2022

01 Sep 2022

14 Jun 2023

For more details about EUS, see Extended Update Support.

For more details about EUS Term 2, see Extended Update Support Term 2.

1.2. OADP release notes

1.2.1. OADP 1.6 release notes

Review new features, resolved issues, known issues, Technology Preview features, removed features, and other additions in OpenShift API for Data Protection (OADP) 1.6.

Note

For additional information about OADP, see OpenShift API for Data Protection (OADP) FAQ.

1.2.1.1. OADP 1.6.1 release notes

OpenShift API for Data Protection (OADP) 1.6.1 release notes list resolved and known issues.

The following Red Hat Security Advisory (RHSA) is available for OADP 1.6.1:

1.2.1.1.1. Resolved issues
OADP CLI no longer exposes the namespace flag for non-admin operations

Before this update, the OADP command-line interface (CLI) exposed the global --namespace or -n flag in the help output of all oc oadp nonadmin commands, even though non-admin operations do not support namespace selection because of multi-tenancy boundaries. As a consequence, if you specified the namespace flag during a non-admin backup or restore operation, the CLI ignored the provided value and used the active oc project context instead, potentially creating resources in an unintended namespace.

With this update, the --namespace flag is hidden from all oc oadp nonadmin subcommands so that it is no longer displayed in the help output and does not accept user input for non-admin operations. As a result, non-admin CLI commands operate correctly only within your current project context, which eliminates the risk of namespace mismatch.

OADP-7955

Wildcard handling for excludedClusterScopedResources no longer causes validation failures

Before this update, the OADP Operator incorrectly appended default resources to excludedClusterScopedResources in the NonAdminBackup custom resource (CR) when a wildcard (*) was present. As a consequence, FailedValidation errors occurred when you excluded all cluster-scoped resources. With this update, the Operator correctly handles the wildcard without appending default resources. As a result, validation failures no longer occur when you use a wildcard to exclude cluster-scoped resources.

OADP-7897

VolumePolicy support for phase conditions to allow skipping PVCs

Before this update, OADP volume policies lacked the capability to evaluate and filter persistent volume claims (PVCs) based on their lifecycle status phase. As a consequence, backups could trigger unwanted warnings or errors for unbound or pending PVCs that you excluded, such as in scaled-down serverless environments.

With this update, volume policy actions have been improved to natively support the phase condition of PVCs. As a result, you can define custom volume policies in your ConfigMap object to automatically skip pending or unbound PVCs during backup operations. For example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: volume-policy
  namespace: velero
data:
  policy.yaml: |
    version: v1
    volumePolicies:
    - conditions:
        pvcPhase: ["Pending", "Lost"]
      action:
        type: skip

OADP-6428

1.2.1.1.2. Known issues
CloudStorage bucket creation fails on Google Cloud with WIF authentication

When you configure CloudStorage on OpenShift clusters on Google Cloud by using Workload Identity Federation (WIF) authentication, the CloudStorage resource fails to detect that a Google Cloud bucket does not exist. As a consequence, the resource returns a BucketCheckError condition and prevents automatic bucket creation.

To work around this problem, use an existing Google Cloud bucket when configuring CloudStorage on OpenShift clusters on Google Cloud using Workload Identity Federation (WIF) authentication. As a result, the controller skips bucket creation and proceeds with the existing bucket for backup storage.

OADP-8085

Non-admin backups of ODF encrypted volumes fail with OADP

When the OADP Data Mover backs up an encrypted Red Hat OpenShift Data Foundation (ODF) volume, it creates a temporary persistent volume claim (PVC) in the openshift-adp namespace. The ceph-csi driver then attempts to fetch the ceph-csi-kms-token secret from the openshift-adp namespace instead of the original application namespace that contains the secret. As a consequence, the NonAdminBackup custom resource (CR) fails with an error similar to the following output:

openshift-adp   2h11m       Warning   ProvisioningFailed     persistentvolumeclaim/NAMESPACE-rh-cf00000c-0000-0000-0000-22abf391t6dr   failed to provision volume with StorageClass "ocs-storagecluster-ceph-rbd-encrypted": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed fetching token from openshift-adp/ceph-csi-kms-token: secrets "ceph-csi-kms-token" not found

To work around this problem, copy ceph-csi-kms-token and the associated Key Management Service (KMS) config map from the application namespace to openshift-adp before running the backup. As a result, the ceph-csi driver can locate the encryption credentials and the backup completes successfully.

OADP-7972

1.2.1.2. OADP 1.6.0 release notes

Review new features and enhancements, resolved issues, known issues, Technology Preview features, and removed features in OpenShift API for Data Protection (OADP) 1.6.0.

1.2.1.2.1. New features
Parallel backup processing

OADP supports parallel backup processing. Before this update, OADP processed backups sequentially where each backup had to complete before the next one could begin. This created bottlenecks for environments with many backup schedules or large-scale data protection needs, increasing the overall time required to complete all backup operations. With this release, multiple backups can run in parallel if they do not share namespaces. As a result, you can now define more granular or frequent backup schedules without worrying about queuing delays.

OADP-6579

File-level restore with VMFR from KubeVirt virtual machine backups

OADP introduces the virtual machine file restore (VMFR) feature. Before this release, restoring even a single file from a KubeVirt VM backup required restoring the entire VM, which was time-consuming and resource-intensive. With this release, you can selectively recover individual files from KubeVirt VM backups directly within Red Hat OpenShift Service on AWS classic architecture. VMFR also includes RestoreItemAction, a plugin for collision prevention in multi-backup scenarios. This plugin renames PVCs during restore, ensuring unique names and avoiding conflicts, thereby improving the reliability and efficiency of VMFR backups and restores.

OADP-6864

File-level backups with VMDP in OpenShift Virtualization environments

OADP introduces the virtual machine data protection (VMDP) feature, a command-line tool that runs inside virtual machines (VMs). Before this update, OADP supported only snapshot-based backup of entire VMs, with no option for granular, user-driven file-level data protection from within a VM. Using VMDP, you can selectively back up and restore individual files and directories by using a Kopia-based client/server architecture without cluster administrator intervention.

OADP-6583

OADP CLI plugin

OADP includes a kubectl command-line interface (CLI) plugin that provides a kubectl native interface for managing backups and restores. Before this update, you aliased to the Velero CLI to perform backup operations. Non-cluster administrators had no way to independently manage their own backups, creating a dependency on cluster admins for routine data protection tasks. As a result, admins can run kubectl oadp commands to create, delete, and retrieve logs for backups and restores. Non-cluster admins can independently create, delete, and inspect non-admin backups within their permitted namespaces.

OADP-6544

OADP must-gather collects VM file restore CRDs

With this enhancement, the oadp-must-gather tool collects the VirtualMachineBackupsDiscovery and VirtualMachineFileRestore custom resource definitions (CRDs) introduced in OADP 1.6 for the virtual machine file-level restore feature. As a result, when you run the oc adm must-gather command, both new resource types are automatically collected cluster-wide, including CRDs, per-namespace CR YAMLs, and dedicated sections in the markdown summary. This gives support teams and administrators full visibility into VM file restore operations.

OADP-7872

Backup and restore of custom SCC through ClusterRole and ClusterRoleBinding

OADP supports backup and restore of custom security context constraints (SCC) with ServiceAccounts through ClusterRole and ClusterRoleBinding resources, not only those listed in the SCC user fields. This enhancement simplifies the backup and restore process of custom SCCs, as it no longer relies on the user field of the SCC. Instead, it uses the location of the SCC used by a pod and the service account associated with the pod. This results in a more reliable and secure backup and restore process for Red Hat OpenShift Service on AWS classic architecture clusters, essential for IBM Cloud Pak for Data.

OADP-7513

VolumePolicy support for bypassing pending PVCs during backups

With this update, you can bypass unbound persistent volume claims (PVCs) during backups. This enhancement improves backup success rates, especially when PVCs are in a Pending state. Additionally, it expands the capabilities of volume policy actions, offering greater control and flexibility in managing backups.

OADP-6739

Optimized Data Mover performance in containerized environments with new Go version

With the migration to Go 1.25, the Data Mover defaults the MaxParallelFileReads value to the cgroup CPU bandwidth limit corresponding to the container CPU limit rather than the total number of CPU cores on the node. Before this update, MaxParallelFileReads used the full CPU count of the node unless datamoverConfig.ParallelFilesUpload was explicitly set in the node-agent-config map, which could lead to overuse of resources in constrained container environments. With this update, Data Mover operations automatically respect container CPU limits, improving resource efficiency and stability without requiring manual configuration changes.

OADP-6735

Custom labels and annotations for repository maintenance job pods

Before this update, maintenance job pods did not support custom labels, which made it difficult to apply namespace-level network policies or firewall rules that rely on pod labels. With this release, you can configure custom podLabels and podAnnotations for repository maintenance jobs through the repositoryMaintenance section of the DataProtectionApplication custom resource.

OADP-6734

Prometheus-based monitoring for maintenance jobs

With this update, you can monitor maintenance job success and failures more efficiently using Prometheus metrics. This enhancement replaces the current method of tracking job success by using the --keep-latest-maintenance-jobs argument. The new solution provides a unified approach for monitoring various aspects of the backup process, using existing tools for alerts and notifications. This improvement reduces the risk of performance degradation due to job failures and helps you set up alerts for maintenance job issues, improving overall backup management and system health.

OADP-6733

Improved error messaging for ConfigMap validation in the Velero CLI

Before this update, ConfigMap validation failures did not return meaningful errors, making it difficult to diagnose and resolve configuration issues. With this release, the Velero command-line interface (CLI) displays detailed error messages when ConfigMap validation fails for backup repository, node agent, and backup repository maintenance job configurations. As a result, you can quickly identify and fix invalid ConfigMap configurations with actionable error messages.

OADP-6732

Improved diagnostics for Data Mover expose failures

The Data Mover expose diagnostic includes events from backup pods, restore pods, backup persistent volume claims (PVCs), and restore PVCs for both CSI snapshot data movement and file system backup. Before this update, informative messages from these resources were only available in their individual events and were not captured in the diagnostic output. As a result, you can get a more complete picture of expose failures directly from the diagnostic, without manually inspecting events on each related resource.

OADP-6731

VolumeSnapshotClass no longer required for CSI backup and restore

The Container Storage Interface (CSI) backup and restore process no longer includes or requires the VolumeSnapshotClass resource. Before this update, the VolumeSnapshotClass was included in CSI backups and needed to be present during restore, adding an unnecessary dependency. With this update, CSI snapshot-based backups and restores complete successfully without VolumeSnapshotClass in the cluster, simplifying the backup resource footprint and improving restore reliability in environments where the VolumeSnapshotClass might not be pre-configured.

OADP-6726

Configurable cache volumes for node agent pods

This release introduces configurable repository cache storage locations for the node agent Data Mover and pod volume restore operations. Before this update, backup engines such as Kopia stored metadata and snapshot caches implicitly within the ephemeral root file system of the pod. This often caused host node disk exhaustion, DiskPressure taints, and unexpected node evictions during heavy or concurrent operations.

With this release, administrators can isolate and route repository cache files to dedicated local volumes or dynamically provisioned persistent volume claims. As a result, cache volumes protect the system disk of the host node from running out of space and this increases the reliability of large-scale backups and restores.

OADP-6720

Repository maintenance jobs inherit tolerations from the Velero deployment

Repository maintenance jobs inherit tolerations defined in the DataProtectionApplication configuration. Before this update, maintenance pods did not inherit tolerations from the Velero deployment, which caused scheduling failures in clusters where nodes have taints configured. As a result, repository maintenance jobs can be scheduled correctly on tainted nodes, ensuring reliable backup repository maintenance in environments that use node taints and tolerations.

OADP-6694

Opt-in automatic space reclamation for restored PVCs on ODF clusters

You can enable automatic space reclamation on restored persistent volume claims (PVCs) by setting the oadp.openshift.io/reclaim-space-schedule annotation on the Restore custom resource. Before this update, after a Data Mover restore on a Red Hat OpenShift Data Foundation (ODF) cluster, you had to manually run rbd sparsify to reclaim unused block-device space. As a result, the PVC restore plugin propagates the schedule annotation to each restored PVC, and the Container Storage Interface controller automatically creates a ReclaimSpaceCronJob that handles sparsification without manual intervention. This annotation is opt-in and has no effect on non-ODF clusters.

OADP-6693

Scheduled backup queue no longer accumulates during extended blocking scenarios

Velero scheduled backups no longer accumulate in a New state queue when backup processing is blocked for extended periods, for example, during long-running backups, Velero server downtime, or infrastructure issues. Before this update, the schedule controller prevented new backup execution if any backup from that schedule was in a New or InProgress state, which led to indefinite queuing and backup accumulation. As a result, unreconciled backups, including those with an empty phase, are properly handled by the scheduler, preventing it from creating excessive new backup runs and ensuring reliable scheduled backup behavior.

OADP-6688

Added annotations to control cluster-autoscaler behavior

With this update, you can add annotations to Velero and node agent pods in OADP. This enhancement provides control over cluster-autoscaler behavior, preventing data operations from failing due to the interaction with the autoscaler. By adding annotations, users can mitigate an issue with OADP 1.5 and earlier, where data uploads and downloads can be canceled on node agent startup. This improvement ensures smoother operation and reduced downtime, and provides flexibility to customize the application according to specific requirements.

OADP-6360

Wildcard support for namespace selection during backup operations

You can use wildcard patterns when specifying namespaces in backup operations. Before this update, you had to explicitly list every namespace in the backup object and update it each time a new namespace was created. With this release, you can use patterns such as *-helm in the --include-namespaces flag to dynamically match multiple namespaces, simplifying backup management and reducing manual configuration overhead.

OADP-6176

Encryption of backup data with SSE-C for AWS S3 buckets

With this update, you can secure your backup data by using server side encryption with customer-provided keys (SSE-C) in OADP. This enhancement improves overall data protection and privacy. The feature is compatible with AWS S3 buckets. Additionally, it provides a workaround for using S3-compatible storage.

OADP-3971

Configurable priority class for node agent and Velero pods

With this update, you can configure the priorityClassName field for node agent and Velero pods in a DataProtectionApplication object, prioritizing these pods during resource contention. This is particularly useful after worker node outages, ensuring critical pods are scheduled first.

OADP-2785

1.2.1.2.2. Resolved issues
Velero backup with labelSelector no longer has degraded performance during resource discovery

Before this update, a performance regression in the resource discovery phase caused Velero to issue an excessive number of API list calls when a backup combined includedNamespaces: '*', includeClusterResources: true, labelSelector, and a large excludedResources list. As a consequence, backups that completed before in seconds took about 18 minutes to back up as few as 8 resources. With this release, the resource discovery logic is optimized so that unnecessary API list calls are eliminated during the item collection phase. As a result, backup performance is restored to expected levels, completing in seconds.

OADP-7977

CSI snapshot early frequent polling can be configured in the DPA CR

Before this update, a Velero change introduced 1-second Container Storage Interface (CSI) snapshot readiness polling for the first 10 seconds of every backup, designed specifically for Microsoft Volume Shadow Copy Service (VSS) workloads on Windows. OADP did not expose this behavior as a configurable option in the DataProtectionApplication (DPA) custom resource (CR). As a consequence, all CSI-enabled backups used the earlier polling behavior regardless of whether VSS compatibility was needed, and you had no way to disable or control this behavior through OADP. This release adds a new enableCSISnapshotEarlyFrequentPolling field under configuration.velero in the DPA CR. By default, this feature is disabled. If your workloads require this frequent polling behavior, you must enable it.

OADP-7882

Controller-manager no longer enters a CrashLoopBackOff loop on single-node OpenShift clusters without Console CRD

Before this update, the controller-manager entered a CrashLoopBackOff loop due to unconditional interaction with a missing ConsoleCLIDownload custom resource definition (CRD) on clusters without Console capability. As a consequence, you experienced pod crashes on single-node OpenShift clusters due to the missing ConsoleCLIDownload CRD. With this release, the OADP Operator skips CLI download setup when the Console CRD is not available. As a result, the controller-manager no longer enters a CrashLoopBackOff loop on single-node OpenShift clusters, improving its stability.

OADP-7868

DPA no longer reports an error when the podConfig value is empty and nodeAgent affinity is specified

Before this update, DataProtectionApplication (DPA) custom resource reported an error due to an empty podConfig with specified nodeAgent affinity. This caused DPA to fail to reconcile. With this release, DPA reports error only when nodeAgent.podConfig.nodeSelector does not match spec.configuration.nodeAgent.LoadAffinityConfig when podConfig is explicitly set.

OADP-7673

Backup phase Queued and ReadyToStart no longer cause reconcile issues in nonAdminController

Before this update, the Queued and ReadyToStart backup phases in nonAdminController caused reconcile issues due to their unsupported status. With this release, the Queued and ReadyToStart no longer cause reconcile issues in nonAdminController.

OADP-7382

CSI VolumeGroupSnapshot restore no longer fails with Ceph RBD storage class

Before this update, restoring a VolumeGroupSnapshot (VGS) backed up from Ceph RBD storage failed with a cannot find the group snapshot for VolumeSnapshot error message. This error occurred because Velero deleted the VolumeGroupSnapshotContent after backup, leaving no matching resource during restore. Also the VolumeSnapshotClassName was deleted from restored VolumeSnapshotContent, preventing the Container Storage Interface (CSI) controller from authenticating to verify snapshots. With this update, Velero preserves the group snapshot handle as an annotation during backup, recreates a stub VolumeGroupSnapshotContent during restores, and restores the correct VolumeSnapshotClass reference. As a result, VGS restores with Ceph RBD complete successfully. Note that VGS is a Technology Preview feature.

OADP-7381

SCC map is synchronized in ServiceAccount

Before this update, the ServiceAccount backup plugin failed with a PartiallyFailed error due to unsynchronized access to the security context constraints (SCC) map. As a consequence, backup failures occurred, affecting data integrity in multiple namespaces. With this release, the SCC map is synchronized in the ServiceAccount plugin, preventing its crash during backup execution.

OADP-7132

PrepareQueueLength is present in DPA

The PrepareQueueLength field is a configuration setting for the node agent that limits the number of intermediate resources created during backup and restore processes. Before this update, PrepareQueueLength was missing in the LoadConcurrency field in the DataProtectionApplication (DPA) configuration. As a consequence, you encountered an error while configuring PrepareQueueLength, causing the configuration to be removed. With this release, the PrepareQueueLength field is added to LoadConcurrency in the DPA configuration.

OADP-7070

BackupRepository maintenance jobs no longer fail when the repository name exceeds 63 characters

Before this update, the BackupRepository name was used directly as a Kubernetes label value for repository maintenance jobs. As a consequence, when the BackupRepository name exceeded the 63-character Kubernetes label limit, the maintenance job failed to build, preventing scheduled repository maintenance from running. With this release, the BackupRepository name is hashed before being used as a label value, ensuring it always fits within the 63-character limit. As a result, repository maintenance jobs are created and completed successfully regardless of the BackupRepository name length.

OADP-6955

Restore transitions to Completed only after PodVolumeRestores completes

Before this update, restores were marked as completed prematurely before all PodVolumeRestores. With this release, restores complete only after PodVolumeRestores.

OADP-6911

Node agent pod restart no longer cancels DataUploads across the entire cluster

Before this update, when any node agent pod restarted, Data Mover canceled all DataUploads in Accepted status across all node agent pods, regardless of which pod was handling them. As a consequence, backup operations failed and the backup queue could be blocked for extended periods even when the remaining node agent pods were healthy. With this release, the controller only cancels DataUploads that were accepted by the specific node agent pod that restarted, leaving DataUploads on other nodes unaffected. As a result, backups handled by healthy node agent pods complete successfully and the backup queue is no longer blocked by an unrelated pod restart.

OADP-6896

Improved volume policy performance for clusters with many pods and PVCs

Before this update, the volume policy check required scanning every pod in the namespace for every individual persistent volume claim (PVC). As a consequence, backup performance degraded significantly in clusters with large numbers of pods and PVCs. With this release, a PVC-to-Pod cache is added to eliminate redundant pod listings and reduce iteration overhead. As a result, all PV snapshot checks complete within the same second, improving performance for backup operations that were taking excessively long.

OADP-6879

Custom labels and annotations are supported on DPA resources

Before this update, the DataProtectionApplication (DPA) custom resource did not support custom labels or annotations, and it was not possible to propagate them to child resources managed by the OADP Operator. As a consequence, it was not possible to exclude these child resources from external reconciliation tools at the DPA level, causing unnecessary reconciliation loops and high CPU usage. With this release, the DPA includes resourceLabels and resourceAnnotations fields that propagate user-defined labels and annotations to all Operator-managed resources. As a result, you can control reconciliation behavior for all managed resources from a single configuration point, reducing unnecessary processing and CPU usage.

OADP-6846

File system backup works on clusters requiring privileged pod security context

Before this update, Velero did not provide an option to run file system backup in a privileged security context. As a consequence, file system backups failed with permission errors on OpenShift clusters that require privileged security context to access mount paths. With this release, a new --privileged-fs-backup install and node agent server option was added, allowing file system backup pods to run in privileged mode independently of the existing node agent privileged flag. As a result, file system backups complete successfully on clusters that require privileged pod security context.

OADP-6764

OADP removes OVN-K and Multus CNI annotations during pod restore

Before this update, pod network annotations injected by OVN-K and Multus container network interface (CNI) were restored during backup. As a consequence, this caused conflicts with the CNI and required manual intervention. With this release, OADP removes OVN-K and Multus CNI annotations during pod restore.

OADP-6699

Kopia BackupRepository no longer displays wrong resticIdentifier path

Before this update, creating a backup with Kopia generated a BackupRepository resource that contained a resticIdentifier field referencing a Restic path instead of the actual Kopia repository path. As a consequence, this caused user confusion during troubleshooting. With this release, the resticIdentifier field is removed from Kopia-type BackupRepository resources, as it was only relevant for Restic repositories.

OADP-5829

AWS S3 bucket region is auto-detected in BackupStorageLocation configuration

Before this update, the AWS bucket region was missing in the DataProtectionApplication configuration when using S3 BackupStorageLocation (BSL) with snapshotLocations. As a consequence, backups failed due to missing region in AWS S3 bucket. With this release, AWS S3 bucket regions are automatically detected in BSL.

OADP-5777

BSL status message returns an error when an Azure storage bucket does not exist

Before this update, when a DataProtectionApplication was configured with a non-existing Azure storage bucket, the BackupStorageLocation (BSL) validation returned a raw HTTP response in the status.message field. As a consequence, the BSL status was unclear. With this release, the BSL status.message field returns a concise, human-readable error such as ContainerNotFound: The specified container does not exist.

OADP-5171

OADP Data Mover restore operations honor loadAffinity node selection

Before this update, the node agent loadAffinity setting was implemented only for backup operations, not for restore operations. As a consequence, when nodeSelector was configured in the DataProtectionApplication specification, the Data Mover pod could be scheduled to a node without a running node agent during a restore. This caused the restore to hang indefinitely. With this release, the loadAffinity configuration is applied to both backup and restore operations. As a result, Data Mover restore pods are correctly scheduled on nodes with running node agents, preventing restore hangs and PartiallyFailed errors.

OADP-5114

Kopia cache no longer persists on worker nodes after backup and restore operations

Before this update, during file system restore, the /var partition on worker nodes increased because the cache was not deleted after backup completion. Only a node agent restart cleared the cache. As a consequence, unintended disk space consumption occurred on worker nodes after file system restore. With this release, the cache for Kopia operation stored in ephemeral-storage is cleared after restore operation.

OADP-4855

Data Mover restore no longer fails when nodeSelector is used

Before this update, when a DataProtectionApplication (DPA) custom resource was created with the nodeSelector field set in the nodeAgent parameter, nodeSelector was not properly handled for the Data Mover restore action. As a consequence, Data Mover restore actions partially failed because they were scheduled on a wrong node. With this release, nodeSelector is correctly handled for Data Mover restore actions. As a result, restore operations complete successfully when the field is configured in the DPA.

OADP-4743

Velero CLI automatically uses the CA certificate from BackupStorageLocation

Before this update, running Velero containers did not store the certificate authority (CA) certificate as a file. As a consequence, you had to manually pass the --cacert flag with a path to a CA certificate file on every Velero command-line interface (CLI) command, or download the Velero binary and CA certificate bundle to your workstation. This made the experience inefficient in environments with custom or self-signed certificates.

With this release, the Velero CLI automatically reads the CA certificate from the BackupStorageLocation (BSL) configuration, and gracefully falls back to the caCert of the BSL if a --cacert flag is provided but points to an invalid path. As a result, you can run Velero CLI commands through the oc exec alias without manually specifying --cacert, significantly simplifying day-to-day backup and restore operations in environments that use custom CA certificates.

OADP-4668

Velero no longer restores unnecessary restore-wait init containers

Before this update, Velero did not check whether the restore-wait init container was necessary during a restore operation. As a consequence, when a namespace was backed up with CSI or native Data Mover and then was restored, Velero incorrectly injected the restore-wait init container into the application pod spec. With this release, Velero skips restoring the restore-wait init container when it is not required by the backup method used.

OADP-3759

The defaultVolumesToFSBackup flag matches defaultVolumesToFsBackup

Before this update, the dpa.spec.configuration.velero.defaultVolumesToFSBackup flag used different casing than the backup.spec.defaultVolumesToFsBackup flag. As a consequence, the inconsistency between the two flags could cause confusion when configuring pod volume file system backup settings. With this release, the two flags are identical.

OADP-3692

OADP CLI automatically inherits the CA certificate from BackupStorageLocation

Before this update, the OADP command-line interface (CLI) did not inherit the certificate authority (CA) certificate from the BackupStorageLocation (BSL) when retrieving backup logs. As a consequence, you were unable to retrieve backup logs by using oc oadp backup logs, even though the backup itself completed successfully and the DataProtectionApplication (DPA) custom resource was correctly configured with a valid CA certificate. With this release, the OADP CLI automatically uses the CA certificate from the BSL when retrieving backup logs. As a result, backup log retrieval works correctly when a valid CA certificate is present in the BSL.

OADP-3378

PVC no longer retains stale CSI snapshot labels after a file system restore

Before this update, during a Container Storage Interface (CSI) backup, Velero added a velero.io/volume-snapshot-name label to the persistent volume claim (PVC), and this label was not removed when next file system backup and restore was performed. With this release, PVCs no longer contain leftover velero.io/volume-snapshot-name labels after a successful file system restore.

OADP-3360

PodVolumeBackup and PodVolumeRestore CRs are marked as failed when the parent backup or restore fails

Before this update, when the backup or restore CR failed, the associated PodVolumeBackup and PodVolumeRestore custom resources (CRs) continued their data transfer. As a consequence, this led to an inconsistent state where individual volume CRs reported Completed while the parent operation was Failed. With this release, PodVolumeBackup and PodVolumeRestore CRs are properly canceled and marked as Failed when the parent backup or restore CR fails. As a result, volume-level CR statuses are consistent with the parent operation status.

OADP-3039

1.2.1.2.3. Known issues
CloudStorage bucket creation fails on Google Cloud with WIF authentication

The CloudStorage resource fails to detect that a Google Cloud bucket does not exist and returns a BucketCheckError condition, preventing automatic bucket creation.

To work around this problem, use an existing Google Cloud bucket when configuring CloudStorage on OpenShift clusters on Google Cloud using Workload Identity Federation (WIF) authentication. As a result, the controller skips bucket creation and proceeds with the existing bucket for backup storage.

OADP-8085

Non-admin backups of ODF encrypted volumes fail with OADP

When OADP Data Mover backs up an encrypted Red Hat OpenShift Data Foundation (ODF) volume, it creates a temporary persistent volume claim (PVC) in the openshift-adp namespace. The ceph-csi driver then attempts to fetch the ceph-csi-kms-token secret from openshift-adp instead of the original application namespace that has the secret. As a consequence, the NonAdminBackup fails with an error similar to the following:

openshift-adp   2h11m       Warning   ProvisioningFailed     persistentvolumeclaim/NAMESPACE-rh-cf00000c-0000-0000-0000-22abf391t6dr   failed to provision volume with StorageClass "ocs-storagecluster-ceph-rbd-encrypted": rpc error: code = InvalidArgument desc = invalid encryption kms configuration: failed fetching token from openshift-adp/ceph-csi-kms-token: secrets "ceph-csi-kms-token" not found

To work around this problem, copy ceph-csi-kms-token and the associated Key Management Service (KMS) config map from the application namespace to openshift-adp before running the backup. As a result, the ceph-csi driver can locate the encryption credentials and the backup completes successfully.

OADP-7972

The Velero server might be terminated with an OOMKilled event when accessing large Kopia repositories

When the Velero server accesses large Kopia repositories, for example, during backup deletion, the Velero server might be terminated by the Out Of Memory (OOM) process. This is typically due to high memory usage loading many Kopia index blobs and related repository metadata.

To work around this problem, perform the following actions:

  1. Check Velero logs around the OOMKilled event for repository-open or index-loading activity, and note the approximate maximum memory usage observed for the Velero pod (from oc or kubectl top pods or similar).
  2. Increase the Velero container memory request or limit in spec.velero.podConfig in the DataProtectionApplication custom resource to exceed that observed peak.
  3. Redeploy the Velero pod until OOMKilled events no longer occur.
  4. Record the final stable memory values for your environment for future sizing guidance.

As a result, the Velero server has enough memory to load the repository metadata without being terminated.

OADP-6736

1.2.1.2.4. Technology Preview features
Incremental QCOW2 backups for KubeVirt VMs (Technology Preview)

OADP introduces storage agnostic QCOW2 incremental backup for KubeVirt virtual machines (VMs) as a Technology Preview feature. This feature provides a lightweight alternative to traditional VolumeSnapshot-based backups by using libvirt tools to perform incremental QCOW2 backups. After an initial full backup, later backups capture only changed blocks, improving backup performance and reducing resource usage compared to Container Storage Interface (CSI) snapshot methods. This enables more frequent backup schedules with minimal impact on running VM performance.

To use this feature, enable QEMU backup on VM disks (CBT = true) and set up the volume policy to virt for target volumes.

OADP-6584

VolumeGroupSnapshot (Technology Preview)

With VolumeGroupSnapshot (VGS) you can group and snapshot multiple persistent volume claims (PVCs) together using label-based selection. This ensures point-in-time data consistency across related volumes, improving backup reliability for stateful applications that use multiple persistent volumes. VGS is a Technology Preview feature.

OADP-6581

1.2.1.2.5. Removed features
Restic backup is removed

Creating file system backups with Restic as an uploader type is removed and is no longer supported. Setting uploaderType: restic in spec.configuration.nodeAgent.uploaderType in the DataProtectionApplication (DPA) custom resource (CR) results in a reconcile error. You can create file system backups by using Kopia instead.

Note that restoring existing Restic backups remains supported. To continue restoring Restic backups, configure Kopia in the DPA CR. OADP then uses Restic to restore Restic backups.

OADP-8091

velero-restore-helper config map is removed

The fs-restore-action-config config map, which pointed to a separate velero-restore-helper container image for file system restore operations, is removed and is no longer supported. You can restore workloads by using the built-in restore helper binary included in the Velero image instead.

OADP-5906

Changing PVC selected-node is removed

The Changing PVC selected-node feature, which allowed overriding the volume.kubernetes.io/selected-node annotation to change the node selected for a persistent volume claim (PVC) during restore, is removed and is no longer supported. You can ensure PVCs are scheduled to the correct node by using the native Kubernetes WaitForFirstConsumer volume binding mode instead.

OADP-6729

1.2.2. Upgrading OADP 1.5 to 1.6

Learn how to upgrade your existing OpenShift API for Data Protection 1.5 installation to OADP 1.6.

Note

Always upgrade to the next minor version. Do not skip versions. To update to a later version, upgrade only one channel at a time. For example, to upgrade from OADP 1.1 to 1.3, upgrade first to 1.2, and then to 1.3.

1.2.2.1. Changes from OADP 1.5 to 1.6

The Velero server has been updated from version 1.16 to 1.18.

This brings the following changes:

Parallel backup processing
OADP supports parallel backup processing. Before this update, OADP processed backups sequentially where each backup had to complete before the next one could begin. This created bottlenecks for environments with many backup schedules or large-scale data protection needs, increasing the overall time required to complete all backup operations. With this release, multiple backups can run in parallel if they do not share namespaces. As a result, you can now define more granular or frequent backup schedules without worrying about queuing delays.
File-level restore with VMFR from KubeVirt virtual machine backups
OADP introduces the virtual machine file restore (VMFR) feature. Before this release, restoring even a single file from a KubeVirt VM backup required restoring the entire VM, which was time-consuming and resource-intensive. With this release, you can selectively recover individual files from KubeVirt VM backups directly within Red Hat OpenShift Service on AWS classic architecture. VMFR also includes RestoreItemAction, a plugin for collision prevention in multi-backup scenarios. This plugin renames PVCs during restore, ensuring unique names and avoiding conflicts, thereby improving the reliability and efficiency of VMFR backups and restores.
File-level backups with VMDP in OpenShift Virtualization environments
OADP introduces the virtual machine data protection (VMDP) feature, a command-line tool that runs inside virtual machines (VMs). Before this update, OADP supported only snapshot-based backup of entire VMs, with no option for granular, user-driven file-level data protection from within a VM. Using VMDP, you can selectively back up and restore individual files and directories by using a Kopia-based client/server architecture without cluster administrator intervention.
OADP CLI plugin
OADP includes a kubectl command-line interface (CLI) plugin that provides a kubectl native interface for managing backups and restores. Before this update, you aliased to the Velero CLI to perform backup operations. Non-cluster administrators had no way to independently manage their own backups, creating a dependency on cluster admins for routine data protection tasks. As a result, admins can run kubectl oadp commands to create, delete, and retrieve logs for backups and restores. Non-cluster admins can independently create, delete, and inspect non-admin backups within their permitted namespaces.
Configurable priority class for node agent and Velero pods
With this update, you can configure the priorityClassName field for node agent and Velero pods in a DataProtectionApplication object, prioritizing these pods during resource contention. This is particularly useful after worker node outages, ensuring critical pods are scheduled first.
Wildcard support for namespace selection during backup operations
You can use wildcard patterns when specifying namespaces in backup operations. Before this update, you had to explicitly list every namespace in the backup object and update it each time a new namespace was created. With this release, you can use patterns such as *-helm in the --include-namespaces flag to dynamically match multiple namespaces, simplifying backup management and reducing manual configuration overhead.
VolumeSnapshotClass no longer required for CSI backup and restore
The Container Storage Interface (CSI) backup and restore process no longer includes or requires the VolumeSnapshotClass resource. Before this update, the VolumeSnapshotClass was included in CSI backups and needed to be present during restore, adding an unnecessary dependency. With this update, CSI snapshot-based backups and restores complete successfully without VolumeSnapshotClass in the cluster, simplifying the backup resource footprint and improving restore reliability in environments where the VolumeSnapshotClass might not be pre-configured.
Optimized Data Mover performance in containerized environments with new Go version
With the migration to Go 1.25, the Data Mover defaults the MaxParallelFileReads value to the cgroup CPU bandwidth limit corresponding to the container CPU limit rather than the total number of CPU cores on the node. Before this update, MaxParallelFileReads used the full CPU count of the node unless datamoverConfig.ParallelFilesUpload was explicitly set in the node-agent-config map, which could lead to overuse of resources in constrained container environments. With this update, Data Mover operations automatically respect container CPU limits, improving resource efficiency and stability without requiring manual configuration changes.
Changing PVC selected-node is removed
The Changing PVC selected-node feature, which allowed overriding the volume.kubernetes.io/selected-node annotation to change the node selected for a persistent volume claim (PVC) during restore, is removed and is no longer supported. You can ensure PVCs are scheduled to the correct node by using the native Kubernetes WaitForFirstConsumer volume binding mode instead.

1.2.2.2. Backing up the DPA configuration

You must back up your current DataProtectionApplication (DPA) configuration.

Procedure

  • Save your current DPA configuration by running the following command:

    Example command

    $ oc get dpa -n openshift-adp -o yaml > dpa.orig.backup

1.2.2.3. Upgrading the OADP Operator

You can upgrade the OpenShift API for Data Protection (OADP) Operator by using the following procedure.

Note

Do not install OADP 1.6.0 on a OpenShift 4.21 cluster.

Prerequisites

  • You have installed the latest OADP 1.5.5.
  • You have backed up your data.

Procedure

  1. Upgrade OpenShift 4.21 to OpenShift 4.22.
  2. Ensure your subscription channel for the OADP Operator is stable.
  3. Wait for the Operator and containers to update and restart.

OpenShift API for Data Protection (OADP) 1.5 is not supported on OpenShift 4.22. You can convert OADP to the new 1.6 version by using the new spec.configuration.nodeAgent field and its sub-fields.

Procedure

  1. To configure nodeAgent daemon set, use the spec.configuration.nodeAgent parameter in DPA. See the following example:

    Example DataProtectionApplication configuration

    ...
     spec:
       configuration:
         nodeAgent:
           enable: true
           uploaderType: kopia
    ...

  2. To configure nodeAgent daemon set by using the ConfigMap resource named node-agent-config, see the following example configuration:

    Example config map

    ...
     spec:
       configuration:
         nodeAgent:
           backupPVC:
             ...
           loadConcurrency:
             ...
           podResources:
             ...
           restorePVC:
            ...
    ...

1.2.2.5. Verifying the upgrade

You can verify the OpenShift API for Data Protection (OADP) upgrade by using the following procedure.

Procedure

  1. Verify that the DataProtectionApplication (DPA) has been reconciled successfully:

    $ oc get dpa dpa-sample -n openshift-adp

    Example output

    NAME            RECONCILED   AGE
    dpa-sample      True         2m51s

    Note

    The RECONCILED column must be True.

  2. Verify that the installation finished by viewing the OADP resources by running the following command:

    $ oc get all -n openshift-adp

    Example output

    NAME                                                    READY   STATUS    RESTARTS   AGE
    pod/node-agent-9pjz9                                    1/1     Running   0          3d17h
    pod/node-agent-fmn84                                    1/1     Running   0          3d17h
    pod/node-agent-xw2dg                                    1/1     Running   0          3d17h
    pod/openshift-adp-controller-manager-76b8bc8d7b-kgkcw   1/1     Running   0          3d17h
    pod/velero-64475b8c5b-nh2qc                             1/1     Running   0          3d17h
    
    NAME                                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
    service/openshift-adp-controller-manager-metrics-service   ClusterIP   172.30.194.192   <none>        8443/TCP   3d17h
    service/openshift-adp-velero-metrics-svc                   ClusterIP   172.30.190.174   <none>        8085/TCP   3d17h
    
    NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
    daemonset.apps/node-agent   3         3         3       3            3           <none>          3d17h
    
    NAME                                               READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/openshift-adp-controller-manager   1/1     1            1           3d17h
    deployment.apps/velero                             1/1     1            1           3d17h
    
    NAME                                                          DESIRED   CURRENT   READY   AGE
    replicaset.apps/openshift-adp-controller-manager-76b8bc8d7b   1         1         1       3d17h
    replicaset.apps/openshift-adp-controller-manager-85fff975b8   0         0         0       3d17h
    replicaset.apps/velero-64475b8c5b                             1         1         1       3d17h
    replicaset.apps/velero-8b5bc54fd                              0         0         0       3d17h
    replicaset.apps/velero-f5c9ffb66                              0         0         0       3d17h

  3. Verify the backup storage location and confirm that the PHASE is Available by running the following command:

    $ oc get backupstoragelocations.velero.io -n openshift-adp

    Example output

    NAME           PHASE       LAST VALIDATED   AGE     DEFAULT
    dpa-sample-1   Available   1s               3d16h   true

1.3. OADP performance

1.4. OADP features and plugins

Review OpenShift API for Data Protection (OADP) features and default plugins that integrate Velero with cloud providers to back up and restore Red Hat OpenShift Service on AWS classic architecture resources. This helps you to select the right plugins and features for your backup and restore environment.

1.4.1. OADP features

Review the backup, restore, and scheduling features of OpenShift API for Data Protection (OADP) for protecting applications on Red Hat OpenShift Service on AWS classic architecture. This helps you to understand the available capabilities for your data protection strategy.

Backup

You can use OADP to back up all applications on the OpenShift Platform, or you can filter the resources by type, namespace, or label.

OADP backs up Kubernetes objects and internal images by saving them as an archive file on object storage. OADP backs up persistent volumes (PVs) by creating snapshots with the native cloud snapshot API or with the Container Storage Interface (CSI). For cloud providers that do not support snapshots, OADP backs up resources and PV data with Restic.

Note

You must exclude Operators from the backup of an application for backup and restore to succeed.

Restore

You can restore resources and PVs from a backup. You can restore all objects in a backup or filter the objects by namespace, PV, or label.

Note

You must exclude Operators from the backup of an application for backup and restore to succeed.

Schedule
You can schedule backups at specified intervals.
Hooks
You can use hooks to run commands in a container on a pod, for example, fsfreeze to freeze a file system. You can configure a hook to run before or after a backup or restore. Restore hooks can run in an init container or in the application container.

1.4.2. OADP plugins

Review the default Velero plugins provided by OpenShift API for Data Protection (OADP) that integrate with storage providers to support backup and snapshot operations. This helps you to select and configure the right plugins for your cloud environment.

OADP also provides plugins for Red Hat OpenShift Service on AWS classic architecture resource backups, OpenShift Virtualization resource backups, and Container Storage Interface (CSI) snapshots.

Expand
Table 1.3. OADP plugins
OADP pluginFunctionStorage location

aws

Backs up and restores Kubernetes objects.

AWS S3

Backs up and restores volumes with snapshots.

AWS EBS

openshift

Backs up and restores Red Hat OpenShift Service on AWS classic architecture resources. [1]

Object store

kubevirt

Backs up and restores OpenShift Virtualization resources. [2]

Object store

csi

Backs up and restores volumes with CSI snapshots. [3]

Cloud storage that supports CSI snapshots

hypershift

Backs up and restores HyperShift hosted cluster resources. [4]

Object store

  1. Mandatory.
  2. Virtual machine disks are backed up with CSI snapshots or Restic.
  3. The csi plugin uses the Kubernetes CSI snapshot API.

    • OADP 1.1 or later uses snapshot.storage.k8s.io/v1
    • OADP 1.0 uses snapshot.storage.k8s.io/v1beta1
  4. Do not add the hypershift plugin in the DataProtectionApplication custom resource if the cluster is not a HyperShift hosted cluster.

1.4.3. About OADP Velero plugins

Review how to configure default cloud provider plugins or install custom plugins during the OADP deployment to connect your specific storage solutions. This helps you to successfully back up and restore resources across your environments.

1.4.3.1. Default Velero cloud provider plugins

You can install any of the following default Velero cloud provider plugins when you configure the oadp_v1alpha1_dpa.yaml file during deployment:

  • aws (Amazon Web Services)
  • openshift (OpenShift Velero plugin)
  • csi (Container Storage Interface)
  • kubevirt (KubeVirt)

You specify the desired default plugins in the oadp_v1alpha1_dpa.yaml file during deployment.

The following .yaml file installs the openshift, aws, azure, and gcp plugins:

 apiVersion: oadp.openshift.io/v1alpha1
 kind: DataProtectionApplication
 metadata:
   name: dpa-sample
 spec:
   configuration:
     velero:
       defaultPlugins:
       - openshift
       - aws
       - azure
       - gcp

1.4.3.2. Custom Velero plugins

You can install a custom Velero plugin by specifying the plugin image and name when you configure the oadp_v1alpha1_dpa.yaml file during deployment.

You specify the desired custom plugins in the oadp_v1alpha1_dpa.yaml file during deployment.

The following .yaml file installs the default openshift, azure, and gcp plugins and a custom plugin that has the name custom-plugin-example and the image quay.io/example-repo/custom-velero-plugin:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
 name: dpa-sample
spec:
 configuration:
   velero:
     defaultPlugins:
     - openshift
     - azure
     - gcp
     customPlugins:
     - name: custom-plugin-example
       image: quay.io/example-repo/custom-velero-plugin

1.4.4. OADP and FIPS

Federal Information Processing Standards (FIPS) are a set of computer security standards developed by the United States federal government inline with the Federal Information Security Management Act (FISMA).

OpenShift API for Data Protection (OADP) has been tested and works on FIPS-enabled Red Hat OpenShift Service on AWS classic architecture clusters.

1.4.5. Avoiding the Velero plugin panic error

Label a custom Backup Storage Location (BSL) to resolve Velero plugin panic errors during imagestream backups. This helps you to ensure the OADP controller creates the required registry secret when you manage the BSL outside the DataProtectionApplication (DPA) CR.

A missing secret can cause a panic error for the Velero plugin during image stream backups. When the backup and the BSL are managed outside the scope of the DPA, the OADP controller does not create the relevant oadp-<bsl_name>-<bsl_provider>-registry-secret parameter.

During the backup operation, the OpenShift Velero plugin panics on the imagestream backup, with the following panic error:

024-02-27T10:46:50.028951744Z time="2024-02-27T10:46:50Z" level=error msg="Error backing up item"
backup=openshift-adp/<backup name> error="error executing custom action (groupResource=imagestreams.image.openshift.io,
namespace=<BSL Name>, name=postgres): rpc error: code = Aborted desc = plugin panicked:
runtime error: index out of range with length 1, stack trace: goroutine 94…

Procedure

  1. Label the custom BSL with the relevant label by using the following command:

    $ oc label backupstoragelocations.velero.io <bsl_name> app.kubernetes.io/component=bsl
  2. After the BSL is labeled, wait until the DPA reconciles.

    Note

    You can force the reconciliation by making any minor change to the DPA itself.

Verification

  • After the DPA is reconciled, confirm that the parameter has been created and that the correct registry data has been populated into it by entering the following command:

    $ oc -n openshift-adp get secret/oadp-<bsl_name>-<bsl_provider>-registry-secret -o json | jq -r '.data'

Define either velero or cloudstorage in your Data Protection Application (DPA) configuration to prevent indefinite pod crashes. This configuration resolves a segmentation fault in the openshift-adp-controller-manager pod that occurs when both components are enabled.

The openshift-adp-controller-manager pod fails with a crash loop segmentation fault due to the following settings:

  • If you define both velero and cloudstorage, the openshift-adp-controller-manager fails.
  • If you do not define both velero and cloudstorage, the openshift-adp-controller-manager fails.

See OADP-1054 for more information.

1.5. OADP use cases

To back up and restore workloads on Red Hat OpenShift Service on AWS classic architecture, you can use OADP. You can create a backup of a workload, restore it from the backup, and verify the restoration. You can also clean up the OADP Operator, backup storage, and AWS resources when they are no longer needed.

Perform a backup by using OpenShift API for Data Protection (OADP) with Red Hat OpenShift Service on AWS classic architecture. The following example hello-world application has no persistent volumes (PVs) attached.

Either Data Protection Application (DPA) configuration will work.

Procedure

  1. Create a workload to back up by running the following commands:

    $ oc create namespace hello-world
    $ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
  2. Expose the route by running the following command:

    $ oc expose service/hello-openshift -n hello-world
  3. Check that the application is working by running the following command:

    $ curl `oc get route/hello-openshift -n hello-world -o jsonpath='{.spec.host}'`

    You should see an output similar to the following example:

    Hello OpenShift!
  4. Back up the workload by running the following command:

    $ cat << EOF | oc create -f -
      apiVersion: velero.io/v1
      kind: Backup
      metadata:
        name: hello-world
        namespace: openshift-adp
      spec:
        includedNamespaces:
        - hello-world
        storageLocation: ${CLUSTER_NAME}-dpa-1
        ttl: 720h0m0s
    EOF
  5. Wait until the backup is complete, and then run the following command:

    $ watch "oc -n openshift-adp get backup hello-world -o json | jq .status"

    You should see an output similar to the following example:

    {
      "completionTimestamp": "2022-09-07T22:20:44Z",
      "expiration": "2022-10-07T22:20:22Z",
      "formatVersion": "1.1.0",
      "phase": "Completed",
      "progress": {
        "itemsBackedUp": 58,
        "totalItems": 58
      },
      "startTimestamp": "2022-09-07T22:20:22Z",
      "version": 1
    }
  6. Delete the demo workload by running the following command:

    $ oc delete ns hello-world
  7. Restore the workload from the backup by running the following command:

    $ cat << EOF | oc create -f -
      apiVersion: velero.io/v1
      kind: Restore
      metadata:
        name: hello-world
        namespace: openshift-adp
      spec:
        backupName: hello-world
    EOF
  8. Wait for the Restore to finish by running the following command:

    $ watch "oc -n openshift-adp get restore hello-world -o json | jq .status"

    You should see an output similar to the following example:

    {
      "completionTimestamp": "2022-09-07T22:25:47Z",
      "phase": "Completed",
      "progress": {
        "itemsRestored": 38,
        "totalItems": 38
      },
      "startTimestamp": "2022-09-07T22:25:28Z",
      "warnings": 9
    }
  9. Check that the workload is restored by running the following command:

    $ oc -n hello-world get pods

    You should see an output similar to the following example:

    NAME                              READY   STATUS    RESTARTS   AGE
    hello-openshift-9f885f7c6-kdjpj   1/1     Running   0          90s
  10. Check the JSONPath by running the following command:

    $ curl `oc get route/hello-openshift -n hello-world -o jsonpath='{.spec.host}'`

    You should see an output similar to the following example:

    Hello OpenShift!
    Note

    For troubleshooting tips, see the troubleshooting documentation.

Uninstall the OpenShift API for Data Protection (OADP) Operator together with the backups and the S3 bucket from the hello-world example.

Procedure

  1. Delete the workload by running the following command:

    $ oc delete ns hello-world
  2. Delete the Data Protection Application (DPA) by running the following command:

    $ oc -n openshift-adp delete dpa ${CLUSTER_NAME}-dpa
  3. Delete the cloud storage by running the following command:

    $ oc -n openshift-adp delete cloudstorage ${CLUSTER_NAME}-oadp
    Warning

    If this command hangs, you might need to delete the finalizer by running the following command:

    $ oc -n openshift-adp patch cloudstorage ${CLUSTER_NAME}-oadp -p '{"metadata":{"finalizers":null}}' --type=merge
  4. If the Operator is no longer required, remove it by running the following command:

    $ oc -n openshift-adp delete subscription oadp-operator
  5. Remove the namespace from the Operator:

    $ oc delete ns openshift-adp
  6. If the backup and restore resources are no longer required, remove them from the cluster by running the following command:

    $ oc delete backups.velero.io hello-world
  7. To delete backup, restore and remote objects in AWS S3 run the following command:

    $ velero backup delete hello-world
  8. If you no longer need the Custom Resource Definitions (CRD), remove them from the cluster by running the following command:

    $ for CRD in `oc get crds | grep velero | awk '{print $1}'`; do oc delete crd $CRD; done
  9. Delete the AWS S3 bucket by running the following commands:

    $ aws s3 rm s3://${CLUSTER_NAME}-oadp --recursive
    $ aws s3api delete-bucket --bucket ${CLUSTER_NAME}-oadp
  10. Detach the policy from the role by running the following command:

    $ aws iam detach-role-policy --role-name "${ROLE_NAME}"  --policy-arn "${POLICY_ARN}"
  11. Delete the role by running the following command:

    $ aws iam delete-role --role-name "${ROLE_NAME}"

Following is a use case for using OADP to restore a backup to a different namespace.

Restore a backup of an application by using OADP to a new target namespace, test-restore-application. To restore a backup, you create a restore custom resource (CR) as shown in the following example. In the restore CR, the source namespace refers to the application namespace that you included in the backup. You then verify the restore by changing your project to the new restored namespace and verifying the resources.

Prerequisites

  • You installed the OADP Operator.
  • You have the backup of an application to be restored.

Procedure

  1. Create a restore CR as shown in the following example:

    apiVersion: velero.io/v1
    kind: Restore
    metadata:
      name: test-restore
      namespace: openshift-adp
    spec:
      backupName: <backup_name>
      restorePVs: true
      namespaceMapping:
        <application_namespace>: test-restore-application

    where:

    test-restore
    Specifies the name of the restore CR.
    <backup_name>
    Specifies the name of the backup.
    <application_namespace>
    Specifies the target namespace to restore to. namespaceMapping maps the source application namespace to the target application namespace. test-restore-application is the name of target namespace where you want to restore the backup.
  2. Apply the restore CR by running the following command:

    $ oc apply -f <restore_cr_filename>

Verification

  1. Verify that the restore is in the Completed phase by running the following command:

    $ oc describe restores.velero.io <restore_name> -n openshift-adp
  2. Change to the restored namespace test-restore-application by running the following command:

    $ oc project test-restore-application
  3. Verify the restored resources such as persistent volume claim (pvc), service (svc), deployment, secret, and config map by running the following command:

    $ oc get pvc,svc,deployment,secret,configmap

    Example output

    NAME                          STATUS   VOLUME
    persistentvolumeclaim/mysql   Bound    pvc-9b3583db-...-14b86
    
    NAME               TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
    service/mysql      ClusterIP   172....157     <none>        3306/TCP   2m56s
    service/todolist   ClusterIP   172.....15     <none>        8000/TCP   2m56s
    
    NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/mysql   0/1     1            0           2m55s
    
    NAME                                         TYPE                      DATA   AGE
    secret/builder-dockercfg-6bfmd               kubernetes.io/dockercfg   1      2m57s
    secret/default-dockercfg-hz9kz               kubernetes.io/dockercfg   1      2m57s
    secret/deployer-dockercfg-86cvd              kubernetes.io/dockercfg   1      2m57s
    secret/mysql-persistent-sa-dockercfg-rgp9b   kubernetes.io/dockercfg   1      2m57s
    
    NAME                                 DATA   AGE
    configmap/kube-root-ca.crt           1      2m57s
    configmap/openshift-service-ca.crt   1      2m57s

1.6. Installing and configuring OADP

1.6.1. Installing OADP

Use OpenShift API for Data Protection (OADP) with Red Hat OpenShift Service on AWS classic architecture clusters to back up and restore application data.

Before installing OpenShift API for Data Protection (OADP), you must set up role and policy credentials for OADP so that it can use the Amazon Web Services API.

This process is performed in the following two stages:

  1. Prepare AWS credentials
  2. Install the OADP Operator and give it an IAM role

1.6.1.1. Preparing AWS credentials for OADP

Prepare and configure an Amazon Web Services account to install OpenShift API for Data Protection (OADP).

Procedure

  1. Create the following environment variables by running the following commands:

    Important

    Change the cluster name to match your cluster, and ensure you are logged into the cluster as an administrator. Ensure that all fields are outputted correctly before continuing.

    $ export CLUSTER_NAME=<my_cluster>

    Replace <my_cluster> with your cluster name.

    $ export ROSA_CLUSTER_ID=$(rosa describe cluster -c ${CLUSTER_NAME} --output json | jq -r .id)
    $ export REGION=$(rosa describe cluster -c ${CLUSTER_NAME} --output json | jq -r .region.id)
    $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o jsonpath='{.spec.serviceAccountIssuer}' | sed 's|^https://||')
    $ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
    $ export CLUSTER_VERSION=$(rosa describe cluster -c ${CLUSTER_NAME} -o json | jq -r .version.raw_id | cut -f -2 -d '.')
    $ export ROLE_NAME="${CLUSTER_NAME}-openshift-oadp-aws-cloud-credentials"
    $ export SCRATCH="/tmp/${CLUSTER_NAME}/oadp"
    $ mkdir -p ${SCRATCH}
    $ echo "Cluster ID: ${ROSA_CLUSTER_ID}, Region: ${REGION}, OIDC Endpoint:
      ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"
  2. On the AWS account, create an IAM policy to allow access to AWS S3:

    1. Check to see if the policy exists by running the following command:

      $ POLICY_ARN=$(aws iam list-policies --query "Policies[?PolicyName=='RosaOadpVer1'].{ARN:Arn}" --output text)
      • RosaOadp: Replace RosaOadp with your policy name.
    2. Enter the following command to create the policy JSON file and then create the policy:

      Note

      If the policy ARN is not found, the command creates the policy. If the policy ARN already exists, the if statement intentionally skips the policy creation.

      $ if [[ -z "${POLICY_ARN}" ]]; then
        cat << EOF > ${SCRATCH}/policy.json
        {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Effect": "Allow",
            "Action": [
              "s3:CreateBucket",
              "s3:DeleteBucket",
              "s3:PutBucketTagging",
              "s3:GetBucketTagging",
              "s3:PutEncryptionConfiguration",
              "s3:GetEncryptionConfiguration",
              "s3:PutLifecycleConfiguration",
              "s3:GetLifecycleConfiguration",
              "s3:GetBucketLocation",
              "s3:ListBucket",
              "s3:GetObject",
              "s3:PutObject",
              "s3:DeleteObject",
              "s3:ListBucketMultipartUploads",
              "s3:AbortMultipartUpload",
              "s3:ListMultipartUploadParts",
              "ec2:DescribeSnapshots",
              "ec2:DescribeVolumes",
              "ec2:DescribeVolumeAttribute",
              "ec2:DescribeVolumesModifications",
              "ec2:DescribeVolumeStatus",
              "ec2:CreateTags",
              "ec2:CreateVolume",
              "ec2:CreateSnapshot",
              "ec2:DeleteSnapshot"
            ],
            "Resource": "*"
          }
        ]}
      EOF
      
        POLICY_ARN=$(aws iam create-policy --policy-name "RosaOadpVer1" \
        --policy-document file:///${SCRATCH}/policy.json --query Policy.Arn \
        --tags Key=rosa_openshift_version,Value=${CLUSTER_VERSION} Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-oadp Key=operator_name,Value=openshift-oadp \
        --output text)
        fi
      • SCRATCH: SCRATCH is a name for a temporary directory created for the environment variables.
    3. View the policy ARN by running the following command:

      $ echo ${POLICY_ARN}
  3. Create an IAM role trust policy for the cluster:

    1. Create the trust policy file by running the following command:

      $ cat <<EOF > ${SCRATCH}/trust-policy.json
        {
            "Version": "2012-10-17",
            "Statement": [{
              "Effect": "Allow",
              "Principal": {
                "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_ENDPOINT}"
              },
              "Action": "sts:AssumeRoleWithWebIdentity",
              "Condition": {
                "StringEquals": {
                  "${OIDC_ENDPOINT}:sub": [
                    "system:serviceaccount:openshift-adp:openshift-adp-controller-manager",
                    "system:serviceaccount:openshift-adp:velero"]
                }
              }
            }]
        }
      EOF
    2. Create the role by running the following command:

      $ ROLE_ARN=$(aws iam create-role --role-name \
        "${ROLE_NAME}" \
        --assume-role-policy-document file://${SCRATCH}/trust-policy.json \
        --tags Key=rosa_cluster_id,Value=${ROSA_CLUSTER_ID} \
               Key=rosa_openshift_version,Value=${CLUSTER_VERSION} \
               Key=rosa_role_prefix,Value=ManagedOpenShift \
               Key=operator_namespace,Value=openshift-adp \
               Key=operator_name,Value=openshift-oadp \
        --query Role.Arn --output text)
    3. View the role ARN by running the following command:

      $ echo ${ROLE_ARN}
  4. Attach the IAM policy to the IAM role by running the following command:

    $ aws iam attach-role-policy --role-name "${ROLE_NAME}" \
      --policy-arn ${POLICY_ARN}

Install OpenShift API for Data Protection (OADP) on clusters with AWS STS. AWS Security Token Service (AWS STS) is a global web service that provides short-term credentials for IAM or federated users. Red Hat OpenShift Service on AWS classic architecture with STS is the recommended credential mode.

Important

Restic is unsupported.

Kopia file system backup (FSB) is supported when backing up file systems that do not support Container Storage Interface (CSI) snapshots.

Example file systems include the following:

  • Amazon Elastic File System (EFS)
  • Network File System (NFS)
  • emptyDir volumes
  • Local volumes

For backing up volumes on clusters with AWS STS, OADP recommends native snapshots and Container Storage Interface (CSI) snapshots. Data Mover backups are supported, but can be slower than native snapshots.

In a Red Hat OpenShift Service on AWS classic architecture cluster that uses STS authentication, restoring backed-up data in a different AWS region is not supported.

Prerequisites

  • A Red Hat OpenShift Service on AWS classic architecture cluster with the required access and tokens. For instructions, see the previous procedure Preparing AWS credentials for OADP. If you plan to use two different clusters for backing up and restoring, you must prepare AWS credentials, including ROLE_ARN, for each cluster.

Procedure

  1. Create a Red Hat OpenShift Service on AWS classic architecture secret from your AWS token file by entering the following commands:

    1. Create the credentials file:

      $ cat <<EOF > ${SCRATCH}/credentials
        [default]
        role_arn = ${ROLE_ARN}
        web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
        region = <aws_region>
      EOF

      Replace <aws_region> with the AWS region to use for the STS endpoint.

    2. Create a namespace for OADP:

      $ oc create namespace openshift-adp
    3. Create the Red Hat OpenShift Service on AWS classic architecture secret:

      $ oc -n openshift-adp create secret generic cloud-credentials \
        --from-file=${SCRATCH}/credentials
      Note

      In Red Hat OpenShift Service on AWS classic architecture versions 4.15 and later, the OADP Operator supports a new standardized STS workflow through the Operator Lifecycle Manager (OLM) and Cloud Credentials Operator (CCO). In this workflow, you do not need to create the above secret, you only need to supply the role ARN during the installation of OLM-managed operators using the Red Hat OpenShift Service on AWS classic architecture web console, for more information see Installing from software catalog using the web console.

      The preceding secret is created automatically by CCO.

  2. Install the OADP Operator:

    1. In the Red Hat OpenShift Service on AWS classic architecture web console, browse to Ecosystem Software Catalog.
    2. Search for the OADP Operator.
    3. In the role_ARN field, paste the role_arn that you created previously and click Install.
  3. Create AWS cloud storage using your AWS credentials by entering the following command:

    $ cat << EOF | oc create -f -
      apiVersion: oadp.openshift.io/v1alpha1
      kind: CloudStorage
      metadata:
        name: ${CLUSTER_NAME}-oadp
        namespace: openshift-adp
      spec:
        creationSecret:
          key: credentials
          name: cloud-credentials
        enableSharedConfig: true
        name: ${CLUSTER_NAME}-oadp
        provider: aws
        region: $REGION
    EOF
  4. Check your application’s storage default storage class by entering the following command:

    $ oc get pvc -n <namespace>
    NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    applog   Bound    pvc-351791ae-b6ab-4e8b-88a4-30f73caf5ef8   1Gi        RWO            gp3-csi        4d19h
    mysql    Bound    pvc-16b8e009-a20a-4379-accc-bc81fedd0621   1Gi        RWO            gp3-csi        4d19h
  5. Get the storage class by running the following command:

    $ oc get storageclass
    NAME                PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
    gp2                 kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   true                   4d21h
    gp2-csi             ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
    gp3                 ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
    gp3-csi (default)   ebs.csi.aws.com         Delete          WaitForFirstConsumer   true                   4d21h
    Note

    The following storage classes will work:

    • gp3-csi
    • gp2-csi
    • gp3
    • gp2

    If the application or applications that are being backed up are all using persistent volumes (PVs) with Container Storage Interface (CSI), it is advisable to include the CSI plugin in the OADP DPA configuration.

  6. Create the DataProtectionApplication resource to configure the connection to the storage where the backups and volume snapshots are stored:

    1. If you are using only CSI volumes, deploy a Data Protection Application by entering the following command:

      $ cat << EOF | oc create -f -
        apiVersion: oadp.openshift.io/v1alpha1
        kind: DataProtectionApplication
        metadata:
          name: ${CLUSTER_NAME}-dpa
          namespace: openshift-adp
        spec:
          backupImages: true
          features:
            dataMover:
              enable: false
          backupLocations:
          - bucket:
              cloudStorageRef:
                name: ${CLUSTER_NAME}-oadp
              credential:
                key: credentials
                name: cloud-credentials
              prefix: velero
              default: true
              config:
                region: ${REGION}
          configuration:
            velero:
              defaultPlugins:
              - openshift
              - aws
              - csi
            nodeAgent:
              enable: false
              uploaderType: kopia
      EOF

      where:

      backupImages
      Red Hat OpenShift Service on AWS classic architecture supports internal image backup. Set this field to false if you do not want to use image backup.
      nodeAgent
      See the important note regarding the nodeAgent attribute at the end of this procedure.
      uploaderType
      Specifies the type of uploader. The built-in Data Mover uses Kopia as the default uploader mechanism regardless of the value of the uploaderType field.
    2. If you are using CSI or non-CSI volumes, deploy a Data Protection Application by entering the following command:

      $ cat << EOF | oc create -f -
        apiVersion: oadp.openshift.io/v1alpha1
        kind: DataProtectionApplication
        metadata:
          name: ${CLUSTER_NAME}-dpa
          namespace: openshift-adp
        spec:
          backupImages: true
          backupLocations:
          - bucket:
              cloudStorageRef:
                name: ${CLUSTER_NAME}-oadp
              credential:
                key: credentials
                name: cloud-credentials
              prefix: velero
              default: true
              config:
                region: ${REGION}
          configuration:
            velero:
              defaultPlugins:
              - openshift
              - aws
            nodeAgent:
              enable: false
              uploaderType: restic
          snapshotLocations:
            - velero:
                config:
                  credentialsFile: /tmp/credentials/openshift-adp/cloud-credentials-credentials
                  enableSharedConfig: "true"
                  profile: default
                  region: ${REGION}
                provider: aws
      EOF

      where:

      backupImages
      Red Hat OpenShift Service on AWS classic architecture supports internal image backup. Set this field to false if you do not want to use image backup.
      nodeAgent
      See the important note regarding the nodeAgent attribute at the end of this procedure.
      credentialsFile
      Specifies the mounted location of the bucket credential on the pod.
      enableSharedConfig
      Specifies whether the snapshotLocations can share or reuse the credential defined for the bucket.
      profile
      Specifies the profile name set in the AWS credentials file.
      region

      Specifies your AWS region. This must be the same as the cluster region.

      You are now ready to back up and restore Red Hat OpenShift Service on AWS classic architecture applications, as described in Backing up applications.

      Important

      The enable parameter of restic is set to false in this configuration, because OADP does not support Restic in Red Hat OpenShift Service on AWS classic architecture environments.

    If you want to use two different clusters for backing up and restoring, the two clusters must have the same AWS S3 storage names in both the cloud storage CR and the OADP DataProtectionApplication configuration.

Update the OADP Operator subscription to fix an installation error due to incorrect IAM role Amazon Resource Name (ARN).

While installing the OADP Operator on a Red Hat OpenShift Service on AWS classic architecture cluster, if you provide an incorrect IAM role Amazon Resource Name (ARN), the openshift-adp-controller pod gives an error. The credential requests that are generated contain the wrong IAM role ARN. To update the credential requests object with the correct IAM role ARN, you can edit the OADP Operator subscription and patch the IAM role ARN with the correct value. By editing the OADP Operator subscription, you do not have to uninstall and reinstall OADP to update the IAM role ARN.

Prerequisites

  • You have a Red Hat OpenShift Service on AWS classic architecture cluster with the required access and tokens.
  • You have installed OADP on the Red Hat OpenShift Service on AWS classic architecture cluster.

Procedure

  1. To verify that the OADP subscription has the wrong IAM role ARN environment variable set, run the following command:

    $ oc get sub -o yaml redhat-oadp-operator

    Example subscription

    apiVersion: operators.coreos.com/v1alpha1
    kind: Subscription
    metadata:
      annotations:
      creationTimestamp: "2025-01-15T07:18:31Z"
      generation: 1
      labels:
        operators.coreos.com/redhat-oadp-operator.openshift-adp: ""
      name: redhat-oadp-operator
      namespace: openshift-adp
      resourceVersion: "77363"
      uid: 5ba00906-5ad2-4476-ae7b-ffa90986283d
    spec:
      channel: stable-1.4
      config:
        env:
        - name: ROLEARN
          value: arn:aws:iam::11111111:role/wrong-role-arn
      installPlanApproval: Manual
      name: redhat-oadp-operator
      source: prestage-operators
      sourceNamespace: openshift-marketplace
      startingCSV: oadp-operator.v1.4.2

    where:

    ROLEARN
    Verify the value of ROLEARN you want to update.
  2. Update the ROLEARN field of the subscription with the correct role ARN by running the following command:

    $ oc patch subscription redhat-oadp-operator -p '{"spec": {"config": {"env": [{"name": "ROLEARN", "value": "<role_arn>"}]}}}' --type='merge'

    where:

    <role_arn>
    Specifies the IAM role ARN to be updated. For example, arn:aws:iam::160…​..6956:role/oadprosa…​..8wlf.
  3. Verify that the secret object is updated with correct role ARN value by running the following command:

    $ oc get secret cloud-credentials -o jsonpath='{.data.credentials}' | base64 -d

    Example output

    [default]
    sts_regional_endpoints = regional
    role_arn = arn:aws:iam::160.....6956:role/oadprosa.....8wlf
    web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

  4. Configure the DataProtectionApplication custom resource (CR) manifest file as shown in the following example:

    apiVersion: oadp.openshift.io/v1alpha1
    kind: DataProtectionApplication
    metadata:
      name: test-rosa-dpa
      namespace: openshift-adp
    spec:
      backupLocations:
      - bucket:
          config:
            region: us-east-1
          cloudStorageRef:
            name: <cloud_storage>
          credential:
            name: cloud-credentials
            key: credentials
          prefix: velero
          default: true
      configuration:
        velero:
          defaultPlugins:
          - aws
          - openshift

    where:

    <cloud_storage>
    Specifies the CloudStorage CR.
  5. Create the DataProtectionApplication CR by running the following command:

    $ oc create -f <dpa_manifest_file>
  6. Verify that the DataProtectionApplication CR is reconciled and the status is set to "True" by running the following command:

    $  oc get dpa -n openshift-adp -o yaml

    Example DataProtectionApplication

    apiVersion: oadp.openshift.io/v1alpha1
    kind: DataProtectionApplication
    ...
    status:
        conditions:
        - lastTransitionTime: "2023-07-31T04:48:12Z"
          message: Reconcile complete
          reason: Complete
          status: "True"
          type: Reconciled

  7. Verify that the BackupStorageLocation CR is in an available state by running the following command:

    $ oc get backupstoragelocations.velero.io -n openshift-adp

    Example BackupStorageLocation

    NAME       PHASE       LAST VALIDATED   AGE   DEFAULT
    ts-dpa-1   Available   3s               6s    true

1.7. Uninstalling OADP

You uninstall the OpenShift API for Data Protection (OADP) by deleting the OADP Operator.

1.8. OADP backing up

1.8.1. Backing up applications

Back up applications by creating a Backup custom resource (CR) using snapshots, CSI, or File System Backup with Kopia or Restic.

Frequent backups might consume storage on the backup storage location. Check the frequency of backups, retention time, and the amount of data of the persistent volumes (PVs) if using non-local backups, for example, S3 buckets. Because all backups are retained until they expire, check the time to live (TTL) setting of the schedule.

Review the following information regarding backing up applications by using OADP:

The Backup CR creates backup files for Kubernetes resources and internal images on S3 object storage.

Important

The …​/.snapshot directory is a snapshot copy directory, which is used by several NFS servers. This directory has read-only access by default, so Velero cannot restore to this directory.

Do not give Velero write access to the .snapshot directory, and disable client access to this directory.

Preview the backup and restore resources in advance by doing a dry run of the backup and restore operations. This helps you to verify which resources will be included before committing to a full backup or restore.

OADP backs up application resources based on the type, namespace, or label. This means that you can view the resources after the backup is complete. Similarly, you can view the restored objects based on the namespace, persistent volume (PV), or label after a restore operation is complete.

Prerequisites

  • You have installed the OADP Operator.

Procedure

  1. To preview the resources included in the backup before running the actual backup, run the following command:

    $ velero backup create <backup-name> --snapshot-volumes false

    Specify the value of --snapshot-volumes parameter as false.

  2. To know more details about the backup resources, run the following command:

    $ velero describe backup <backup_name> --details

    Replace <backup_name> with the name of the backup.

  3. To preview the resources included in the restore before running the actual restore, run the following command:

    $ velero restore create --from-backup <backup_name>

    Replace <backup_name> with the name of the backup.

    Important

    The velero restore create command creates restore resources in the cluster. You must delete the resources created as part of the restore, after you review the resources.

  4. To know more details about the restore resources, run the following command:

    $ velero describe restore <restore_name> --details

    Replace <restore_name> with the name of the restore.

1.8.2. Creating a Backup CR

Back up Kubernetes resources, internal images, and persistent volumes (PVs) by creating a Backup custom resource (CR). This helps you to protect your application data and configuration for disaster recovery.

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.
  • The DataProtectionApplication CR must be in a Ready state.
  • Backup location prerequisites:

    • You must have S3 object storage configured for Velero.
    • You must have a backup location configured in the DataProtectionApplication CR.
  • Snapshot location prerequisites:

    • Your cloud provider must have a native snapshot API or support Container Storage Interface (CSI) snapshots.
    • For CSI snapshots, you must create a VolumeSnapshotClass CR to register the CSI driver.
    • You must have a volume location configured in the DataProtectionApplication CR.

Procedure

  1. Retrieve the backupStorageLocations CRs by entering the following command:

    $ oc get backupstoragelocations.velero.io -n openshift-adp
    NAMESPACE       NAME              PHASE       LAST VALIDATED   AGE   DEFAULT
    openshift-adp   velero-sample-1   Available   11s              31m
  2. Create a Backup CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Backup
    metadata:
      name: <backup>
      labels:
        velero.io/storage-location: default
      namespace: openshift-adp
    spec:
      hooks: {}
      includedNamespaces:
      - <namespace>
      includedResources: []
      excludedResources: []
      storageLocation: <velero-sample-1>
      ttl: 720h0m0s
      labelSelector:
        matchLabels:
          app: <label_1>
          app: <label_2>
          app: <label_3>
      orLabelSelectors:
      - matchLabels:
          app: <label_1>
          app: <label_2>
          app: <label_3>

    where:

    <namespace>
    Specifies an array of namespaces to back up.
    includedResources
    Optional: Specifies an array of resources to include in the backup. Resources might be shortcuts (for example, po for pods) or fully-qualified. If unspecified, all resources are included.
    excludedResources
    Optional: Specifies an array of resources to exclude from the backup. Resources might be shortcuts (for example, po for pods) or fully-qualified.
    <velero-sample-1>
    Specifies the name of the backupStorageLocations CR.
    labelSelector
    Specifies a map of {key,value} pairs of backup resources that have all the specified labels.
    orLabelSelectors
    Specifies a map of {key,value} pairs of backup resources that have one or more of the specified labels.
  3. Verify that the status of the Backup CR is Completed:

    $ oc get backups.velero.io -n openshift-adp <backup> -o jsonpath='{.status.phase}'

1.8.3. Parallel backup processing in OADP

With parallel backup processing in OpenShift API for Data Protection, you can back up several applications from different namespaces simultaneously. This concurrent processing ensures that a single large backup task does not delay the backups for other applications in your cluster.

By default, Velero processes backup resources sequentially - one resource in the InProgress phase at a time. Parallel backup changes this behavior to handle backups concurrently.

The following considerations apply for enabling parallel backups in OADP:

  • Resource allocation: Increasing the concurrency settings requires additional CPU and memory for the Velero container. Monitor resource consumption to ensure cluster stability.
  • Backup scope: Parallel processing provides the most benefit for backups that contain a large number of Kubernetes resources or many smaller volumes. Backups dominated by a few large volumes see less benefit because most time is spent waiting for asynchronous data movement to complete.
  • Namespace partitioning: To keep data integrity, OADP uses namespace-level isolation when processing parallel backups. Two backups cannot run simultaneously if they share any included namespaces.

    Expand
    Table 1.4. Parallel backup execution examples
    ScenarioIncluded namespacesProcessing behavior

    Backup A and Backup B

    namespace1 and namespace2

    Parallel: The namespaces are isolated.

    Backup A and Backup B

    namespace1 and namespace1

    Serialized: The shared namespace creates a lock.

    Full cluster and Backup A

    All namespaces and namespace1

    Serialized: A cluster-wide backup includes all namespaces and locks the entire cluster.

1.8.4. Enabling parallel backup processing

By default, OpenShift API for Data Protection processes only one backup in the InProgress phase at a time. Configure the DataProtectionApplication (DPA) custom resource (CR) to run several backups simultaneously to prevent smaller backups from being queued behind larger operations.

Prerequisites

  • You must be logged in as a user with cluster-admin privileges.
  • You must have a DPA CR configured and deployed in your cluster.

Procedure

  1. Edit your DPA CR:

    $ oc edit dpa <dpa_name> -n openshift-adp
  2. Add the concurrentBackups field in the spec.configuration.velero section of your DPA CR:

      configuration:
        velero:
          concurrentBackups: <integer_limit>
          defaultPlugins:
            - openshift
            - aws
            - csi

    Replace <integer_limit> with the maximum number of backups you want OADP to process simultaneously. The default value is 1. Backups that target overlapping namespaces are automatically serialized.

Verification

  • Verify that all backups are in the InProgress phase simultaneously:

    $ oc get backups.velero.io -n openshift-adp

    Example output

    NAME                STATUS       CREATED                        EXPIRES
    backup-namespace1   InProgress   2026-04-28 10:00:00 +0000 UTC  29d
    backup-namespace2   InProgress   2026-04-28 10:00:05 +0000 UTC  29d

1.8.5. Creating backup hooks

Create backup hooks to run commands in a container in a pod by editing the Backup custom resource (CR). This helps you to run pre-backup and post-backup actions such as quiescing a database or flushing data to disk.

The commands can be configured to performed before any custom action processing (Pre hooks), or after all custom actions have been completed and any additional items specified by the custom action have been backed up (Post hooks).

Procedure

  • Add a hook to the spec.hooks block of the Backup CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Backup
    metadata:
      name: <backup>
      namespace: openshift-adp
    spec:
      hooks:
        resources:
          - name: <hook_name>
            includedNamespaces:
            - <namespace>
            excludedNamespaces:
            - <namespace>
            includedResources: []
            - pods
            excludedResources: []
            labelSelector:
              matchLabels:
                app: velero
                component: server
            pre:
              - exec:
                  container: <container>
                  command:
                  - /bin/uname
                  - -a
                  onError: Fail
                  timeout: 30s
            post:
    ...

    where:

    <namespace>
    Optional: Specifies the namespaces to which the hook applies. If this value is not specified, the hook applies to all namespaces.
    excludedNamespaces
    Optional: Specifies the namespaces to which the hook does not apply.
    pods
    Currently, pods are the only supported resource that hooks can apply to.
    excludedResources
    Optional: Specifies the resources to which the hook does not apply.
    labelSelector
    Optional: This hook only applies to objects matching the label. If this value is not specified, the hook applies to all objects.
    pre
    Specifies an array of hooks to run before the backup.
    <container>
    Optional: Specifies the container in which the command runs. If the container is not specified, the command runs in the first container in the pod.
    /bin/uname
    Specifies the entry point for the init container being added.
    onError: Fail
    Specifies the error handling behavior. Allowed values are Fail and Continue. The default is Fail.
    timeout: 30s
    Optional: Specifies how long to wait for the commands to run. The default is 30s.
    post
    Specifies an array of hooks to run after the backup, with the same parameters as the pre-backup hooks.

1.8.6. Scheduling backups using Schedule CR

Schedule backup operations by creating a Schedule custom resource (CR) with a Cron expression. This helps you to automate recurring backups of your application data at regular intervals.

You schedule backups by creating a Schedule custom resource (CR) instead of a Backup CR.

Warning

Leave enough time in your backup schedule for a backup to finish before another backup is created.

For example, if a backup of a namespace typically takes 10 minutes, do not schedule backups more frequently than every 15 minutes.

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.
  • The DataProtectionApplication CR must be in a Ready state.

Procedure

  1. Retrieve the backupStorageLocations CRs:

    $ oc get backupStorageLocations -n openshift-adp
    NAMESPACE       NAME              PHASE       LAST VALIDATED   AGE   DEFAULT
    openshift-adp   velero-sample-1   Available   11s              31m
  2. Create a Schedule CR, as in the following example:

    $ cat << EOF | oc apply -f -
    apiVersion: velero.io/v1
    kind: Schedule
    metadata:
      name: <schedule>
      namespace: openshift-adp
    spec:
      schedule: 0 7 * * *
      template:
        hooks: {}
        includedNamespaces:
        - <namespace>
        storageLocation: <velero-sample-1>
        defaultVolumesToFsBackup: true
        ttl: 720h0m0s
    EOF

    where:

    schedule: 0 7 * * *

    Specifies the cron expression to schedule the backup, for example, 0 7 * * * to perform a backup every day at 7:00.

    Note

    To schedule a backup at specific intervals, enter the <duration_in_minutes> in the following format:

      schedule: "*/10 * * * *"

    Enter the minutes value between quotation marks (" ").

    <namespace>
    Specifies an array of namespaces to back up.
    <velero-sample-1>
    Specifies the name of the backupStorageLocations CR.
    defaultVolumesToFsBackup: true
    Optional: In OADP version 1.2 and later, add the defaultVolumesToFsBackup: true key-value pair to your configuration when performing backups of volumes with Restic. In OADP version 1.1, add the defaultVolumesToRestic: true key-value pair when you back up volumes with Restic.

Verification

  • Verify that the status of the Schedule CR is Completed after the scheduled backup runs:

    $ oc get schedule -n openshift-adp <schedule> -o jsonpath='{.status.phase}'

1.8.7. Deleting backups

Delete a backup by creating the DeleteBackupRequest custom resource (CR) or by running the velero backup delete command. This helps you to free up storage space and remove outdated backup artifacts.

The volume backup artifacts are deleted at different times depending on the backup method:

  • Restic: The artifacts are deleted in the next full maintenance cycle, after the backup is deleted.
  • Container Storage Interface (CSI): The artifacts are deleted immediately when the backup is deleted.
  • Kopia: The artifacts are deleted after three full maintenance cycles of the Kopia repository, after the backup is deleted.

Delete a backup by creating a DeleteBackupRequest custom resource (CR). This helps you to remove specific backups and their associated volume artifacts from storage.

Prerequisites

  • You have run a backup of your application.

Procedure

  1. Create a DeleteBackupRequest CR manifest file:

    apiVersion: velero.io/v1
    kind: DeleteBackupRequest
    metadata:
      name: deletebackuprequest
      namespace: openshift-adp
    spec:
      backupName: <backup_name>

    Replace <backup_name> with the name of the backup.

  2. Apply the DeleteBackupRequest CR to delete the backup:

    $ oc apply -f <deletebackuprequest_cr_filename>

1.8.7.2. Deleting a backup by using the Velero CLI

Delete a backup by using the Velero CLI to run the velero backup delete command. This helps you to quickly remove backups and their associated volume artifacts from storage.

Prerequisites

  • You have run a backup of your application.
  • You downloaded the Velero CLI and can access the Velero binary in your cluster.

Procedure

  • To delete the backup, run the following Velero command:

    $ velero backup delete <backup_name> -n openshift-adp

    Replace <backup_name> with the name of the backup.

1.8.7.3. About Kopia repository maintenance

Kopia repository maintenance has two types, quick and full, that run automatically to optimize index performance and perform garbage collection. Knowing the two types helps you understand the maintenance cycle and how long it takes for backup artifacts to be deleted.

Quick maintenance
  • Runs every hour to keep the number of index blobs (n) low. A high number of indexes negatively affects the performance of Kopia operations.
  • Does not delete any metadata from the repository without ensuring that another copy of the same metadata exists.
Full maintenance
  • Runs every 24 hours to perform garbage collection of repository contents that are no longer needed.
  • snapshot-gc, a full maintenance task, finds all files and directory listings that are no longer accessible from snapshot manifests and marks them as deleted.
  • A full maintenance is a resource-costly operation, as it requires scanning all directories in all snapshots that are active in the cluster.
1.8.7.3.1. Kopia maintenance in OADP

The repo-maintain-job jobs are executed in the namespace where OADP is installed, as shown in the following example:

pod/repo-maintain-job-173...2527-2nbls                             0/1     Completed   0          168m
pod/repo-maintain-job-173....536-fl9tm                             0/1     Completed   0          108m
pod/repo-maintain-job-173...2545-55ggx                             0/1     Completed   0          48m

You can check the logs of the repo-maintain-job for more details about the cleanup and the removal of artifacts in the backup object storage. You can find a note, as shown in the following example, in the repo-maintain-job when the next full cycle maintenance is due:

not due for full maintenance cycle until 2024-00-00 18:29:4
Important

Three successful executions of a full maintenance cycle are required for the objects to be deleted from the backup object storage. This means you can expect up to 72 hours for all the artifacts in the backup object storage to be deleted.

1.8.7.4. Deleting a backup repository

Delete the backuprepository custom resource (CR) to complete the backup deletion process. This helps you to ensure that all backup metadata and artifacts are fully removed from storage.

After you delete the backup, and after the Kopia repository maintenance cycles to delete the related artifacts are complete, the backup is no longer referenced by any metadata or manifest objects.

Prerequisites

  • You have deleted the backup of your application.
  • You have waited up to 72 hours after the backup is deleted. This time frame allows Kopia to run the repository maintenance cycles.

Procedure

  1. To get the name of the backup repository CR for a backup, run the following command:

    $ oc get backuprepositories.velero.io -n openshift-adp
  2. To delete the backup repository CR, run the following command:

    $ oc delete backuprepository <backup_repository_name> -n openshift-adp

    Replace <backup_repository_name> with the name of the backup repository.

1.9. OADP restoring

1.9.1. Restoring applications

Restore application backups by previewing resources before running the restore, creating a Restore custom resource (CR), and configuring restore hooks to run commands in restored pods. This helps you to recover your application data and configuration while controlling the restore process.

Preview the backup and restore resources in advance by doing a dry run of the backup and restore operations. This helps you to verify which resources will be included before committing to a full backup or restore.

OADP backs up application resources based on the type, namespace, or label. This means that you can view the resources after the backup is complete. Similarly, you can view the restored objects based on the namespace, persistent volume (PV), or label after a restore operation is complete.

Prerequisites

  • You have installed the OADP Operator.

Procedure

  1. To preview the resources included in the backup before running the actual backup, run the following command:

    $ velero backup create <backup-name> --snapshot-volumes false

    Specify the value of --snapshot-volumes parameter as false.

  2. To know more details about the backup resources, run the following command:

    $ velero describe backup <backup_name> --details

    Replace <backup_name> with the name of the backup.

  3. To preview the resources included in the restore before running the actual restore, run the following command:

    $ velero restore create --from-backup <backup_name>

    Replace <backup_name> with the name of the backup.

    Important

    The velero restore create command creates restore resources in the cluster. You must delete the resources created as part of the restore, after you review the resources.

  4. To know more details about the restore resources, run the following command:

    $ velero describe restore <restore_name> --details

    Replace <restore_name> with the name of the restore.

1.9.1.2. Creating a Restore CR

Restore a Backup custom resource (CR) by creating a Restore CR.

When you restore a stateful application that uses the azurefile-csi storage class, the restore operation remains in the Finalizing phase.

Prerequisites

  • You must install the OpenShift API for Data Protection (OADP) Operator.
  • The DataProtectionApplication CR must be in a Ready state.
  • You must have a Velero Backup CR.
  • The persistent volume (PV) capacity must match the requested size at backup time. Adjust the requested size if needed.

Procedure

  1. Create a Restore CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Restore
    metadata:
      name: <restore>
      namespace: openshift-adp
    spec:
      backupName: <backup>
      includedResources: []
      excludedResources:
      - nodes
      - events
      - events.events.k8s.io
      - backups.velero.io
      - restores.velero.io
      - resticrepositories.velero.io
      restorePVs: true

    where:

    <backup>
    Specifies the name of the Backup CR.
    includedResources
    Optional: Specifies an array of resources to include in the restore process. Resources might be shortcuts (for example, po for pods) or fully-qualified. If unspecified, all resources are included.
    restorePVs: true
    Optional: The restorePVs parameter can be set to false to turn off restore of PersistentVolumes from VolumeSnapshot of Container Storage Interface (CSI) snapshots or from native snapshots when VolumeSnapshotLocation is configured.
  2. Verify that the status of the Restore CR is Completed by entering the following command:

    $ oc get restores.velero.io -n openshift-adp <restore> -o jsonpath='{.status.phase}'
  3. Verify that the backup resources have been restored by entering the following command:

    $ oc get all -n <namespace>

    where:

    <namespace>
    Specifies the namespace that you backed up.
  4. If you restore DeploymentConfig with volumes or if you use post-restore hooks, run the dc-post-restore.sh cleanup script by entering the following command:

    $ bash dc-restic-post-restore.sh -> dc-post-restore.sh
    Note

    During the restore process, the OADP Velero plug-ins scale down the DeploymentConfig objects and restore the pods as standalone pods. This is done to prevent the cluster from deleting the restored DeploymentConfig pods immediately on restore and to allow the restore and post-restore hooks to complete their actions on the restored pods. The cleanup script shown below removes these disconnected pods and scales any DeploymentConfig objects back up to the appropriate number of replicas.

    #!/bin/bash
    set -e
    
    # if sha256sum exists, use it to check the integrity of the file
    if command -v sha256sum >/dev/null 2>&1; then
      CHECKSUM_CMD="sha256sum"
    else
      CHECKSUM_CMD="shasum -a 256"
    fi
    
    label_name () {
        if [ "${#1}" -le "63" ]; then
    	echo $1
    	return
        fi
        sha=$(echo -n $1|$CHECKSUM_CMD)
        echo "${1:0:57}${sha:0:6}"
    }
    
    if [[ $# -ne 1 ]]; then
        echo "usage: ${BASH_SOURCE} restore-name"
        exit 1
    fi
    
    echo "restore: $1"
    
    label=$(label_name $1)
    echo "label:   $label"
    
    echo Deleting disconnected restore pods
    oc delete pods --all-namespaces -l oadp.openshift.io/disconnected-from-dc=$label
    
    for dc in $(oc get dc --all-namespaces -l oadp.openshift.io/replicas-modified=$label -o jsonpath='{range .items[*]}{.metadata.namespace}{","}{.metadata.name}{","}{.metadata.annotations.oadp\.openshift\.io/original-replicas}{","}{.metadata.annotations.oadp\.openshift\.io/original-paused}{"\n"}')
    do
        IFS=',' read -ra dc_arr <<< "$dc"
        if [ ${#dc_arr[0]} -gt 0 ]; then
    	echo Found deployment ${dc_arr[0]}/${dc_arr[1]}, setting replicas: ${dc_arr[2]}, paused: ${dc_arr[3]}
    	cat <<EOF | oc patch dc  -n ${dc_arr[0]} ${dc_arr[1]} --patch-file /dev/stdin
    spec:
      replicas: ${dc_arr[2]}
      paused: ${dc_arr[3]}
    EOF
        fi
    done

1.9.1.3. Creating restore hooks

Create restore hooks to run commands in a container in a pod by editing the Restore custom resource (CR).

You can create two types of restore hooks:

  • An init hook adds an init container to a pod to perform setup tasks before the application container starts.

    If you restore a Restic backup, the restic-wait init container is added before the restore hook init container.

  • An exec hook runs commands or scripts in a container of a restored pod.

Procedure

  • Add a hook to the spec.hooks block of the Restore CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Restore
    metadata:
      name: <restore>
      namespace: openshift-adp
    spec:
      hooks:
        resources:
          - name: <hook_name>
            includedNamespaces:
            - <namespace>
            excludedNamespaces:
            - <namespace>
            includedResources:
            - pods
            excludedResources: []
            labelSelector:
              matchLabels:
                app: velero
                component: server
            postHooks:
            - init:
                initContainers:
                - name: restore-hook-init
                  image: alpine:latest
                  volumeMounts:
                  - mountPath: /restores/pvc1-vm
                    name: pvc1-vm
                  command:
                  - /bin/ash
                  - -c
                timeout:
            - exec:
                container: <container>
                command:
                - /bin/bash
                - -c
                - "psql < /backup/backup.sql"
                waitTimeout: 5m
                execTimeout: 1m
                onError: Continue

    where:

    <namespace>
    Optional: Specifies an array of namespaces to which the hook applies. If this value is not specified, the hook applies to all namespaces.
    pods
    Currently, pods are the only supported resource that hooks can apply to.
    labelSelector
    Optional: This hook only applies to objects matching the label selector.
    timeout
    Optional: Specifies the maximum length of time Velero waits for initContainers to complete.
    <container>
    Optional: Specifies the container in which the command runs. If the container is not specified, the command runs in the first container in the pod.
    /bin/bash
    Specifies the entrypoint for the init container being added.
    waitTimeout: 5m
    Optional: Specifies how long to wait for a container to become ready. This should be long enough for the container to start and for any preceding hooks in the same container to complete. If not set, the restore process waits indefinitely.
    execTimeout: 1m
    Optional: Specifies how long to wait for the commands to run. The default is 30s.
    onError: Continue

    Specifies the error handling behavior. Allowed values are Fail and Continue:

    • Continue: Only command failures are logged.
    • Fail: No more restore hooks run in any container in any pod. The status of the Restore CR will be PartiallyFailed.

During a File System Backup (FSB) restore operation, a Deployment resource referencing an ImageStream is not restored properly. The restored pod that runs the FSB, and the postHook is terminated prematurely.

This happens because, during the restore operation, OpenShift controller updates the spec.template.spec.containers[0].image field in the Deployment resource with an updated ImageStreamTag hash. The update triggers the rollout of a new pod, terminating the pod on which velero runs the FSB and the post restore hook. For more information about image stream trigger, see "Triggering updates on image stream changes".

The workaround for this behavior is a two-step restore process:

  1. First, perform a restore excluding the Deployment resources, for example:

    $ velero restore create <RESTORE_NAME> \
      --from-backup <BACKUP_NAME> \
      --exclude-resources=deployment.apps
  2. After the first restore is successful, perform a second restore by including these resources, for example:

    $ velero restore create <RESTORE_NAME> \
      --from-backup <BACKUP_NAME> \
      --include-resources=deployment.apps
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

Legal Notice

Theme

© 2026 Red Hat
Back to top