SupportConsoleDevelopersStart a trialContact

Chapter 1. Backing up etcd

etcd is the key-value store for OpenShift Container Platform, which persists the state of all resource objects.

Back up your cluster’s etcd data regularly and store in a secure location ideally outside the OpenShift Container Platform environment. Do not take an etcd backup before the first certificate rotation completes, which occurs 24 hours after installation, otherwise the backup will contain expired certificates. It is also recommended to take etcd backups during non-peak usage hours, as it is a blocking action.

Once you have an etcd backup, you can recover from lost master hosts and restore to a previous cluster state.

You can perform the etcd data backup process on any master host that has connectivity to the etcd cluster, where the proper certificates are provided.

1.1. Backing up etcd data

Follow these steps to back up etcd data by creating an etcd snapshot and backing up static Kubernetes API server resources. This backup can be saved and used at a later time if you need to restore etcd.

You should only save a backup from a single master host. You do not need a backup from each master host in the cluster.

Note

If you are taking an etcd backup on OpenShift Container Platform 4.3.0 or 4.3.1, then this procedure generates a single file that contains the etcd snapshot and static Kubernetes API server resources. When restoring, the etcd-snapshot-restore.sh script is backward compatible to accept this single file.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin role.

  • You have checked whether the cluster-wide proxy is enabled.

    Tip

    You can check whether the proxy is enabled by reviewing the output of oc get proxy cluster -o yaml. The proxy is enabled if the httpProxy, httpsProxy, and noProxy fields have values set.

Procedure

  1. Start a debug session for a master node: 

    $ oc debug node/<node_name>

  2. Change your root directory to the host: 

    sh-4.2# chroot /host

  3. Change to the /home/core directory: 

    sh-4.4# cd /home/core
  4. If the cluster-wide proxy is enabled, be sure that you have exported the NO_PROXY, HTTP_PROXY, and HTTPS_PROXY environment variables.

  5. Run the etcd-snapshot-backup.sh script and pass in the location to save the backup to. 

    sh-4.4# /usr/local/bin/etcd-snapshot-backup.sh /home/core/assets/backup

    In this example, two files are created in the /home/core/assets/backup/ directory on the master host:

    • snapshot_<datetimestamp>.db: This file is the etcd snapshot.

    • static_kuberesources_<datetimestamp>.tar.gz: This file contains the static Kubernetes API server resources. If etcd encryption is enabled, it also contains the encryption keys for the etcd snapshot.

      Note

      If etcd encryption is enabled, it is recommended to store this second file separately from the etcd snapshot for security reasons. However, this file is required in order to restore from the etcd snapshot.

      Keep in mind that etcd encryption only encrypts values, not keys. This means that resource types, namespaces, and object names are unencrypted.

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.