Chapter 8. Backing Up Satellite Server and Capsule Server
This chapter describes the minimum backup procedure to ensure the continuity of your Red Hat Satellite deployment and associated data in the event of a disaster. If your deployment uses custom configurations, you must consider how to handle these custom configurations when you plan your backup and disaster recovery policy.
8.1. Backing up Satellite Server or Capsule Server
Use this section to create a backup of your Satellite Server or Capsule Server and all associated data using the foreman-maintain backup
script. Backing up to a separate storage device on a separate system is highly recommended. Satellite services are unavailable during the backup. The backup can be scheduled for a quiet time using cron
, see the A Weekly Full Backup Followed by Daily Incremental Backups.
During offline or snapshot backups, the services are inactive and Satellite is in a maintenance mode. All the traffic from outside on port 443 is rejected by a firewall to ensure there are no modifications triggered.
A backup contains sensitive information from the /root/ssl-build
directory. For example, it can contain hostnames, ssh keys, request files and SSL certificates. Either encrypt or move the backup to a secure location to minimize the risk of damage or unauthorized access to the hosts.
Prerequisites
Ensure that no other tasks are scheduled by other administrators for the same time. This is particularly important when administrators are working in different locations and time zones.
Conventional Backup Methods
You can also use conventional backup methods such as that described in the System Backup and Recovery section of the Red Hat Enterprise Linux 7 System Administrator’s Guide. When creating a snapshot or conventional backup, stop all services. Do not do this if using the foreman-maintain backup
script:
# foreman-maintain service stop
Start the services after creating a snapshot or conventional backup:
# foreman-maintain service start
8.1.1. Estimating the size of a Backup
The full backup creates uncompressed archives of MongoDB, PostgreSQL, and Pulp database files and Satellite configuration files. Compression occurs after the archives are created to decrease the time when Satellite services are unavailable. Consequently, a full backup requires space to store the following data:
- Uncompressed Satellite databases and configuration files.
- Compressed Satellite databases and configuration files.
- An extra 20% of the total estimated space to ensure a reliable backup.
To Estimate the Size of a Backup
Enter the
du
command to estimate the size of uncompressed directories containing Satellite databases and configuration files:# du -sh /var/lib/mongodb /var/lib/pgsql/data /var/lib/pulp 480G /var/lib/mongodb 100G /var/lib/pgsql/data 100G /var/lib/pulp # du -csh /var/lib/qpidd /var/lib/tftpboot /etc /root/ssl-build \ /var/www/html/pub /opt/puppetlabs 886M /var/lib/qpidd 16M /var/lib/tftpboot 37M /etc 900K /root/ssl-build 100K /var/www/html/pub 2M /opt/puppetlabs 942M total
Calculate how much space is required to store the compressed data.
Table 8.1, “Backup Data Compression Ratio” demonstrates compression ratio of all data items used in the backup.
Table 8.1. Backup Data Compression Ratio Data type Directory Ratio Example results MongoDB database files
/var/lib/mongodb
85 - 90 %
480 GB
60 GB PostgreSQL database files
/var/lib/pgsql/data
80 - 85%
100 GB
20 GB Pulp RPM files
/var/lib/pulp
(not compressed)
100 GB
Configuration files
/var/lib/qpidd
/var/lib/tftpboot
/etc
/root-ssl/build
/var/www/html/pub
/opt/puppetlabs
85%
942 MB
141 MB In this example, the compressed backup data occupies 180 GB in total.
To calculate how much space you need to store a backup, add the total of the estimated values of compressed and uncompressed backup data and then add an extra 20% to ensure a reliable backup.
This example requires 681 GB plus 180 GB for the uncompressed and compressed backup data, 861 GB in total. With 172 GB of extra space, 1033 GB must be allocated for the backup location.
8.1.2. Performing a Full Backup of Satellite Server or Capsule Server
Red Hat Satellite 6.4 uses the foreman-maintain backup
script to make backups. To see the usage statement, enter the following command:
# foreman-maintain backup --help
There are three main methods of backing up Satellite Server:
- Offline backup
- Online backup
Snapshot backups
For more information about each of these methods, you can view the usage statements for each script.
For offline backups:
# foreman-maintain backup offline --help
For online backups:
# foreman-maintain backup online --help
For backups from snapshots:
# foreman-maintain backup snapshot --help
Directory creation
The foreman-maintain backup
script creates a time-stamped subdirectory in the backup directory you specify. The foreman-maintain backup
script does not overwrite backups. You must select the correct directory or subdirectory when restoring from a backup or an incremental backup. The script stops and restarts services as required.
If you must set the directory name yourself add the --preserve-directory
option and add a directory name. The backup is then stored in the directory you provide on the command line.
If you use --preserve-directory
, no data is removed if the backup fails.
Note that user postgres
needs write access to that directory if you have a local PgSQL database.
Remote databases
You can use the foreman-maintain backup
script to back up remote databases.
You can use both online and offline methods to back up remote databases, but if you use offline methods, such as snapshot, the foreman-maintain backup
script performs a database dump.
To Perform a Full Offline Backup of Satellite Server or Capsule Server:
Use this procedure to perform a full offline backup. Satellite services are unavailable during the backup process.
Request other users of Satellite Server or Capsule Server to save any changes and warn them that Satellite services are unavailable for the duration of the backup. Ensure no other tasks are scheduled for the same time as the backup.
- Ensure that your backup location has enough disk space to store the backup. For more information, see Section 8.1.1, “Estimating the size of a Backup”.
Run the backup script:
# foreman-maintain backup offline /var/backup_directory
This process can take a long time to complete, because of the amount of data to copy.
8.1.3. Performing a Backup without Pulp Content
To Perform a Backup without Pulp Content:
Use this procedure to perform an offline backup but excludes the contents of the Pulp directory. This backup is useful for debugging purposes and is only intended to provide access to configuration files without spending time backing up the Pulp database. You cannot restore from a directory that does not contain Pulp content.
Request other users of Satellite Server or Capsule Server to save any changes and warn them that Satellite services are unavailable for the duration of the backup. Ensure no other tasks are scheduled for the same time as the backup.
- Ensure that your backup location has enough disk space to store the backup. For more information, see Section 8.1.1, “Estimating the size of a Backup”.
Run the backup script:
# foreman-maintain backup offline --skip-pulp-content /var/backup_directory
8.1.4. Performing an Incremental Backup
To Perform an Incremental Backup:
Use this procedure to perform an offline backup of any changes since a previous backup. Use a full backup as a reference to make the first incremental backup of a sequence. Keep at least the last known good full backup and a complete sequence of incremental backups to restore from.
Request other users of Satellite Server or Capsule Server to save any changes and warn them that Satellite services are unavailable for the duration of the backup. Ensure no other tasks are scheduled for the same time as the backup.
- Ensure your backup location has enough disk space to store the backup. For more information, see Section 8.1.1, “Estimating the size of a Backup”.
Create a full backup:
# foreman-maintain backup offline /var/backup_directory
Run the backup script with the
--incremental
argument. This creates a directory within your backup directory to store the first incremental back up.# foreman-maintain backup offline --incremental /var/backup_directory/full_backup /var/backup_directory
Run the backup script again to create the second incremental backup. Point to your first incremental backup to indicate the starting point for the next increment. This creates a directory for the second incremental backup in your backup directory.
# foreman-maintain backup offline --incremental /var/backup_directory/first_incremental_backup /var/backup_directory
If you want to point to a different version of the backup, and make a series of increments with that version of the backup as the starting point, you can do this at any time. For example, if you want to make a new incremental backup from the full backup rather than the first or second incremental backup, point to the full backup directory:
# foreman-maintain backup offline --incremental /var/backup_directory/full_backup /var/backup_directory
8.1.5. Example of a Weekly Full Backup Followed by Daily Incremental Backups
A Weekly Full Backup Followed by Daily Incremental Backups
The script makes a full backup on a Sunday and incremental backups of the following days. Each day that a backup is made, a new subdirectory is created. This script requires a daily cron job.
#!/bin/bash -e PATH=/sbin:/bin:/usr/sbin:/usr/bin DESTINATION=/var/backup_directory if [[ $(date +%w) == 0 ]]; then foreman-maintain backup offline --assumeyes $DESTINATION else LAST=$(ls -td -- $DESTINATION/*/ | head -n 1) foreman-maintain backup offline --assumeyes --incremental "$LAST" $DESTINATION fi exit 0
Note that the foreman-maintain backup
script requires /sbin
and /usr/sbin
directories to be in PATH
, and the --assumeyes
option, because the command prompts for confirmation for the backup to proceed.
8.1.6. Performing an Online Backup
Perform an online backup only for debugging purposes. If there are procedures affecting the Pulp database, the Pulp part of the backup procedure repeats until it is no longer being altered. Because the backup of the Pulp database is the most time consuming part of backing up Satellite, if you make a change that alters the Pulp database during this time, the backup procedure keeps restarting.
Risks Associated with Online Backups
Data mismatches can occur between Mongo and Postgres databases while the services are online.
When performing an online backup, if there are procedures affecting the Pulp database, the Pulp part of the backup procedure repeats until it is no longer being altered. Because the backup of the Pulp database is the most time consuming part of backing up Satellite, if you make a change that alters the Pulp database during this time, the backup procedure keeps restarting.
For production environments, use the snapshot method. For more information, see Section 8.1.7, “Performing a Snapshot Backup”. If you want to use the online backup method in production, proceed with caution and ensure that no modifications occur during the backup.
Request other users of Satellite Server or Capsule Server to save any changes and warn them that Satellite services are unavailable for the duration of the backup. Ensure no other tasks are scheduled for the same time as the backup.
To Perform an Online Backup:
- Ensure that your backup location has enough disk space to store the backup. For more information, see Section 8.1.1, “Estimating the size of a Backup”.
Run the backup script:
# foreman-maintain backup online /var/backup_directory
8.1.7. Performing a Snapshot Backup
The snapshot backup method uses Logical Volume Manager (LVM) snapshots of the Pulp, MongoDB, and PostgreSQL directories. The backup is then created from the LVM snapshots and not from the running Satellite as with online backup, which mitigates the risk of creating an inconsistent backup. The snapshot backup method is faster than a full offline backup, which reduces Satellite downtime.
To Perform a Snapshot Backup:
To view the usage statement, enter the following command:
foreman-maintain backup snapshot -h
Prerequisites
Before you perform the snapshot backup, ensure that the following conditions exist:
-
The system uses LVM for the directories that you snapshot:
/var/lib/pulp/
,/var/lib/mongodb/
, and/var/lib/pgsql/
. - The free disk space in the relevant volume group (VG) is three times the size of the snapshot. More precisely, the VG must have enough space unreserved by the member logical volumes (LVs) to accommodate new snapshots. In addition, one of the LVs must have enough free space for the backup directory.
- The target backup directory is on a different LV than the directories that you snapshot.
Request other Satellite Server or Capsule Server users to save any changes and warn them that Satellite services are unavailable for the duration of the backup. Ensure no other tasks are scheduled for the same time as the backup.
Run the backup script:
# foreman-maintain backup snapshot /var/backup_directory
The foreman-maintain backup snapshot
makes snapshots when the services are active, and stops all services which could impact the backup. This makes the maintenance window shorter. After the successful snapshot, all services are restarted and LVM snapshots are removed.
8.1.8. White-listing and Skipping Steps
A backup using the foreman-maintain backup
script proceeds in a sequence of steps. To skip part of the backup add the --whitelist
option to the command and add the step label that you want to omit. For example:
# foreman-maintain backup online --whitelist backup-metadata -y /var/backup_directory
To see a list of available step labels use:
# foreman-maintain advanced procedure run -h