Chapter 3. Migrating from Internal Satellite Databases to External Databases
When you install Red Hat Satellite, the satellite-installer command installs MongoDB and PostgreSQL databases on the same server as Satellite. If you are using the default internal databases but want to start using external databases to help with the server load, you can migrate your internal databases to external databases. Depending on your requirements, you can use external databases for either MongoDB or PostgreSQL database, or both.
To confirm whether your Satellite Server has internal or external databases, you can query the status of your databases:
For PostgreSQL, enter the following command:
satellite-maintain service status --only postgresql
# satellite-maintain service status --only postgresql
For MongoDB, enter the following command:
satellite-maintain service status --only rh-mongodb34-mongod
# satellite-maintain service status --only rh-mongodb34-mongod
Use MongoDB as an External Database Considerations and PostgreSQL as an External Database Considerations to decide if you want to use external databases for your Satellite deployment.
Red Hat does not provide support or tools for external database maintenance. This includes backups, upgrades, and database tuning. You must have your own database administrator to support and maintain external databases.
To migrate from the default internal databases to external databases, you must complete the following procedures:
- Section 3.3, “Preparing a Host for External Databases”. Prepare a Red Hat Enterprise Linux 7 server to host the external databases.
-
Section 3.4, “Installing MongoDB”. Prepare MongoDB with user
pulp
owning thepulp_database
- Section 3.5, “Installing PostgreSQL”. Prepare PostgreSQL with databases for Satellite and Candlepin and dedicated users owning them.
-
Section 3.6, “Migrating to External Databases”. Edit the parameters of
satellite-installer
to point to the new databases, and runsatellite-installer
.
3.1. MongoDB as an External Database Considerations
Pulp uses the MongoDB database. If you want to use MongoDB as an external database, the following information can help you decide if this option is right for your Satellite configuration. Satellite supports MongoDB version 3.4.
Advantages of External MongoDB
- Increase in free memory and free CPU on Satellite
- Flexibility to tune the MongoDB server’s system without adversely affecting Satellite operations
Disadvantages of External MongoDB
- Increase in deployment complexity that can make troubleshooting more difficult
- An external MongoDB server is an additional system to patch and maintain
- If either the Satellite or the Mongo database server suffers a hardware or storage failure, Satellite is not operational
- If there is latency between the Satellite and the external database server, performance can suffer
FIPS-related Restrictions
- You cannot use an external MongoDB with Satellite in FIPS mode.
3.2. PostgreSQL as an External Database Considerations
Foreman, Katello, and Candlepin use the PostgreSQL database. If you want to use PostgreSQL as an external database, the following information can help you decide if this option is right for your Satellite configuration. Satellite supports PostgreSQL version 12.1.
Advantages of External PostgreSQL:
- Increase in free memory and free CPU on Satellite
-
Flexibility to set
shared_buffers
on the PostgreSQL database to a high number without the risk of interfering with other services on Satellite - Flexibility to tune the PostgreSQL server’s system without adversely affecting Satellite operations
Disadvantages of External PostgreSQL
- Increase in deployment complexity that can make troubleshooting more difficult
- The external PostgreSQL server is an additional system to patch and maintain
- If either Satellite or the PostgreSQL database server suffers a hardware or storage failure, Satellite is not operational
- If there is latency between the Satellite server and database server, performance can suffer
If you suspect that the PostgreSQL database on your Satellite is causing performance problems, use the information in Satellite 6: How to enable postgres query logging to detect slow running queries to determine if you have slow queries. Queries that take longer than one second are typically caused by performance issues with large installations, and moving to an external database might not help. If you have slow queries, contact Red Hat Support.
3.3. Preparing a Host for External Databases
Install a freshly provisioned system with the latest Red Hat Enterprise Linux 7 server to host the external databases.
Subscriptions for Red Hat Software Collections and Red Hat Enterprise Linux do not provide the correct service level agreement for using Satellite with external databases. You must also attach a Satellite subscription to the base operating system that you want to use for the external databases.
Prerequisites
- The Red Hat Enterprise Linux 7 server must meet Satellite’s Storage Requirements.
Procedure
- Use the instructions in Attaching the Satellite Infrastructure Subscription to attach a Satellite subscription to your server.
Disable all repositories and enable only the following repositories:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow subscription-manager repos --disable '*' subscription-manager repos --enable=rhel-server-rhscl-7-rpms \ --enable=rhel-7-server-rpms --enable=rhel-7-server-satellite-6.9-rpms
# subscription-manager repos --disable '*' # subscription-manager repos --enable=rhel-server-rhscl-7-rpms \ --enable=rhel-7-server-rpms --enable=rhel-7-server-satellite-6.9-rpms
3.4. Installing MongoDB
You can install only the same version of MongoDB that is installed with the satellite-installer
tool during an internal database installation. You can install MongoDB using Red Hat Software Collections (RHSCL) repositories or from an external source, as long as the version is supported. Satellite supports MongoDB version 3.4.
Procedure
To install MongoDB, enter the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow yum install rh-mongodb34 rh-mongodb34-syspaths
# yum install rh-mongodb34 rh-mongodb34-syspaths
Start and enable the rh-mongodb34 service:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow systemctl start rh-mongodb34-mongod systemctl enable rh-mongodb34-mongod
# systemctl start rh-mongodb34-mongod # systemctl enable rh-mongodb34-mongod
Create a Pulp user on MongoDB for database pulp_database:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow mongo pulp_database \ --eval "db.createUser({user:'pulp',pwd:'pulp_password',roles:[{role:'dbOwner', db:'pulp_database'},{ role: 'readWrite', db: 'pulp_database'}]})"
# mongo pulp_database \ --eval "db.createUser({user:'pulp',pwd:'pulp_password',roles:[{role:'dbOwner', db:'pulp_database'},{ role: 'readWrite', db: 'pulp_database'}]})"
In the
/etc/opt/rh/rh-mongodb34/mongod.conf
file, specify the bind IP:Copy to Clipboard Copied! Toggle word wrap Toggle overflow bindIp: your_mongodb_server_bind_IP,::1
bindIp: your_mongodb_server_bind_IP,::1
Edit the
/etc/opt/rh/rh-mongodb34/mongod.conf
file to enable authentication in thesecurity
section:Copy to Clipboard Copied! Toggle word wrap Toggle overflow security: authorization: enabled
security: authorization: enabled
Restart the
rh-mongodb34-mongod
service:Copy to Clipboard Copied! Toggle word wrap Toggle overflow systemctl restart rh-mongodb34-mongod
# systemctl restart rh-mongodb34-mongod
Open port 27017 for MongoDB:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow firewall-cmd --add-port=27017/tcp firewall-cmd --runtime-to-permanent
# firewall-cmd --add-port=27017/tcp # firewall-cmd --runtime-to-permanent
From Satellite Server, test that you can access the database. If the connection succeeds, the command returns
1
.Copy to Clipboard Copied! Toggle word wrap Toggle overflow scl enable rh-mongodb34 " mongo --host mongo.example.com \ -u pulp -p pulp_password --port 27017 --eval 'ping:1' pulp_database"
# scl enable rh-mongodb34 " mongo --host mongo.example.com \ -u pulp -p pulp_password --port 27017 --eval 'ping:1' pulp_database"
3.5. Installing PostgreSQL
You can install only the same version of PostgreSQL that is installed with the satellite-installer
tool during an internal database installation. You can install PostgreSQL using Red Hat Enterprise Linux Server 7 repositories or from an external source, as long as the version is supported. Satellite supports PostgreSQL version 12.1.
Procedure
To install PostgreSQL, enter the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow yum install rh-postgresql12-postgresql-server \ rh-postgresql12-syspaths \ rh-postgresql12-postgresql-evr
# yum install rh-postgresql12-postgresql-server \ rh-postgresql12-syspaths \ rh-postgresql12-postgresql-evr
To initialize PostgreSQL, enter the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow postgresql-setup initdb
# postgresql-setup initdb
Edit the
/var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf
file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf
# vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/postgresql.conf
Remove the
#
and edit to listen to inbound connections:Copy to Clipboard Copied! Toggle word wrap Toggle overflow listen_addresses = '*'
listen_addresses = '*'
Edit the
/var/opt/rh/rh-postgresql12/lib/pgsql/data/pg_hba.conf
file:Copy to Clipboard Copied! Toggle word wrap Toggle overflow vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/pg_hba.conf
# vi /var/opt/rh/rh-postgresql12/lib/pgsql/data/pg_hba.conf
Add the following line to the file:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow host all all Satellite_ip/24 md5
host all all Satellite_ip/24 md5
To start, and enable PostgreSQL service, enter the following commands:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow systemctl start postgresql systemctl enable postgresql
# systemctl start postgresql # systemctl enable postgresql
Open the postgresql port on the external PostgreSQL server:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow firewall-cmd --add-service=postgresql firewall-cmd --runtime-to-permanent
# firewall-cmd --add-service=postgresql # firewall-cmd --runtime-to-permanent
Switch to the
postgres
user and start the PostgreSQL client:Copy to Clipboard Copied! Toggle word wrap Toggle overflow su - postgres -c psql
$ su - postgres -c psql
Create three databases and dedicated roles: one for Satellite, one for Candlepin, and one for Pulp:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow CREATE USER "foreman" WITH PASSWORD 'Foreman_Password'; CREATE USER "candlepin" WITH PASSWORD 'Candlepin_Password'; CREATE USER "pulpcore" WITH PASSWORD 'Pulpcore_Password'; CREATE DATABASE foreman OWNER foreman; CREATE DATABASE candlepin OWNER candlepin; CREATE DATABASE pulpcore OWNER pulpcore;
CREATE USER "foreman" WITH PASSWORD 'Foreman_Password'; CREATE USER "candlepin" WITH PASSWORD 'Candlepin_Password'; CREATE USER "pulpcore" WITH PASSWORD 'Pulpcore_Password'; CREATE DATABASE foreman OWNER foreman; CREATE DATABASE candlepin OWNER candlepin; CREATE DATABASE pulpcore OWNER pulpcore;
Exit the
postgres
user:Copy to Clipboard Copied! Toggle word wrap Toggle overflow \q
# \q
From Satellite Server, test that you can access the database. If the connection succeeds, the commands return
1
.Copy to Clipboard Copied! Toggle word wrap Toggle overflow PGPASSWORD='Foreman_Password' psql -h postgres.example.com -p 5432 -U foreman -d foreman -c "SELECT 1 as ping" PGPASSWORD='Candlepin_Password' psql -h postgres.example.com -p 5432 -U candlepin -d candlepin -c "SELECT 1 as ping" PGPASSWORD='Pulpcore_Password' psql -h postgres.example.com -p 5432 -U pulpcore -d pulpcore -c "SELECT 1 as ping"
# PGPASSWORD='Foreman_Password' psql -h postgres.example.com -p 5432 -U foreman -d foreman -c "SELECT 1 as ping" # PGPASSWORD='Candlepin_Password' psql -h postgres.example.com -p 5432 -U candlepin -d candlepin -c "SELECT 1 as ping" # PGPASSWORD='Pulpcore_Password' psql -h postgres.example.com -p 5432 -U pulpcore -d pulpcore -c "SELECT 1 as ping"
3.6. Migrating to External Databases
Back up and transfer existing data, then use the satellite-installer
command to configure Satellite to connect to external MongoDB and PostgreSQL databases.
Prerequisites
- You have installed and configured MongoDB and PostgreSQL databases on a Red Hat Enterprise Linux server.
Procedure
On Satellite Server, stop the
satellite-maintain
services:Copy to Clipboard Copied! Toggle word wrap Toggle overflow satellite-maintain service stop
# satellite-maintain service stop
Start the postgreSQL and mongod services:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow systemctl start postgresql systemctl start mongod
# systemctl start postgresql # systemctl start mongod
Back up the internal databases:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow satellite-maintain backup online --skip-pulp-content --preserve-directory -y /var/migration_backup
# satellite-maintain backup online --skip-pulp-content --preserve-directory -y /var/migration_backup
Transfer the data to the new external databases:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow PGPASSWORD='Foreman_Password' pg_restore -h postgres.example.com -U foreman -d foreman < /var/migration_backup/foreman.dump PGPASSWORD='Candlepin_Password' pg_restore -h postgres.example.com -U candlepin -d candlepin < /var/migration_backup/candlepin.dump mongorestore --host mongo.example.com --db pulp_database --username pulp_user --password pulp_password /var/migration_backup/mongo_dump/pulp_database/
PGPASSWORD='Foreman_Password' pg_restore -h postgres.example.com -U foreman -d foreman < /var/migration_backup/foreman.dump PGPASSWORD='Candlepin_Password' pg_restore -h postgres.example.com -U candlepin -d candlepin < /var/migration_backup/candlepin.dump mongorestore --host mongo.example.com --db pulp_database --username pulp_user --password pulp_password /var/migration_backup/mongo_dump/pulp_database/
Use the
satellite-installer
command to update Satellite to point to the new databases:Copy to Clipboard Copied! Toggle word wrap Toggle overflow satellite-installer --scenario satellite \ --foreman-db-host postgres.example.com \ --foreman-db-password Foreman_Password \ --foreman-db-database foreman \ --foreman-db-manage false \ --katello-candlepin-db-host postgres.example.com \ --katello-candlepin-db-name candlepin \ --katello-candlepin-db-password Candlepin_Password \ --katello-candlepin-manage-db false \ --katello-pulp-db-username pulp \ --katello-pulp-db-password pulp_password \ --katello-pulp-db-seeds mongo.example.com:27017 \ --katello-pulp-db-name pulp_database \ --katello-pulp-manage-db false
satellite-installer --scenario satellite \ --foreman-db-host postgres.example.com \ --foreman-db-password Foreman_Password \ --foreman-db-database foreman \ --foreman-db-manage false \ --katello-candlepin-db-host postgres.example.com \ --katello-candlepin-db-name candlepin \ --katello-candlepin-db-password Candlepin_Password \ --katello-candlepin-manage-db false \ --katello-pulp-db-username pulp \ --katello-pulp-db-password pulp_password \ --katello-pulp-db-seeds mongo.example.com:27017 \ --katello-pulp-db-name pulp_database \ --katello-pulp-manage-db false