Red Hat Summitこのコンテンツは選択した言語では利用できません。
Chapter 4. Restoring the PostgreSQL database for Red Hat Edge Manager on Red Hat OpenShift Container Platform
After you lose data or need to recover the control plane database on Red Hat OpenShift Container Platform, restore the flightctl PostgreSQL database and related state using the backup artifacts and methods that match your backup scope. This topic walks through scaling workloads, running flightctl-restore, and bringing services back. Exact database restore commands before flightctl-restore still depend on your backup format and tooling.
4.1. Prerequisites
- Cluster access to the Kubernetes cluster that hosts the Red Hat Edge Manager deployment.
-
The OpenShift project (Kubernetes namespace) where you installed the Red Hat Edge Manager Helm chart. The examples in this topic use the placeholder
rhem-chart-namespace; substitute your real namespace name everywhere it appears (production deployments often use a single namespace for all Red Hat Edge Manager workloads). -
Kubernetes tools:
kubectlinstalled and configured with administrative permissions. -
Flight Control CLI:
flightctlandflightctl-restoreavailable locally or in your restore environment. - Backup artifacts: access to the database backup files required for recovery.
-
Optional verification tools:
redis-cliandpg_isreadyfor validating service readiness and data integrity.
Database restore steps before you run flightctl-restore must match your backup strategy (for example, pg_restore, psql, or volume-level recovery). Follow your organization’s runbooks together with the outline below.
4.2. Restore procedure
Verify that the
flightctl-restoreversion matches the Red Hat Edge Manager server version:Run the following command for the
flightctlversion:flightctl versionRun the following command for the
flightctl-restoreversion:flightctl-restore versionConfirm that the server version and the restore version are the same:
echo "Server version: $(flightctl version)" echo "Restore version: $(flightctl-restore version)"ImportantThe server version and restore version must match exactly. If they do not match, update your
flightctl-restorebinary to align with the server version before proceeding.
Scale down the Red Hat Edge Manager services to avoid data conflicts during the restore process. On Red Hat OpenShift Container Platform, all of these workloads run in a single OpenShift project; use the same namespace value on every command (the placeholder
rhem-chart-namespacestands for that project—replace it with yours):# Replace rhem-chart-namespace with your OpenShift project (one namespace for all deployments below) kubectl scale deployment flightctl-api --replicas=0 -n rhem-chart-namespace kubectl scale deployment flightctl-worker --replicas=0 -n rhem-chart-namespace kubectl scale deployment flightctl-periodic --replicas=0 -n rhem-chart-namespace kubectl scale deployment flightctl-alert-exporter --replicas=0 -n rhem-chart-namespace kubectl scale deployment flightctl-alertmanager-proxy --replicas=0 -n rhem-chart-namespace
Wait for pods to terminate, then verify:
+
# Same namespace as above
kubectl get pods -n rhem-chart-namespaceRestore the
flightctlPostgreSQL database from your existing backup.After all Red Hat Edge Manager services are scaled down, restore the database using the method that matches your backup strategy.
-
Target database: Restore the PostgreSQL database instance named
flightctl. -
Supported methods: Use your preferred recovery procedure, such as
pg_restore,psql(for SQL dumps), or infrastructure-level volume snapshots. - Verification: Confirm that the database is fully accessible and that the integrity of the restored data is verified before proceeding.
ImportantThe specific restoration commands depend on your backup strategy and tooling. Ensure the database is fully restored and consistent before you continue.
-
Target database: Restore the PostgreSQL database instance named
Retrieve database and KV store credentials (same namespace as the scale commands):
DB_APP_PASSWORD=$(kubectl get secret flightctl-db-app-secret -n rhem-chart-namespace -o jsonpath='{.data.userPassword}' | base64 -d) echo "Database password retrieved successfully"KV_PASSWORD=$(kubectl get secret flightctl-kv-secret -n rhem-chart-namespace -o jsonpath='{.data.password}' | base64 -d)Set up port forwarding for the database and the KV store. Use separate terminal sessions for each port forward, or run them in the background.
Forward the database service:
# Forward database port (run in a separate terminal or in the background) kubectl port-forward svc/flightctl-db 5432:5432 -n rhem-chart-namespace & DB_PORT_FORWARD_PID=$! # Verify database connectivity (if available) pg_isready -h localhost -p 5432Forward the KV store service:
# Forward KV store port (run in a separate terminal or in the background) kubectl port-forward svc/flightctl-kv 6379:6379 -n rhem-chart-namespace & KV_PORT_FORWARD_PID=$! # Verify KV store connectivity (if available) REDISCLI_AUTH="$KV_PASSWORD" redis-cli -h localhost -p 6379 pingRun the restore command using environment variables for the database and KV store passwords:
DB_PASSWORD="$DB_APP_PASSWORD" KV_PASSWORD="$KV_PASSWORD" ./bin/flightctl-restoreMonitor the restore output for errors or completion messages.
Stop the port-forward processes when the restore finishes:
kill $DB_PORT_FORWARD_PID $KV_PORT_FORWARD_PIDIf you ran the port forwards in separate terminals instead, stop them with Ctrl+C in those terminals.
Restart Red Hat Edge Manager services. Scale the deployments back to their normal replica counts in the same OpenShift project as in step 2:
# Replace rhem-chart-namespace with your OpenShift project (same single namespace for every command) kubectl scale deployment flightctl-api --replicas=1 -n rhem-chart-namespace kubectl scale deployment flightctl-worker --replicas=1 -n rhem-chart-namespace kubectl scale deployment flightctl-periodic --replicas=1 -n rhem-chart-namespace kubectl scale deployment flightctl-alert-exporter --replicas=1 -n rhem-chart-namespace kubectl scale deployment flightctl-alertmanager-proxy --replicas=1 -n rhem-chart-namespace kubectl get deployments -n rhem-chart-namespace kubectl get pods -n rhem-chart-namespace
4.3. After you restore
When the restore finishes and Red Hat Edge Manager workloads are healthy again in your Red Hat OpenShift Container Platform namespace, validate the control plane and plan for device reconciliation. Restoring the database changes what the service knows about devices; edge devices must reconnect and compare their live state to the restored specifications.
4.3.1. Operational follow-up
- Re-run any validation steps from Testing backups in the backup topic if you need a structured checklist.
- Document deviations, incidents, and command variants in your runbooks so the next restore follows the same path.
4.3.2. Post-restore device status changes
After a successful restore, devices move through automatic status transitions while they reconnect and reconcile with the restored control plane data.
- AwaitingReconnect
-
Devices are always placed in
AwaitingReconnectfirst. The service waits for each device to report its current state again. Spec reconciliation for those devices remains paused until they reconnect. - Enrollment requests and post-restore approval
Devices approved after the restored backup was taken do not exist after the restore and must be approved again. After restore:
-
Devices created from a restored enrollment request are placed in
AwaitingReconnectand follow the normalAwaitingReconnectbehavior. -
Devices without an enrollment request before backup, with a non-zero deployed specification version, are placed in
AwaitingReconnectand follow the normalAwaitingReconnectbehavior. - Devices without an enrollment request before backup, with a zero specification version, move to normal status.
-
Devices created from a restored enrollment request are placed in
- ConflictPaused
-
After a device reconnects and reports its current state, the service compares the specification stored in the restored backup with the device-reported version. If the restored backup specification is older (for example, the device had moved forward while backups lagged), the device can enter
ConflictPaused. Rendering of new specifications stops for that device until an operator resolves the mismatch. Human review is required before you force configuration forward. - Normal operation
- When the restored specification and the device-reported state are compatible, the device returns to normal operational statuses (for example, online or updating) and usual reconciliation resumes.
4.3.2.1. Monitor device status
Use the flightctl CLI to see which devices need attention:
flightctl get devices
flightctl get devices --field-selector=status.summary.status=AwaitingReconnect
flightctl get devices --field-selector=status.summary.status=ConflictPaused4.3.2.2. Resolve ConflictPaused devices
- Review the specification source: if the device belongs to a fleet, inspect the fleet template and selector; if not, inspect the device spec directly. Review labels and ownership to confirm how the restored specification applies to the device.
When you are confident the restored specification is what you want, resume the device or a group of devices. Replace
example-devicewith your device resource name and adjust selectors to match your environment:flightctl resume device example-device flightctl resume device --selector="environment=production"Use additional
flightctl resume deviceoptions your deployment supports (for example, field selectors) if you need to resume many devices in bulk.
'%20d='M201.5%20305.5c-13.8%200-24.9-11.1-24.9-24.6%200-13.8%2011.1-24.9%2024.9-24.9%2013.6%200%2024.6%2011.1%2024.6%2024.9%200%2013.6-11.1%2024.6-24.6%2024.6zM504%20256c0%20137-111%20248-248%20248S8%20393%208%20256%20119%208%20256%208s248%20111%20248%20248zm-132.3-41.2c-9.4%200-17.7%203.9-23.8%2010-22.4-15.5-52.6-25.5-86.1-26.6l17.4-78.3%2055.4%2012.5c0%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.3%2024.9-24.9s-11.1-24.9-24.9-24.9c-9.7%200-18%205.8-22.1%2013.8l-61.2-13.6c-3-.8-6.1%201.4-6.9%204.4l-19.1%2086.4c-33.2%201.4-63.1%2011.3-85.5%2026.8-6.1-6.4-14.7-10.2-24.1-10.2-34.9%200-46.3%2046.9-14.4%2062.8-1.1%205-1.7%2010.2-1.7%2015.5%200%2052.6%2059.2%2095.2%20132%2095.2%2073.1%200%20132.3-42.6%20132.3-95.2%200-5.3-.6-10.8-1.9-15.8%2031.3-16%2019.8-62.5-14.9-62.5zM302.8%20331c-18.2%2018.2-76.1%2017.9-93.6%200-2.2-2.2-6.1-2.2-8.3%200-2.5%202.5-2.5%206.4%200%208.6%2022.8%2022.8%2087.3%2022.8%20110.2%200%202.5-2.2%202.5-6.1%200-8.6-2.2-2.2-6.1-2.2-8.3%200zm7.7-75c-13.6%200-24.6%2011.1-24.6%2024.9%200%2013.6%2011.1%2024.6%2024.6%2024.6%2013.8%200%2024.9-11.1%2024.9-24.6%200-13.8-11-24.9-24.9-24.9z'/%3e%3c/svg%3e){kind=small}