Install in an air-gapped environment

You can install self-service automation portal in a disconnected OpenShift Container Platform environment.

Installing the self-service automation portal in a disconnected or air-gapped OpenShift Container Platform environment requires preparation and configuration to ensure all necessary container images and assets are available locally without internet access.

Prerequisites
Copy link

Gather the required tools and access credentials necessary for air-gapped installation. This includes the OpenShift CLI, Helm, Podman, and required registry secrets.

  • You have installed the OpenShift CLI (oc).
  • You have installed Helm 3.10 or newer.
  • You have installed and configured Podman for pulling and pushing container images.
  • You have internet access. This is required to pull images and charts from public repositories, including registry.redhat.io and charts.openshift.io.
  • A Red Hat pull secret, for exmaple pull-secret.json or similar credentials file that allows you to pull images from registry.redhat.io.
  • Sufficient disk space to store downloaded images and chart packages.
  • Access to public registries: Docker Hub, quay.io, registry.redhat.io, and your disconnected OpenShift cluster’s internal registry.

Prepare for air-gapped installation
Copy link

Before you can install self-service automation portal in a disconnected OpenShift Container Platform environment, you must complete some processes on a connected bastion host.

Mirror container images
Copy link

Mirror the required container images from the Red Hat registry to your local disconnected registry. This step prepares the images for installing self-service automation portal in an isolated environment.

About this task
Copy link

If you mirror registry.redhat.io content to a different registry host (or to a registry prefix such as quay.io/<org>), you can set redhat-developer-hub.global.imageRegistry so the Helm chart pulls all of its images from that mirrored location. imageRegistry is a single override that controls the registry for the base application image, PostgreSQL image, OCI plug-in artifacts, and Ansible Dev Tools sidecar.

The dynamic plug-in init container does not use cluster-level image mirror configuration (for example, ImageDigestMirrorSet or ImageTagMirrorSet). You must set imageRegistry even if your cluster redirects registry.redhat.io pulls.

Procedure
Copy link

  1. Log in to registry.redhat.io: 
    $ podman login registry.redhat.io
  2. Enter your Red Hat username and password when prompted.
  3. Log in to your disconnected registry: 
    $ podman login <disconnected_registry_url>
  4. Pull the Red Hat Developer Hub image from registry.redhat.io: 
    $ podman pull registry.redhat.io/rhdh/rhdh-hub-rhel9:x.y.z
  5. Tag the image for your disconnected registry: 
    $ podman tag registry.redhat.io/rhdh/rhdh-hub-rhel9:x.y.z <disconnected_registry_url>/<your_namespace>/rhdh/rhdh-hub-rhel9:x.y.z
  6. Push the tagged image to your disconnected registry: 
    $ podman push <disconnected_registry_url>/<your_namespace>/rhdh/rhdh-hub-rhel9:x.y.z
  7. If you use the built-in PostgreSQL database, mirror the PostgreSQL 15 image. An external database is the supported production architecture and does not require this step: 
    $ podman pull registry.redhat.io/rhel9/postgresql-15:<tag>
$ podman tag registry.redhat.io/rhel9/postgresql-15:<tag> <disconnected_registry_url>/<your_namespace>/rhel9/postgresql-15:<tag>
$ podman push <disconnected_registry_url>/<your_namespace>/rhel9/postgresql-15:<tag>
  8. If you use OCI container delivery, mirror the Ansible plug-ins OCI artifacts image: 
    $ podman pull registry.redhat.io/ansible-automation-platform/automation-portal:<plugin-version>
$ podman tag registry.redhat.io/ansible-automation-platform/automation-portal:<plugin-version> <disconnected_registry_url>/<your_namespace>/ansible-automation-platform/automation-portal:<plugin-version>
$ podman push <disconnected_registry_url>/<your_namespace>/ansible-automation-platform/automation-portal:<plugin-version>
  9. Mirror the Ansible Dev Tools sidecar image. Use the path that matches your Red Hat Ansible Automation Platform version:

    Red Hat Ansible Automation Platform 2.5:

    $ podman pull registry.redhat.io/ansible-automation-platform-25/ansible-dev-tools-rhel9:latest
$ podman tag registry.redhat.io/ansible-automation-platform-25/ansible-dev-tools-rhel9:latest <disconnected_registry_url>/<your_namespace>/ansible-automation-platform-25/ansible-dev-tools-rhel9:latest
$ podman push <disconnected_registry_url>/<your_namespace>/ansible-automation-platform-25/ansible-dev-tools-rhel9:latest

    Red Hat Ansible Automation Platform 2.6:

    $ podman pull registry.redhat.io/ansible-automation-platform-26/ansible-dev-tools-rhel9:latest
$ podman tag registry.redhat.io/ansible-automation-platform-26/ansible-dev-tools-rhel9:latest <disconnected_registry_url>/<your_namespace>/ansible-automation-platform-26/ansible-dev-tools-rhel9:latest
$ podman push <disconnected_registry_url>/<your_namespace>/ansible-automation-platform-26/ansible-dev-tools-rhel9:latest

Download the helm chart package
Copy link

Download the Helm chart package and modify the internal image references to point to your disconnected registry. This prepares the installation package for the air-gapped environment.

Procedure
Copy link

  1. Add the OpenShift Helm charts repository: 
    $ helm repo add openshift-helm-charts https://charts.openshift.io/
  2. Update your Helm repositories to fetch the latest chart information: 
    $ helm repo update
  3. Pull the chart: 
    $ helm pull openshift-helm-charts/redhat-rhaap-portal --version x.y.z

    This command downloads the chart as a .tgz file, for example redhat-rhaap-portal-1.0.1.tgz.

  4. Unpack the chart: 
    $ tar -xvf redhat-rhaap-portal-x.y.z.tgz

    This creates a directory with a name similar to redhat-rhaap-portal-1.0.1/.

  5. Navigate to the unpacked chart directory (for example, cd redhat-rhaap-portal-1.0.1) and open the values.yaml file in a text editor.
  6. Find all the image: entries in values.yaml and replace the original image references with the full path to the image in your disconnected registry.

    For example, replace image: registry.redhat.io/rhdh/rhdh-hub-rhel9:x.y.z with image: <disconnected_registry_url>/<your_namespace>/rhdh-hub-rhel9:x.y.z

  7. Repack the modified chart: 
    $ helm package redhat-rhaap-portal-x.y.z

    This creates a new .tgz file with your changes (for example, redhat-rhaap-portal-1.0.1.tgz).

Transfer assets to the disconnected environment
Copy link

Transfer the modified Helm chart package from the connected bastion host to a machine inside your disconnected network. This action stages the installation assets for deployment within the isolated OpenShift environment.

Procedure
Copy link

Copy the modified Helm chart .tgz file or files (for example, redhat-rhaap-portal-1.0.1.tgz) from your connected bastion host to a machine or jump box within your disconnected OpenShift network.

Install the Helm chart in the disconnected OpenShift environment
Copy link

You can install the modified Helm chart using the helm install command in your disconnected OpenShift environment. This deploys the self-service automation portal using the locally available assets.

After preparing the disconnected environment with mirrored images and transferred assets, install the Helm chart to deploy the self-service automation portal.

Access the disconnected OpenShift environment
Copy link

Ensure your disconnected OpenShift cluster is configured to trust the private registry containing the mirrored container images. This step is crucial for successful image pulling during installation.

Before you begin
Copy link

  • You have the necessary kubeconfig and permissions. For example cluster-admin, for setting up image pull secrets or insecure registries.

About this task
Copy link

Procedure
Copy link

  1. In a terminal, log in to your disconnected OpenShift cluster using the oc CLI. 
    oc login --token=<your_token> --server=<your_openshift_api_url>

    Use the following command if you have a kubeconfig:

    export KUBECONFIG=/path/to/your/kubeconfig
oc login
  2. Ensure that your OpenShift cluster is configured to trust your disconnected registry:
    1. Use ImageContentSourcePolicy for mirroring.
    2. Use additionalTrustedCA in image.config.openshift.io/cluster for self-signed certificates.
    3. Use insecure-registries for plain HTTP.

Install the Helm chart
Copy link

Install self-service automation portal by using the helm install command. You must reference the local Helm chart file and include the required configuration using a values file (-f) or --set flags.

Procedure
Copy link

  1. On the machine within your disconnected environment, navigate to the directory where you placed the transferred Helm chart .tgz file: 
    $ cd /path/to/your/transferred/charts/
  2. Define your namespace and cluster router base as environment variables: 
    $ export MY_NAMESPACE="<your_namespace_name>"
$ export MY_CLUSTER_ROUTER_BASE="<your_cluster_router_base>"
  3. If the namespace does not exist, create it: 
    $ oc new-project "${MY_NAMESPACE}"
  4. Create a custom values file and reuse it for install and upgrade. Choose the example that matches your plug-in delivery method.

    OCI delivery (values-oci.yaml):

    redhat-developer-hub:
  global:
    clusterRouterBase: <your_cluster_router_base>
    pluginMode: oci
    imageTagInfo: "<plugin-version>"
    # Registry host only. Replaces registry.redhat.io for all images.
    imageRegistry: "<disconnected_registry_url>"
    # Optional. Full OCI plugin image path for mirrors with non-standard
    # repository paths. Overrides imageRegistry for OCI plugin artifacts only.
    # Use when your mirror does not preserve the default repository path
    # (ansible-automation-platform/automation-portal).
    # ociPluginImage: "<disconnected_registry_url>/custom-path/automation-portal"
  upstream:
    backstage:
      # The Helm chart pins images by SHA256 digest by default.
      # When mirroring with podman tag/push, the digest is not preserved.
      # Override with tag-based references.
      image:
        repository: rhdh/rhdh-hub-rhel9
        tag: "1.8"
      appConfig:
        ansible:
          rhaap:
            checkSSL: true  # Set to false if using self-signed certificates.
        catalog:
          providers:
            rhaap:
              '{{- include "catalog.providers.env" . }}':
                orgs: "<your-aap-organization-name>"
    # If you use the built-in PostgreSQL database, uncomment the following lines.
    # An external database is the supported production architecture.
    # postgresql:
    #   image:
    #     repository: rhel9/postgresql-15
    #     tag: "latest"

    HTTP plug-in registry delivery (values-tarball.yaml):

    redhat-developer-hub:
  global:
    clusterRouterBase: <your_cluster_router_base>
    pluginMode: tarball
    # Registry host only. Replaces registry.redhat.io for the images.
    imageRegistry: "<disconnected_registry_url>"
  upstream:
    backstage:
      image:
        repository: rhdh/rhdh-hub-rhel9
        tag: "1.8"
      appConfig:
        ansible:
          rhaap:
            checkSSL: true  # Set to false if using self-signed certificates.
        catalog:
          providers:
            rhaap:
              '{{- include "catalog.providers.env" . }}':
                orgs: "<your-aap-organization-name>"
    # If you use the built-in PostgreSQL database, uncomment the following lines.
    # An external database is the supported production architecture.
    # postgresql:
    #   image:
    #     repository: rhel9/postgresql-15
    #     tag: "latest"
    Important

    Set imageRegistry to the registry host only (for example, mirror.example.com). Do not include repository paths. The chart appends the default repository path ansible-automation-platform/automation-portal automatically. If your mirror uses a different repository structure, set ociPluginImage to the full image path instead.
  5. Install the chart using the helm install command: 
    $ helm install redhat-rhaap-portal \
  redhat-rhaap-portal-x.y.z.tgz \
  --namespace "${MY_NAMESPACE}" \
  -f /path/to/values-oci.yaml

    To apply changes after deployment, upgrade the Helm release. If you install from the OpenShift web console, use Developer Helm select the release Actions Upgrade YAML view.

    To upgrade from the command line:

    $ helm upgrade redhat-rhaap-portal \
  redhat-rhaap-portal-x.y.z.tgz \
  --namespace "${MY_NAMESPACE}" \
  -f /path/to/values-oci.yaml

    Alternatively, you can pass values using --set flags.

    OCI delivery:

    $ helm install redhat-rhaap-portal \
  redhat-rhaap-portal-x.y.z.tgz \
  --namespace "${MY_NAMESPACE}" \
  --set redhat-developer-hub.global.clusterRouterBase="${MY_CLUSTER_ROUTER_BASE}" \
  --set redhat-developer-hub.global.imageRegistry="<disconnected_registry_url>" \
  --set redhat-developer-hub.global.pluginMode=oci

    To also set ociPluginImage via --set:

      --set redhat-developer-hub.global.ociPluginImage="<disconnected_registry_url>/custom-path/automation-portal"

    HTTP plug-in registry delivery:

    $ helm install redhat-rhaap-portal \
  redhat-rhaap-portal-x.y.z.tgz \
  --namespace "${MY_NAMESPACE}" \
  --set redhat-developer-hub.global.clusterRouterBase="${MY_CLUSTER_ROUTER_BASE}" \
  --set redhat-developer-hub.global.imageRegistry="<disconnected_registry_url>" \
  --set redhat-developer-hub.global.pluginMode=tarball

Configure CA certificates for private registries
Copy link

If your private registry uses a certificate signed by an internal or self-signed CA, the install-dynamic-plugins init container fails with x509: certificate signed by unknown authority. Mount the CA certificate into the init container so that skopeo trusts the registry.

Procedure
Copy link

  1. Obtain the CA certificate chain that signed your mirror registry's TLS certificate. If the registry uses a certificate chain, include the full chain: 
    $ cat registry.crt intermediate.crt root.crt > ca-bundle.crt
  2. Create a ConfigMap from the CA certificate: 
    $ oc create configmap registry-ca-crt \
  --from-file=ca.crt=ca-bundle.crt -n <namespace>
  3. Add the volume and mount to your values file: 
    redhat-developer-hub:
  upstream:
    backstage:
      extraVolumes:
        - name: registry-ca-crt
          configMap:
            name: registry-ca-crt
      initContainers:
        - name: install-dynamic-plugins
          volumeMounts:
            - name: registry-ca-crt
              mountPath: /etc/containers/certs.d/<registry-host>
              readOnly: true
    Important

    The mountPath must be the registry hostname only (for example, /etc/containers/certs.d/mirror.example.com). Do not include repository paths. If the registry uses a non-standard port, include it in the path (for example, /etc/containers/certs.d/mirror.example.com:5000).

Verify the deployment
Copy link

Verify the successful installation of the Helm chart in the disconnected environment. Check the Helm release status, monitor the pods, and verify that the application routes are accessible.

Procedure
Copy link

  1. Check the Helm release status: 
    $ helm list -n ${MY_NAMESPACE}
  2. Monitor the pods in your namespace to ensure they are running: 
    $ oc get pods -n ${MY_NAMESPACE}
  3. Check for ImagePullBackOff or other errors in pod events: 
    $ oc describe pod <pod_name> -n ${MY_NAMESPACE}
  4. If the chart uses routes to expose services, verify that the routes are created and accessible: 
    $ oc get route -n ${MY_NAMESPACE}

Troubleshooting disconnected installations
Copy link

Use this reference to troubleshoot common issues that occur during disconnected self-service automation portal installations.

Expand
Symptom Cause Solution
authentication required or unauthorized in install-dynamic-plugins init container logs Auth secret missing or malformed. The init container uses skopeo and does not use cluster pull secrets. Create <release-name>-dynamic-plugins-registry-auth secret. Use base64 -w0 to avoid multiline values that corrupt auth.json.
Duplicate path in OCI URL (for example, .../ansible-automation-platform/ansible-automation-platform/...) imageRegistry includes a repository path instead of the registry host only. Set imageRegistry to the registry host only. If your mirror uses a different repository structure, use ociPluginImage to set the full image path.
x509: certificate signed by unknown authority in init container logs Private registry uses a self-signed or internal CA certificate. Mount the CA certificate into the init container. See Configure CA certificates for private registries.