Chapter 2. Installing a cluster on Oracle Cloud Infrastructure (OCI) by using the Agent-based Installer

1. Create OCI resources and services (Oracle).
2. Disconnected environments: Prepare a web server that is accessible by OCI instances (Red Hat).
3. Prepare configuration files for the Agent-based Installer (Red Hat).
4. Generate the agent ISO image (Red Hat).
5. Disconnected environments: Upload the rootfs image to the web server (Red Hat).
6. Configure your firewall for OpenShift Container Platform (Red Hat).
7. Upload the agent ISO image to a storage bucket (Oracle).
8. Create a custom image from the uploaded agent ISO image (Oracle).
9. Create compute instances on OCI (Oracle).
10. Verify that your cluster runs on OCI (Oracle).

Procedure

1. Create an installation directory to store configuration files in by running the following command:

   $ mkdir ~/<directory_name>

2. Configure the install-config.yaml configuration file to meet the needs of your organization and save the file in the directory you created.

   install-config.yaml file that sets an external platform

   # install-config.yaml
   apiVersion: v1
   baseDomain: <base_domain> 1
   networking:
     clusterNetwork:
     - cidr: 10.128.0.0/14
       hostPrefix: 23
     network type: OVNKubernetes
     machineNetwork:
     - cidr: <ip_address_from_cidr> 2
     serviceNetwork:
     - 172.30.0.0/16
   compute:
     - architecture: amd64 3
     hyperthreading: Enabled
     name: worker
     replicas: 0
   controlPlane:
     architecture: amd64 4
     hyperthreading: Enabled
     name: master
     replicas: 3
   platform:
      external:
       platformName: oci 5
       cloudControllerManager: External
   sshKey: <public_ssh_key> 6
   pullSecret: '<pull_secret>' 7
   # ...

   1

   The base domain of your cloud provider.

   2

   The IP address from the virtual cloud network (VCN) that the CIDR allocates to resources and components that operate on your network.

   3 4

   Depending on your infrastructure, you can select either arm64 or amd64.

   5

   Set OCI as the external platform, so that OpenShift Container Platform can integrate with OCI.

   6

   Specify your SSH public key.

   7

   The pull secret that you need for authenticate purposes when downloading container images for OpenShift Container Platform components and services, such as Quay.io. See Install OpenShift Container Platform 4 from the Red Hat Hybrid Cloud Console.

3. Create a directory on your local system named openshift. This must be a subdirectory of the installation directory.

   Important

   Do not move the install-config.yaml or agent-config.yaml configuration files to the openshift directory.

4. If you used a stack to provision OCI infrastructure resources: Copy and paste the dynamic_custom_manifest output of the OCI stack into a file titled manifest.yaml and save the file in the openshift directory.

5. If you did not use a stack to provision OCI infrastructure resources: Download and prepare custom manifests to create an Agent ISO image:

   1. Go to Configuration Files (Oracle documentation) and follow the link to the custom manifests directory on GitHub.
   2. Copy the contents of the condensed-manifest.yml file and save it locally to a file in the openshift directory.
   3. In the condensed-manifest.yml file, update the sections marked with TODO to specify the compartment Oracle® Cloud Identifier (OCID), VCN OCID, subnet OCID from the load balancer, and the security lists OCID.

6. Configure the agent-config.yaml configuration file to meet your organization’s requirements.

   Sample agent-config.yaml file for an IPv4 network.

   apiVersion: v1beta1
   metadata:
     name: <cluster_name> 1
     namespace: <cluster_namespace> 2
   rendezvousIP: <ip_address_from_CIDR> 3
   bootArtifactsBaseURL: <server_URL> 4
   # ...

   1

   The cluster name that you specified in your DNS record.

   2

   The namespace of your cluster on OpenShift Container Platform.

   3

   If you use IPv4 as the network IP address format, ensure that you set the rendezvousIP parameter to an IPv4 address that the VCN’s Classless Inter-Domain Routing (CIDR) method allocates on your network. Also ensure that at least one instance from the pool of instances that you booted with the ISO matches the IP address value you set for the rendezvousIP parameter.

   4

   The URL of the server where you want to upload the rootfs image. This parameter is required only for disconnected environments.

7. Generate a minimal ISO image, which excludes the rootfs image, by entering the following command in your installation directory:

   $ ./openshift-install agent create image --log-level debug

   The command also completes the following actions:

   • Creates a subdirectory, ./<installation_directory>/auth directory:, and places kubeadmin-password and kubeconfig files in the subdirectory.
   • Creates a rendezvousIP file based on the IP address that you specified in the agent-config.yaml configuration file.

   • Optional: Any modifications you made to agent-config.yaml and install-config.yaml configuration files get imported to the Zero Touch Provisioning (ZTP) custom resources.

     Important

     The Agent-based Installer uses Red Hat Enterprise Linux CoreOS (RHCOS). The rootfs image, which is mentioned in a later step, is required for booting, recovering, and repairing your operating system.

8. Disconnected environments only: Upload the rootfs image to a web server.

   1. Go to the ./<installation_directory>/boot-artifacts directory that was generated when you created the minimal ISO image.

   2. Use your preferred web server, such as any Hypertext Transfer Protocol daemon (httpd), to upload the rootfs image to the location specified in the bootArtifactsBaseURL parameter of the agent-config.yaml file.

      For example, if the bootArtifactsBaseURL parameter states http://192.168.122.20, you would upload the generated rootfs image to this location so that the Agent-based installer can access the image from http://192.168.122.20/agent.x86_64-rootfs.img. After the Agent-based installer boots the minimal ISO for the external platform, the Agent-based Installer downloads the rootfs image from the http://192.168.122.20/agent.x86_64-rootfs.img location into the system memory.

      Note

      The Agent-based Installer also adds the value of the bootArtifactsBaseURL to the minimal ISO Image’s configuration, so that when the Operator boots a cluster’s node, the Agent-based Installer downloads the rootfs image into system memory.

      Important

      Consider that the full ISO image, which is in excess of 1 GB, includes the rootfs image. The image is larger than the minimal ISO Image, which is typically less than 150 MB.

Procedure

1. Set the following registry URLs for your firewall’s allowlist:

   URL	Port	Function
   registry.redhat.io	443	Provides core container images
   access.redhat.com	443	Hosts a signature store that a container client requires for verifying images pulled from registry.access.redhat.com. In a firewall environment, ensure that this resource is on the allowlist.
   registry.access.redhat.com	443	Hosts all the container images that are stored on the Red Hat Ecosystem Catalog, including core container images.
   quay.io	443	Provides core container images
   cdn.quay.io	443	Provides core container images
   cdn01.quay.io	443	Provides core container images
   cdn02.quay.io	443	Provides core container images
   cdn03.quay.io	443	Provides core container images
   cdn04.quay.io	443	Provides core container images
   cdn05.quay.io	443	Provides core container images
   cdn06.quay.io	443	Provides core container images
   sso.redhat.com	443	The https://console.redhat.com site uses authentication from sso.redhat.com

   • You can use the wildcards *.quay.io and *.openshiftapps.com instead of cdn.quay.io and cdn0[1-6].quay.io in your allowlist.
   • You can use the wildcard *.access.redhat.com to simplify the configuration and ensure that all subdomains, including registry.access.redhat.com, are allowed.
   • When you add a site, such as quay.io, to your allowlist, do not add a wildcard entry, such as *.quay.io, to your denylist. In most cases, image registries use a content delivery network (CDN) to serve images. If a firewall blocks access, image downloads are denied when the initial download request redirects to a hostname such as cdn01.quay.io.
2. Set your firewall’s allowlist to include any site that provides resources for a language or framework that your builds require.

3. If you do not disable Telemetry, you must grant access to the following URLs to access Red Hat Insights:

   URL	Port	Function
   cert-api.access.redhat.com	443	Required for Telemetry
   api.access.redhat.com	443	Required for Telemetry
   infogw.api.openshift.com	443	Required for Telemetry
   console.redhat.com	443	Required for Telemetry and for insights-operator

4. Set your firewall’s allowlist to include the following registry URLs:

   URL	Port	Function
   api.openshift.com	443	Required both for your cluster token and to check if updates are available for the cluster.
   rhcos.mirror.openshift.com	443	Required to download Red Hat Enterprise Linux CoreOS (RHCOS) images.

5. Set your firewall’s allowlist to include the following external URLs. Each repository URL hosts OCI containers. Consider mirroring images to as few repositories as possible to reduce any performance issues.

   URL	Port	Function
   k8s.gcr.io	port	A Kubernetes registry that hosts container images for a community-based image registry. This image registry is hosted on a custom Google Container Registry (GCR) domain.
   ghcr.io	port	A GitHub image registry where you can store and manage Open Container Initiative images. Requires an access token to publish, install, and delete private, internal, and public packages.
   storage.googleapis.com	443	A source of release image signatures, although the Cluster Version Operator needs only a single functioning source.
   registry.k8s.io	port	Replaces the k8s.gcr.io image registry because the k8s.gcr.io image registry does not support other platforms and vendors.

Procedure

1. Upload the agent ISO image to Oracle’s default Object Storage bucket and import the agent ISO image as a custom image to this bucket. Ensure you that you configure the custom image to boot in Unified Extensible Firmware Interface (UEFI) mode. For more information, see Creating the OpenShift Container Platform ISO Image (Oracle documentation).

2. Create a compute instance from the supplied base image for your cluster topology. See Creating the OpenShift Container Platform cluster on OCI (Oracle documentation).

   Important

   Before you create the compute instance, check that you have enough memory and disk resources for your cluster. Additionally, ensure that at least one compute instance has the same IP address as the address stated under rendezvousIP in the agent-config.yaml file.