Chapter 22. Installing on OCI

Procedure

1. Log in to your Oracle Cloud Infrastructure (OCI) account with administrator privileges.
2. Download an archive file from an Oracle resource. The archive file includes files for creating cluster resources and custom manifests. The archive file also includes a script, and when you run the script, the script creates OCI resources, such as DNS records, an instance, and so on. For more information, see Configuration Files (Oracle documentation).

Procedure

1. From the Install OpenShift with the Assisted Installer page on the Hybrid Cloud Console, generate the discovery ISO image by completing all the required Assisted Installer steps.

   1. In the Cluster Details step, complete the following fields:

      Field	Action required
      Cluster name	Specify the name of your cluster, such as ocidemo.
      Base domain	Specify the base domain of the cluster, such as splat-oci.devcluster.openshift.com. Provided you previously created a compartment on OCI, you can get this information by going to DNS management Zones List scope and then selecting the parent compartment. Your base domain should show under the Public zones tab.
      OpenShift version	Specify OpenShift 4.15 or a later version.
      CPU architecture	Specify x86_64 or Arm64.
      Integrate with external partner platforms	Specify Oracle Cloud Infrastructure. After you specify this value, the Include custom manifests checkbox is selected by default.

   2. On the Operators page, click Next.
   3. On the Host Discovery page, click Add hosts.

   4. For the SSH public key field, add your SSH key from your local system.

      Tip

      You can create an SSH authentication key pair by using the ssh-keygen tool.

   5. Click Generate Discovery ISO to generate the discovery ISO image file.
   6. Download the file to your local system.

2. Upload the discovery ISO image to the OCI bucket. See Uploading an Object Storage Object to a Bucket (Oracle documentation).

   1. You must create a pre-authenticated request for your uploaded discovery ISO image. Ensure that you make note of the URL from the pre-authenticated request, because you must specify the URL at a later stage when you create an OCI stack.

Procedure

1. Complete the steps for provisioning OCI infrastructure for your OpenShift Container Platform cluster. See Creating OpenShift Container Platform Infrastructure Using Resource Manager (Oracle documentation).
2. Create a stack, and then edit the custom manifest files according to the steps in the Editing the OpenShift Custom Manifests (Oracle documentation).

Procedure

1. From the Red Hat Hybrid Cloud Console web console, go to the Host discovery page.

2. Under the Role column, select either Control plane node or Worker for each targeted hostname.

   Important

   Before, you can continue to the next steps, wait for each node to reach the Ready status.

3. Accept the default settings for the Storage and Networking steps, and then click Next.

4. On the Custom manifests page, in the Folder field, select manifest. This is the Assisted Installer folder where you want to save the custom manifest file.

   1. In the File name field, enter a value such as oci-ccm.yml.
   2. From the Content section, click Browse, and select the CCM manifest from your drive located in custom_manifest/manifests/oci-ccm.yml.

5. Expand the next Custom manifest section and repeat the same steps for the following manifests:

   • CSI driver manifest: custom_manifest/manifests/oci-csi.yml
   • CCM machine configuration: custom_manifest/openshift/machineconfig-ccm.yml
   • CSI driver machine configuration: custom_manifest/openshift/machineconfig-csi.yml
6. From the Review and create page, click Install cluster to create your OpenShift Container Platform cluster on OCI.

Procedure

1. From the Hybrid Cloud Console, go to Clusters > Assisted Clusters and select your cluster’s name.
2. Check that the Installation progress bar is at 100% and a message displays indicating “Installation completed successfully”.
3. To access the OpenShift Container Platform web console, click the provided Web Console URL.
4. Go to the Nodes menu page.
5. Locate your node from the Nodes table.
6. From the Overview tab, check that your node has a Ready status.
7. Select the YAML tab.
8. Check the labels parameter, and verify that the listed labels apply to your configuration. For example, the topology.kubernetes.io/region=us-sanjose-1 label indicates in what OCI region the node was deployed.

Procedure

1. For a disconnected environment, mirror the Mirror registry for Red Hat OpenShift to your local container image registry.

   Important

   Check that your openshift-install binary version relates to your local image container registry and not a shared registry, such as Red Hat Quay.

   $ ./openshift-install version

   Example output for a shared registry binary

   ./openshift-install 4.15.0
   built from commit ae7977b7d1ca908674a0d45c5c243c766fa4b2ca
   release image registry.ci.openshift.org/origin/release:4.15ocp-release@sha256:0da6316466d60a3a4535d5fed3589feb0391989982fba59d47d4c729912d6363
   release architecture amd64

2. Configure the install-config.yaml configuration file to meet the needs of your organization.

   Example install-config.yaml configuration file that demonstrates setting an external platform

   # install-config.yaml
   apiVersion: v1
   baseDomain: <base_domain> 1
   networking:
     clusterNetwork:
     - cidr: 10.128.0.0/14
       hostPrefix: 23
     network type: OVNKubernetes
     machineNetwork:
     - cidr: <ip_address_from_cidr> 2
     serviceNetwork:
     - 172.30.0.0/16
   compute:
     - architecture: amd64 3
     hyperthreading: Enabled
     name: worker
     replicas: 0
   controlPlane:
     architecture: amd64 4
     hyperthreading: Enabled
     name: master
     replicas: 3
   platform:
      external:
       platformName: oci 5
       cloudControllerManager: External
   sshKey: <public_ssh_key> 6
   pullSecret: '<pull_secret>' 7
   # ...

   1

   The base domain of your cloud provider.

   2

   The IP address from the virtual cloud network (VCN) that the CIDR allocates to resources and components that operate on your network.

   3 4

   Depending on your infrastructure, you can select either x86_64, or amd64.

   5

   Set OCI as the external platform, so that OpenShift Container Platform can integrate with OCI.

   6

   Specify your SSH public key.

   7

   The pull secret that you need for authenticate purposes when downloading container images for OpenShift Container Platform components and services, such as Quay.io. See Install OpenShift Container Platform 4 from the Red Hat Hybrid Cloud Console.

3. Create a directory on your local system named openshift.

   Important

   Do not move the install-config.yaml and agent-config.yaml configuration files to the openshift directory.

4. Complete the steps in the "Configuration Files" section of the Oracle documentation to download Oracle Cloud Controller Manager (CCM) and Oracle Container Storage Interface (CSI) manifests as an archive file and save the archive file in your openshift directory. You need the Oracle CCM manifests for deploying the Oracle CCM during cluster installation so that OpenShift Container Platform can connect to the external OCI platform. You need the Oracle CSI custom manifests for deploying the Oracle CSI driver during cluster installation so that OpenShift Container Platform can claim required objects from OCI.

5. Access the custom manifest files that are provided in the "Configuration Files" section of the Oracle documentation.

   1. Change the oci-cloud-controller-manager secret that is defined in the oci-ccm.yml configuration file to match your organization’s region, compartment OCID, VCN OCID, and the subnet OCID from the load balancer.

6. Use the Agent-based Installer to generate a minimal ISO image, which excludes the rootfs image, by entering the following command in your OpenShift Container Platform CLI. You can use this image later in the process to boot all your cluster’s nodes.

   $ ./openshift-install agent create image --log-level debug

   The command also completes the following actions:

   • Creates a subdirectory, ./<installation_directory>/auth directory:, and places kubeadmin-password and kubeconfig files in the subdirectory.
   • Creates a rendezvousIP file based on the IP address that you specified in the agent-config.yaml configuration file.

   • Optional: Any modifications you made to agent-config.yaml and install-config.yaml configuration files get imported to the Zero Touch Provisioning (ZTP) custom resources.

     Important

     The Agent-based Installer uses Red Hat Enterprise Linux CoreOS (RHCOS). The rootfs image, which is mentioned in a later listed item, is required for booting, recovering, and repairing your operating system.

7. Configure the agent-config.yaml configuration file to meet your organization’s requirements.

   Example agent-config.yaml configuration file that sets values for an IPv4 formatted network.

   apiVersion: v1alpha1
   metadata:
     name: <cluster_name> 1
     namespace: <cluster_namespace> 2
   rendezvousIP: <ip_address_from_CIDR> 3
   bootArtifactsBaseURL: <server_URL> 4
   # ...

   1

   The cluster name that you specified in your DNS record.

   2

   The namespace of your cluster on OpenShift Container Platform.

   3

   If you use IPv4 as the network IP address format, ensure that you set the rendezvousIP parameter to an IPv4 address that the VCN’s Classless Inter-Domain Routing (CIDR) method allocates on your network. Also ensure that at least one instance from the pool of instances that you booted with the ISO matches the IP address value you set for rendezvousIP.

   4

   The URL of the server where you want to upload the rootfs image.

8. Apply one of the following two updates to your agent-config.yaml configuration file:

   • For a disconnected network: After you run the command to generate a minimal ISO Image, the Agent-based installer saves the rootfs image into the ./<installation_directory>/boot-artifacts directory on your local system. Use your preferred web server, such as any Hypertext Transfer Protocol daemon (httpd), to upload rootfs to the location stated in the bootArtifactsBaseURL parameter in the agent-config.yaml configuration file.

     For example, if the bootArtifactsBaseURL parameter states http://192.168.122.20, you would upload the generated rootfs image to this location, so that the Agent-based installer can access the image from http://192.168.122.20/agent.x86_64-rootfs.img. After the Agent-based installer boots the minimal ISO for the external platform, the Agent-based Installer downloads the rootfs image from the http://192.168.122.20/agent.x86_64-rootfs.img location into the system memory.

     Note

     The Agent-based Installer also adds the value of the bootArtifactsBaseURL to the minimal ISO Image’s configuration, so that when the Operator boots a cluster’s node, the Agent-based Installer downloads the rootfs image into system memory.

   • For a connected network: You do not need to specify the bootArtifactsBaseURL parameter in the agent-config.yaml configuration file. The default behavior of the Agent-based Installer reads the rootfs URL location from https://rhcos.mirror.openshift.com. After the Agent-based Installer boots the minimal ISO for the external platform, the Agent-based Installer then downloads the rootfs file into your system’s memory from the default RHCOS URL.

     Important

     Consider that the full ISO image, which is in excess of 1 GB, includes the rootfs image. The image is larger than the minimal ISO Image, which is typically less than 150 MB.

Procedure

1. Set the following registry URLs for your firewall’s allowlist:

   URL	Port	Function
   registry.redhat.io	443	Provides core container images
   access.redhat.com [1]	443	Hosts all the container images that are stored on the Red Hat Ecosytem Catalog, including core container images.
   quay.io	443	Provides core container images
   cdn.quay.io	443	Provides core container images
   cdn01.quay.io	443	Provides core container images
   cdn02.quay.io	443	Provides core container images
   cdn03.quay.io	443	Provides core container images
   sso.redhat.com	443	The https://console.redhat.com site uses authentication from sso.redhat.com

   1. In a firewall environment, ensure that the access.redhat.com resource is on the allowlist. This resource hosts a signature store that a container client requires for verifying images when pulling them from registry.access.redhat.com.

   You can use the wildcards *.quay.io and *.openshiftapps.com instead of cdn.quay.io and cdn0[1-3].quay.io in your allowlist. When you add a site, such as quay.io, to your allowlist, do not add a wildcard entry, such as *.quay.io, to your denylist. In most cases, image registries use a content delivery network (CDN) to serve images. If a firewall blocks access, image downloads are denied when the initial download request redirects to a hostname such as cdn01.quay.io.

2. Set your firewall’s allowlist to include any site that provides resources for a language or framework that your builds require.

3. If you do not disable Telemetry, you must grant access to the following URLs to access Red Hat Insights:

   URL	Port	Function
   cert-api.access.redhat.com	443	Required for Telemetry
   api.access.redhat.com	443	Required for Telemetry
   infogw.api.openshift.com	443	Required for Telemetry
   console.redhat.com	443	Required for Telemetry and for insights-operator

4. Set your firewall’s allowlist to include the following registry URLs:

   URL	Port	Function
   api.openshift.com	443	Required both for your cluster token and to check if updates are available for the cluster.
   rhcos.mirror.openshift.com	443	Required to download Red Hat Enterprise Linux CoreOS (RHCOS) images.

5. Set your firewall’s allowlist to include the following external URLs. Each repository URL hosts OCI containers. Consider mirroring images to as few repositories as possible to reduce any performance issues.

   URL	Port	Function
   k8s.gcr.io	port	A Kubernetes registry that hosts container images for a community-based image registry. This image registry is hosted on a custom Google Container Registry (GCR) domain.
   ghcr.io	port	A GitHub image registry where you can store and manage Open Container Initiative images. Requires an access token to publish, install, and delete private, internal, and public packages.
   storage.googleapis.com	443	A source of release image signatures, although the Cluster Version Operator needs only a single functioning source.
   registry.k8s.io	port	Replaces the k8s.gcr.io image registry because the k8s.gcr.io image registry does not support other platforms and vendors.

Procedure

1. Upload the agent ISO image to Oracle’s default Object Storage bucket and import the agent ISO image as a custom image to this bucket. Ensure you that you configure the custom image to boot in Unified Extensible Firmware Interface (UEFI) mode. For more information, see Creating the OpenShift Container Platform ISO Image (Oracle documentation).

2. Create a compute instance from the supplied base image for your cluster topology. See Creating the OpenShift Container Platform cluster on OCI (Oracle documentation).

   Important

   Before you create the compute instance, check that you have enough memory and disk resources for your cluster. Additionally, ensure that at least one compute instance has the same IP address as the address stated under rendezvousIP in the agent-config.yaml file.