Installing on OCI

Procedure

1. Log in to your Oracle Cloud Infrastructure (OCI) account with administrator privileges.

2. Configure the account by defining the Cloud Accounts and Resources (Oracle documentation). Ensure that you create the following resources:

   1. Create a child compartment for organizing, restricting access, and setting usage limits to OCI resources. For the full procedure, see Creating a Compartment (Oracle documentation).
   2. Create a new object storage bucket into which you will upload the discovery ISO image. For the full procedure, see Creating an Object Storage Bucket (Oracle documentation).

3. Download the latest version of the create-cluster-vX.X.X.zip configuration file from the oracle-quickstart/oci-openshift repository. This file provides the infrastructure for the cluster and contains configurations for the following:

   • Terraform Stacks: The Terraform stack code for provisioning OCI resources to create and manage OpenShift Container Platform clusters on OCI.
   • Custom Manifests: The manifest files needed for the installation of OpenShift Container Platform clusters on OCI.
   Note

   To make any changes to the manifests, you can clone the entire Oracle GitHub repository and access the custom_manifests and terraform-stacks directories directly.

   For details, see Configuration Files (Oracle documentation).

Procedure

1. Log in to your Oracle Cloud Infrastructure (OCI) account.
2. Upload the discovery ISO image from your local drive to the new object storage bucket you created. For the full procedure, see Uploading an Object Storage Object to a Bucket (Oracle documentation).

3. Locate the uploaded discovery ISO, and complete the following steps:

   1. Create a Pre-Authenticated Request (PAR) for the ISO from the adjacent options menu.
   2. Copy the generated URL to use as the OpenShift Image Source URI in the next step.

   For the full procedure, see Creating a Pre-Authenticated Requests in Object Storage (Oracle documentation).

4. Create and apply the Terraform stack:

   Important

   The Terraform stack includes files for creating cluster resources and custom manifests. The stack also includes a script, and when you apply the stack, the script creates OCI resources, such as DNS records, an instance, and so on. For a list of the resources, see Terraform Defined Resources for OpenShift on OCI README file.

   1. Upload the Terraform stack template create-cluster-vX.X.X.zip to the new object storage bucket.

   2. Complete the stack information and click Next.

      Important
      • Make sure that Cluster Name matches Cluster Name in Assisted Installer, and Zone DNS matches Base Domain in Assisted Installer.
      • In the OpenShift Image Source URI field, paste the Pre-Authenticated Request URL link that you generated in the previous step.
      • Ensure that the correct Compute Shape field value is defined, depending on whether you are installing on bare metal or a virtual machine. If not, select a different shape from the list. For details, see Compute Shapes (Oracle documentation).
   3. Click Apply to apply the stack.

   For the full procedure, see Creating OpenShift Container Platform Infrastructure Using Resource Manager (Oracle documentation).

5. Copy the dynamic_custom_manifest.yml file from the Outputs page of the Terraform stack.

   Note

   The YAML file contains all the required manifests, concatenated and preformatted with the configuration values. For details, see the Custom Manifests README file.

   For the full procedure, see Getting the OpenShift Container Platform Custom Manifests for Installation (Oracle documentation).

Procedure

1. From the Red Hat Hybrid Cloud Console, go to Clusters > Assisted Clusters and select your cluster’s name.
2. On the Installation Progress page, check that the Installation progress bar is at 100% and a message displays indicating Installation completed successfully.

3. Under Host inventory, confirm that the status of all control plane and compute nodes is Installed.

   Note

   OpenShift Container Platform designates one of the control plane nodes as the bootstrap virtual machine, eliminating the need for a separate bootstrap machine.

4. Click the Web Console URL, to access the OpenShift Container Platform web console.
5. From the menu, select Compute > Nodes.
6. Locate your node from the Nodes table.
7. From the Terminal tab, verify that iSCSI appears next to the serial number.
8. From the Overview tab, check that your node has a Ready status.
9. Select the YAML tab.
10. Check the labels parameter, and verify that the listed labels apply to your configuration. For example, the topology.kubernetes.io/region=us-sanjose-1 label indicates in what OCI region the node was deployed.

Procedure

1. For a disconnected environment, mirror the Mirror registry for Red Hat OpenShift to your local container image registry.

   Important

   Check that your openshift-install binary version relates to your local image container registry and not a shared registry, such as Red Hat Quay.

   $ ./openshift-install version

   Example output for a shared registry binary

   ./openshift-install 4.16.0
   built from commit ae7977b7d1ca908674a0d45c5c243c766fa4b2ca
   release image registry.ci.openshift.org/origin/release:4.16ocp-release@sha256:0da6316466d60a3a4535d5fed3589feb0391989982fba59d47d4c729912d6363
   release architecture amd64

2. Configure the install-config.yaml configuration file to meet the needs of your organization.

   Example install-config.yaml configuration file that demonstrates setting an external platform

   # install-config.yaml
   apiVersion: v1
   baseDomain: <base_domain> 1
   networking:
     clusterNetwork:
     - cidr: 10.128.0.0/14
       hostPrefix: 23
     network type: OVNKubernetes
     machineNetwork:
     - cidr: <ip_address_from_cidr> 2
     serviceNetwork:
     - 172.30.0.0/16
   compute:
     - architecture: amd64 3
     hyperthreading: Enabled
     name: worker
     replicas: 0
   controlPlane:
     architecture: amd64 4
     hyperthreading: Enabled
     name: master
     replicas: 3
   platform:
      external:
       platformName: oci 5
       cloudControllerManager: External
   sshKey: <public_ssh_key> 6
   pullSecret: '<pull_secret>' 7
   # ...

   1

   The base domain of your cloud provider.

   2

   The IP address from the virtual cloud network (VCN) that the CIDR allocates to resources and components that operate on your network.

   3 4

   Depending on your infrastructure, you can select either x86_64, or amd64.

   5

   Set OCI as the external platform, so that OpenShift Container Platform can integrate with OCI.

   6

   Specify your SSH public key.

   7

   The pull secret that you need for authenticate purposes when downloading container images for OpenShift Container Platform components and services, such as Quay.io. See Install OpenShift Container Platform 4 from the Red Hat Hybrid Cloud Console.

3. Create a directory on your local system named openshift.

   Important

   Do not move the install-config.yaml and agent-config.yaml configuration files to the openshift directory.

4. Complete the steps in the "Configuration Files" section of the Oracle documentation to download Oracle Cloud Controller Manager (CCM) and Oracle Container Storage Interface (CSI) manifests as an archive file and save the archive file in your openshift directory. You need the Oracle CCM manifests for deploying the Oracle CCM during cluster installation so that OpenShift Container Platform can connect to the external OCI platform. You need the Oracle CSI custom manifests for deploying the Oracle CSI driver during cluster installation so that OpenShift Container Platform can claim required objects from OCI.

5. Access the custom manifest files that are provided in the "Configuration Files" section of the Oracle documentation.

   1. Change the oci-cloud-controller-manager secret that is defined in the oci-ccm.yml configuration file to match your organization’s region, compartment OCID, VCN OCID, and the subnet OCID from the load balancer.

6. Use the Agent-based Installer to generate a minimal ISO image, which excludes the rootfs image, by entering the following command in your OpenShift Container Platform CLI. You can use this image later in the process to boot all your cluster’s nodes.

   $ ./openshift-install agent create image --log-level debug

   The command also completes the following actions:

   • Creates a subdirectory, ./<installation_directory>/auth directory:, and places kubeadmin-password and kubeconfig files in the subdirectory.
   • Creates a rendezvousIP file based on the IP address that you specified in the agent-config.yaml configuration file.

   • Optional: Any modifications you made to agent-config.yaml and install-config.yaml configuration files get imported to the Zero Touch Provisioning (ZTP) custom resources.

     Important

     The Agent-based Installer uses Red Hat Enterprise Linux CoreOS (RHCOS). The rootfs image, which is mentioned in a later listed item, is required for booting, recovering, and repairing your operating system.

7. Configure the agent-config.yaml configuration file to meet your organization’s requirements.

   Example agent-config.yaml configuration file that sets values for an IPv4 formatted network.

   apiVersion: v1alpha1
   metadata:
     name: <cluster_name> 1
     namespace: <cluster_namespace> 2
   rendezvousIP: <ip_address_from_CIDR> 3
   bootArtifactsBaseURL: <server_URL> 4
   # ...

   1

   The cluster name that you specified in your DNS record.

   2

   The namespace of your cluster on OpenShift Container Platform.

   3

   If you use IPv4 as the network IP address format, ensure that you set the rendezvousIP parameter to an IPv4 address that the VCN’s Classless Inter-Domain Routing (CIDR) method allocates on your network. Also ensure that at least one instance from the pool of instances that you booted with the ISO matches the IP address value you set for rendezvousIP.

   4

   The URL of the server where you want to upload the rootfs image.

8. Apply one of the following two updates to your agent-config.yaml configuration file:

   • For a disconnected network: After you run the command to generate a minimal ISO Image, the Agent-based installer saves the rootfs image into the ./<installation_directory>/boot-artifacts directory on your local system. Use your preferred web server, such as any Hypertext Transfer Protocol daemon (httpd), to upload rootfs to the location stated in the bootArtifactsBaseURL parameter in the agent-config.yaml configuration file.

     For example, if the bootArtifactsBaseURL parameter states http://192.168.122.20, you would upload the generated rootfs image to this location, so that the Agent-based installer can access the image from http://192.168.122.20/agent.x86_64-rootfs.img. After the Agent-based installer boots the minimal ISO for the external platform, the Agent-based Installer downloads the rootfs image from the http://192.168.122.20/agent.x86_64-rootfs.img location into the system memory.

     Note

     The Agent-based Installer also adds the value of the bootArtifactsBaseURL to the minimal ISO Image’s configuration, so that when the Operator boots a cluster’s node, the Agent-based Installer downloads the rootfs image into system memory.

   • For a connected network: You do not need to specify the bootArtifactsBaseURL parameter in the agent-config.yaml configuration file. The default behavior of the Agent-based Installer reads the rootfs URL location from https://rhcos.mirror.openshift.com. After the Agent-based Installer boots the minimal ISO for the external platform, the Agent-based Installer then downloads the rootfs file into your system’s memory from the default RHCOS URL.

     Important

     Consider that the full ISO image, which is in excess of 1 GB, includes the rootfs image. The image is larger than the minimal ISO Image, which is typically less than 150 MB.

Procedure

1. Set the following registry URLs for your firewall’s allowlist:

   URL	Port	Function
   registry.redhat.io	443	Provides core container images
   access.redhat.com	443	Hosts a signature store that a container client requires for verifying images pulled from registry.access.redhat.com. In a firewall environment, ensure that this resource is on the allowlist.
   registry.access.redhat.com	443	Hosts all the container images that are stored on the Red Hat Ecosystem Catalog, including core container images.
   quay.io	443	Provides core container images
   cdn.quay.io	443	Provides core container images
   cdn01.quay.io	443	Provides core container images
   cdn02.quay.io	443	Provides core container images
   cdn03.quay.io	443	Provides core container images
   cdn04.quay.io	443	Provides core container images
   cdn05.quay.io	443	Provides core container images
   cdn06.quay.io	443	Provides core container images
   sso.redhat.com	443	The https://console.redhat.com site uses authentication from sso.redhat.com

   • You can use the wildcards *.quay.io and *.openshiftapps.com instead of cdn.quay.io and cdn0[1-6].quay.io in your allowlist.
   • You can use the wildcard *.access.redhat.com to simplify the configuration and ensure that all subdomains, including registry.access.redhat.com, are allowed.
   • When you add a site, such as quay.io, to your allowlist, do not add a wildcard entry, such as *.quay.io, to your denylist. In most cases, image registries use a content delivery network (CDN) to serve images. If a firewall blocks access, image downloads are denied when the initial download request redirects to a hostname such as cdn01.quay.io.
2. Set your firewall’s allowlist to include any site that provides resources for a language or framework that your builds require.

3. If you do not disable Telemetry, you must grant access to the following URLs to access Red Hat Insights:

   URL	Port	Function
   cert-api.access.redhat.com	443	Required for Telemetry
   api.access.redhat.com	443	Required for Telemetry
   infogw.api.openshift.com	443	Required for Telemetry
   console.redhat.com	443	Required for Telemetry and for insights-operator

4. Set your firewall’s allowlist to include the following registry URLs:

   URL	Port	Function
   api.openshift.com	443	Required both for your cluster token and to check if updates are available for the cluster.
   rhcos.mirror.openshift.com	443	Required to download Red Hat Enterprise Linux CoreOS (RHCOS) images.

5. Set your firewall’s allowlist to include the following external URLs. Each repository URL hosts OCI containers. Consider mirroring images to as few repositories as possible to reduce any performance issues.

   URL	Port	Function
   k8s.gcr.io	port	A Kubernetes registry that hosts container images for a community-based image registry. This image registry is hosted on a custom Google Container Registry (GCR) domain.
   ghcr.io	port	A GitHub image registry where you can store and manage Open Container Initiative images. Requires an access token to publish, install, and delete private, internal, and public packages.
   storage.googleapis.com	443	A source of release image signatures, although the Cluster Version Operator needs only a single functioning source.
   registry.k8s.io	port	Replaces the k8s.gcr.io image registry because the k8s.gcr.io image registry does not support other platforms and vendors.

Procedure

1. Upload the agent ISO image to Oracle’s default Object Storage bucket and import the agent ISO image as a custom image to this bucket. Ensure you that you configure the custom image to boot in Unified Extensible Firmware Interface (UEFI) mode. For more information, see Creating the OpenShift Container Platform ISO Image (Oracle documentation).

2. Create a compute instance from the supplied base image for your cluster topology. See Creating the OpenShift Container Platform cluster on OCI (Oracle documentation).

   Important

   Before you create the compute instance, check that you have enough memory and disk resources for your cluster. Additionally, ensure that at least one compute instance has the same IP address as the address stated under rendezvousIP in the agent-config.yaml file.