Chapter 4. Installing a cluster with customizations

1. Log in to the OpenShift Container Platform web console using your login credentials.
2. Navigate to Datacenter.
3. Click Run Agent-based Installer locally.
4. Select the operating system and architecture for the OpenShift Installer and Command line interface.
5. Click Download Installer to download and extract the install program.
6. Download or copy the pull secret by clicking on Download pull secret or Copy pull secret.
7. Click Download command-line tools and place the openshift-install binary in a directory that is on your PATH.

Procedure

1. Install nmstate dependency by running the following command:

   $ sudo dnf install /usr/bin/nmstatectl -y

   Copy to Clipboard Toggle word wrap
2. Place the openshift-install binary in a directory that is on your PATH.

3. Create a directory to store the install configuration by running the following command:

   $ mkdir ~/<directory_name>

   Copy to Clipboard Toggle word wrap
   Note

   This is the preferred method for the Agent-based installation. Using GitOps ZTP manifests is optional.

4. Create the install-config.yaml file by running the following command:

   $ cat << EOF > ./<directory_name>/install-config.yaml
   apiVersion: v1
   baseDomain: test.example.com
   compute:
   - architecture: amd64 

   1

   
     hyperthreading: Enabled
     name: worker
     replicas: 0
   controlPlane:
     architecture: amd64
     hyperthreading: Enabled
     name: master
     replicas: 1
   metadata:
     name: sno-cluster 

   2

   
   networking:
     clusterNetwork:
     - cidr: 10.128.0.0/14
       hostPrefix: 23
     machineNetwork:
     - cidr: 192.168.0.0/16
     networkType: OVNKubernetes 

   3

   
     serviceNetwork:
     - 172.30.0.0/16
   platform:
     none: {}
   pullSecret: '<pull_secret>' 

   4

   
   sshKey: '<ssh_pub_key>' 

   5

   
   EOF

   Copy to Clipboard Toggle word wrap

   1

   Specify the system architecture, valid values are amd64 and arm64.

   2

   Required. Specify your cluster name.

   3

   Specify the cluster network plugin to install. The supported values are OVNKubernetes and OpenShiftSDN. The default value is OVNKubernetes.

   4

   Specify your pull secret.

   5

   Specify your SSH public key.

   Note

   If you set the platform to vSphere, baremetal, or none, you can configure IP address endpoints for cluster nodes in three ways:

   • IPv4
   • IPv6
   • IPv4 and IPv6 in parallel (dual-stack)

   Example of dual-stack networking

   networking:
     clusterNetwork:
       - cidr: 172.21.0.0/16
         hostPrefix: 23
       - cidr: fd02::/48
         hostPrefix: 64
     machineNetwork:
       - cidr: 192.168.11.0/16
       - cidr: 2001:DB8::/32
     serviceNetwork:
       - 172.22.0.0/16
       - fd03::/112
     networkType: OVNKubernetes
   platform:
     baremetal:
       apiVIPs:
       - 192.168.11.3
       - 2001:DB8::4
       ingressVIPs:
       - 192.168.11.4
       - 2001:DB8::5

   Copy to Clipboard Toggle word wrap

   Note

   When you use a disconnected mirror registry, you must add the certificate file that you created previously for your mirror registry to the additionalTrustBundle field of the install-config.yaml file.

5. Create the agent-config.yaml file by running the following command:

   $ cat > agent-config.yaml << EOF
   apiVersion: v1alpha1
   kind: AgentConfig
   metadata:
     name: sno-cluster
   rendezvousIP: 192.168.111.80 

   1

   
   hosts: 

   2

   
     - hostname: master-0 

   3

   
       interfaces:
         - name: eno1
           macAddress: 00:ef:44:21:e6:a5
       rootDeviceHints: 

   4

   
         deviceName: /dev/sdb
       networkConfig: 

   5

   
         interfaces:
           - name: eno1
             type: ethernet
             state: up
             mac-address: 00:ef:44:21:e6:a5
             ipv4:
               enabled: true
               address:
                 - ip: 192.168.111.80
                   prefix-length: 23
               dhcp: false
         dns-resolver:
           config:
             server:
               - 192.168.111.1
         routes:
           config:
             - destination: 0.0.0.0/0
               next-hop-address: 192.168.111.2
               next-hop-interface: eno1
               table-id: 254
   EOF

   Copy to Clipboard Toggle word wrap

   1

   This IP address is used to determine which node performs the bootstrapping process as well as running the assisted-service component. You must provide the rendezvous IP address when you do not specify at least one host’s IP address in the networkConfig parameter. If this address is not provided, one IP address is selected from the provided hosts' networkConfig.

   2

   Optional: Host configuration. The number of hosts defined must not exceed the total number of hosts defined in the install-config.yaml file, which is the sum of the values of the compute.replicas and controlPlane.replicas parameters.

   3

   Optional: Overrides the hostname obtained from either the Dynamic Host Configuration Protocol (DHCP) or a reverse DNS lookup. Each host must have a unique hostname supplied by one of these methods.

   4

   Enables provisioning of the Red Hat Enterprise Linux CoreOS (RHCOS) image to a particular device. It examines the devices in the order it discovers them, and compares the discovered values with the hint values. It uses the first discovered device that matches the hint value.

   5

   Optional: Configures the network interface of a host in NMState format.

Procedure

1. Generate ZTP cluster manifests by running the following command:

   $ openshift-install agent create cluster-manifests --dir <installation_directory>

   Copy to Clipboard Toggle word wrap
   Important

   If you have created the install-config.yaml and agent-config.yaml files, those files are deleted and replaced by the cluster manifests generated through this command.

   Any configurations made to the install-config.yaml and agent-config.yaml files are imported to the ZTP cluster manifests when you run the openshift-install agent create cluster-manifests command.

2. Navigate to the cluster-manifests directory by running the following command:

   $ cd <installation_directory>/cluster-manifests

   Copy to Clipboard Toggle word wrap
3. Configure the manifest files in the cluster-manifests directory. For sample files, see the "Sample GitOps ZTP custom resources" section.

4. Disconnected clusters: If you did not define mirror configuration in the install-config.yaml file before generating the ZTP manifests, perform the following steps:

   1. Navigate to the mirror directory by running the following command:

      $ cd ../mirror

      Copy to Clipboard Toggle word wrap
   2. Configure the manifest files in the mirror directory.

Procedure

1. Create the agent image by running the following command:

   $ openshift-install --dir <install_directory> agent create image

   Copy to Clipboard Toggle word wrap
   Note

   Red Hat Enterprise Linux CoreOS (RHCOS) supports multipathing on the primary disk, allowing stronger resilience to hardware failure to achieve higher host availability. Multipathing is enabled by default in the agent ISO image, with a default /etc/multipath.conf configuration.

2. Boot the agent.x86_64.iso or agent.aarch64.iso image on the bare metal machines.

Procedure

1. Wait for the agent console application to check whether or not the configured release image can be pulled from a registry.

2. If the agent console application states that the installer connectivity checks have passed, wait for the prompt to time out to continue with the installation.

   Note

   You can still choose to view or change network configuration settings even if the connectivity checks have passed.

   However, if you choose to interact with the agent console application rather than letting it time out, you must manually quit the TUI to proceed with the installation.

3. If the agent console application checks have failed, which is indicated by a red icon beside the Release image URL pull check, use the following steps to reconfigure the host’s network settings:

   1. Read the Check Errors section of the TUI. This section displays error messages specific to the failed checks.

      

      View larger image

      
   2. Select Configure network to launch the NetworkManager TUI.
   3. Select Edit a connection and select the connection you want to reconfigure.
   4. Edit the configuration and select OK to save your changes.
   5. Select Back to return to the main screen of the NetworkManager TUI.
   6. Select Activate a Connection.
   7. Select the reconfigured network to deactivate it.
   8. Select the reconfigured network again to reactivate it.
   9. Select Back and then select Quit to return to the agent console application.
   10. Wait at least five seconds for the continuous network checks to restart using the new network configuration.
   11. If the Release image URL pull check succeeds and displays a green icon beside the URL, select Quit to exit the agent console application and continue with the installation.

Procedure

1. Optional: To know when the bootstrap host (rendezvous host) reboots, run the following command:

   $ ./openshift-install --dir <install_directory> agent wait-for bootstrap-complete \ 

   1

   
       --log-level=info 

   2

   Copy to Clipboard Toggle word wrap

   1

   For <install_directory>, specify the path to the directory where the agent ISO was generated.

   2

   To view different installation details, specify warn, debug, or error instead of info.

   Example output

   ...................................................................
   ...................................................................
   INFO Bootstrap configMap status is complete
   INFO cluster bootstrap is complete

   Copy to Clipboard Toggle word wrap

   The command succeeds when the Kubernetes API server signals that it has been bootstrapped on the control plane machines.

2. To track the progress and verify successful installation, run the following command:

   $ openshift-install --dir <install_directory> agent wait-for install-complete 

   1

   Copy to Clipboard Toggle word wrap

   1

   For <install_directory> directory, specify the path to the directory where the agent ISO was generated.

   Example output

   ...................................................................
   ...................................................................
   INFO Cluster is installed
   INFO Install complete!
   INFO To access the cluster as the system:admin user when using 'oc', run
   INFO     export KUBECONFIG=/home/core/installer/auth/kubeconfig
   INFO Access the OpenShift web-console here: https://console-openshift-console.apps.sno-cluster.test.example.com

   Copy to Clipboard Toggle word wrap