Chapter 16. Image-based installation for single-node OpenShift

1. Generate an image from a single-node OpenShift cluster.
2. Use the openshift-install program to embed the seed image URL, and other installation artifacts, in a live installation ISO.

3. Start the host using the live installation ISO to preinstall the host.

   During this process, the openshift-install program installs Red Hat Enterprise Linux CoreOS (RHCOS) to the disk, pulls the image you generated, and precaches release container images to the disk.

4. When the installation completes, the host is ready to ship to the remote site for rapid reconfiguration and deployment.

Procedure

1. Create a Namespace object YAML file for the Lifecycle Agent:

   apiVersion: v1
   kind: Namespace
   metadata:
     name: openshift-lifecycle-agent
     annotations:
       workload.openshift.io/allowed: management

   Copy to Clipboard Toggle word wrap

   1. Create the Namespace CR by running the following command:

      $ oc create -f <namespace_filename>.yaml

      Copy to Clipboard Toggle word wrap

2. Create an OperatorGroup object YAML file for the Lifecycle Agent:

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: openshift-lifecycle-agent
     namespace: openshift-lifecycle-agent
   spec:
     targetNamespaces:
     - openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   1. Create the OperatorGroup CR by running the following command:

      $ oc create -f <operatorgroup_filename>.yaml

      Copy to Clipboard Toggle word wrap

3. Create a Subscription CR for the Lifecycle Agent:

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: openshift-lifecycle-agent-subscription
     namespace: openshift-lifecycle-agent
   spec:
     channel: "stable"
     name: lifecycle-agent
     source: redhat-operators
     sourceNamespace: openshift-marketplace

   Copy to Clipboard Toggle word wrap

   1. Create the Subscription CR by running the following command:

      $ oc create -f <subscription_filename>.yaml

      Copy to Clipboard Toggle word wrap

Verification

1. To verify that the installation succeeded, inspect the CSV resource by running the following command:

   $ oc get csv -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                              DISPLAY                     VERSION               REPLACES                           PHASE
   lifecycle-agent.v4.17.0           Openshift Lifecycle Agent   4.17.0                Succeeded

   Copy to Clipboard Toggle word wrap

2. Verify that the Lifecycle Agent is up and running by running the following command:

   $ oc get deploy -n openshift-lifecycle-agent

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
   lifecycle-agent-controller-manager   1/1     1            1           14s

   Copy to Clipboard Toggle word wrap

Procedure

1. In the OpenShift Container Platform web console, navigate to Operators OperatorHub.
2. Search for the Lifecycle Agent from the list of available Operators, and then click Install.
3. On the Install Operator page, under A specific namespace on the cluster select openshift-lifecycle-agent.
4. Click Install.

Verification

1. To confirm that the installation is successful:

   1. Click Operators Installed Operators.

   2. Ensure that the Lifecycle Agent is listed in the openshift-lifecycle-agent project with a Status of InstallSucceeded.

      Note

      During installation an Operator might display a Failed status. If the installation later succeeds with an InstallSucceeded message, you can ignore the Failed message.

1. Click Operators Installed Operators, and inspect the Operator Subscriptions and Install Plans tabs for any failure or errors under Status.
2. Click Workloads Pods, and check the logs for pods in the openshift-lifecycle-agent project.

Procedure

1. Edit the image-based-installation-config.yaml file to configure additional partitions:

   Example image-based-installation-config.yaml file

   apiVersion: v1beta1
   kind: ImageBasedInstallationConfig
   metadata:
     name: example-extra-partition
   seedImage: quay.io/repo-id/seed:latest
   seedVersion: "4.17.0"
   installationDisk: /dev/sda
   pullSecret: '{"auths": ...}'
   # ...
   skipDiskCleanup: true 

   1

   
   coreosInstallerArgs:
      - "--save-partindex" 

   2

   
      - "6" 

   3

   
   ignitionConfigOverride: |
     {
       "ignition": {
         "version": "3.2.0"
       },
       "storage": {
         "disks": [
           {
             "device": "/dev/sda", 

   4

   
             "partitions": [
               {
                 "label": "storage", 

   5

   
                 "number": 6, 

   6

   
                 "sizeMiB": 380000, 

   7

   
                 "startMiB": 500000 

   8

   
               }
             ]
           }
         ]
       }
     }

   Copy to Clipboard Toggle word wrap

   1

   Specify true to skip disk formatting during the installation process.

   2

   Specify this argument to preserve a partition.

   3

   The live installation ISO requires five partitions. Specify a number greater than five to identify the additional partition to preserve.

   4

   Specify the installation disk on the target host.

   5

   Specify the label for the partition.

   6

   Specify the number for the partition.

   7

   Specify the size of parition in MiB.

   8

   Specify the starting position on the disk in MiB for the additional partition. You must specify a starting point larger that the partition for var/lib/containers.

Procedure

1. Create the ibi-ns namespace by running the following command:

   $ oc create namespace ibi-ns

   Copy to Clipboard Toggle word wrap

2. Create the Secret resource for your image registry:

   1. Create a YAML file that defines the Secret resource for your image registry:

      Example secret-image-registry.yaml file

      apiVersion: v1
      kind: Secret
      metadata:
        name: ibi-image-pull-secret
        namespace: ibi-ns
      stringData:
        .dockerconfigjson: <base64-docker-auth-code> 

      1

      
      type: kubernetes.io/dockerconfigjson

      Copy to Clipboard Toggle word wrap

      1

      You must provide base64-encoded credential details. See the "Additional resources" section for more information about using image pull secrets.

   2. Create the Secret resource for your image registry by running the following command:

      $ oc create -f secret-image-registry.yaml

      Copy to Clipboard Toggle word wrap

3. Optional: Configure static networking for the host:

   1. Create a Secret resource containing the static network configuration in nmstate format:

      Example host-network-config-secret.yaml file

      apiVersion: v1
      kind: Secret
      metadata:
       name: host-network-config-secret 

      1

      
       namespace: ibi-ns
      type: Opaque
      stringData:
       nmstate: | 

      2

      
        interfaces:
          - name: ens1f0 

      3

      
            type: ethernet
            state: up
            ipv4:
              enabled: true
              address:
                - ip: 192.168.200.25
                  prefix-length: 24
              dhcp: false 

      4

      
            ipv6:
              enabled: false
        dns-resolver:
          config:
            server:
              - 192.168.15.47 

      5

      
              - 192.168.15.48
        routes:
          config: 

      6

      
            - destination: 0.0.0.0/0
              metric: 150
              next-hop-address: 192.168.200.254
              next-hop-interface: ens1f0
              table-id: 254

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the Secret resource.

      2

      Define the static network configuration in nmstate format.

      3

      Specify the name of the interface on the host. The name of the interface must match the actual NIC name as shown in the operating system. To use your MAC address for NIC matching, set the identifier field to mac-address.

      4

      You must specify dhcp: false to ensure nmstate assigns the static IP address to the interface.

      5

      Specify one or more DNS servers that the system will use to resolve domain names.

      6

      In this example, the default route is configured through the ens1f0 interface to the next hop IP address 192.168.200.254.

4. Create the BareMetalHost and Secret resources:

   1. Create a YAML file that defines the BareMetalHost and Secret resources:

      Example ibi-bmh.yaml file

      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        name: ibi-bmh 

      1

      
        namespace: ibi-ns
      spec:
        online: false 

      2

      
        bootMACAddress: 00:a5:12:55:62:64 

      3

      
        bmc:
          address: redfish-virtualmedia+http://192.168.111.1:8000/redfish/v1/Systems/8a5babac-94d0-4c20-b282-50dc3a0a32b5 

      4

      
          credentialsName: ibi-bmh-bmc-secret 

      5

      
        preprovisioningNetworkDataName: host-network-config-secret 

      6

      
        automatedCleaningMode: disabled 

      7

      
        externallyProvisioned: true 

      8

      
      ---
      apiVersion: v1
      kind: Secret
      metadata:
        name: ibi-bmh-secret 

      9

      
        namespace: ibi-ns
      type: Opaque
      data:
        username: <user_name> 

      10

      
        password: <password> 

      11

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the BareMetalHost resource.

      2

      Specify if the host should be online.

      3

      Specify the host boot MAC address.

      4

      Specify the BMC address. You can only use bare-metal host drivers that support virtual media networking booting, for example redfish-virtualmedia and idrac-virtualmedia.

      5

      Specify the name of the bare-metal host Secret resource.

      6

      Optional: If you require static network configuration for the host, specify the name of the Secret resource containing the configuration.

      7

      You must specify automatedCleaningMode:disabled to prevent the provisioning service from deleting all preinstallation artifacts, such as the seed image, during disk inspection.

      8

      You must specify externallyProvisioned: true to enable the host to boot from the preinstalled disk, instead of the configuration ISO.

      9

      Specify the name for the Secret resource.

      10

      Specify the username.

      11

      Specify the password.

   2. Create the BareMetalHost and Secret resources by running the following command:

      $ oc create -f ibi-bmh.yaml

      Copy to Clipboard Toggle word wrap

5. Create the ClusterImageSet resource:

   1. Create a YAML file that defines the ClusterImageSet resource:

      Example ibi-cluster-image-set.yaml file

      apiVersion: hive.openshift.io/v1
      kind: ClusterImageSet
      metadata:
        name: ibi-img-version-arch 

      1

      
      spec:
        releaseImage: ibi.example.com:path/to/release/images:version-arch 

      2

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the ClusterImageSet resource.

      2

      Specify the address for the release image to use for the deployment. If you use a different image registry compared to the image registry used during seed image generation, ensure that the OpenShift Container Platform version for the release image remains the same.

   2. Create the ClusterImageSet resource by running the following command:

      $ oc apply -f ibi-cluster-image-set.yaml

      Copy to Clipboard Toggle word wrap

6. Create the ImageClusterInstall resource:

   1. Create a YAML file that defines the ImageClusterInstall resource:

      Example ibi-image-cluster-install.yaml file

      apiVersion: extensions.hive.openshift.io/v1alpha1
      kind: ImageClusterInstall
      metadata:
        name: ibi-image-install 

      1

      
        namespace: ibi-ns
      spec:
        bareMetalHostRef:
          name: ibi-bmh 

      2

      
          namespace: ibi-ns
        clusterDeploymentRef:
          name: ibi-cluster-deployment 

      3

      
        hostname: ibi-host 

      4

      
        imageSetRef:
          name: ibi-img-version-arch 

      5

      
        machineNetwork: 10.0.0.0/24 

      6

      
        proxy: 

      7

      
          httpProxy: "http://proxy.example.com:8080"
          #httpsProxy: "http://proxy.example.com:8080"
          #noProxy: "no_proxy.example.com"

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the ImageClusterInstall resource.

      2

      Specify the BareMetalHost resource that you want to target for the image-based installation.

      3

      Specify the name of the ClusterDeployment resource that you want to use for the image-based installation of the target host.

      4

      Specify the hostname for the cluster.

      5

      Specify the name of the ClusterImageSet resource you used to define the container release images to use for deployment.

      6

      Specify the public CIDR (Classless Inter-Domain Routing) of the external network.

      7

      Optional: Specify a proxy to use for the cluster deployment.

      Important

      If your cluster deployment requires a proxy configuration, you must do the following:

      • Create a seed image from a seed cluster featuring a proxy configuration. The proxy configurations do not have to match.
      • Configure the machineNetwork field in your installation manifest.

   2. Create the ImageClusterInstall resource by running the following command:

      $ oc create -f ibi-image-cluster-install.yaml

      Copy to Clipboard Toggle word wrap

7. Create the ClusterDeployment resource:

   1. Create a YAML file that defines the ClusterDeployment resource:

      Example ibi-cluster-deployment.yaml file

      apiVersion: hive.openshift.io/v1
      kind: ClusterDeployment
      metadata:
        name: ibi-cluster-deployment 

      1

      
        namespace: ibi-ns 

      2

      
      spec:
        baseDomain: example.com 

      3

      
        clusterInstallRef:
          group: extensions.hive.openshift.io
          kind: ImageClusterInstall
          name: ibi-image-install 

      4

      
          version: v1alpha1
        clusterName: ibi-cluster 

      5

      
        platform:
          none: {}
        pullSecretRef:
          name: ibi-image-pull-secret 

      6

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the ClusterDeployment resource.

      2

      Specify the namespace for the ClusterDeployment resource.

      3

      Specify the base domain that the cluster should belong to.

      4

      Specify the name of the ImageClusterInstall in which you defined the container images to use for the image-based installation of the target host.

      5

      Specify a name for the cluster.

      6

      Specify the secret to use for pulling images from your image registry.

   2. Create the ClusterDeployment resource by running the following command:

      $ oc apply -f ibi-cluster-deployment.yaml

      Copy to Clipboard Toggle word wrap

8. Create the ManagedCluster resource:

   1. Create a YAML file that defines the ManagedCluster resource:

      Example ibi-managed.yaml file

      apiVersion: cluster.open-cluster-management.io/v1
      kind: ManagedCluster
      metadata:
        name: sno-ibi 

      1

      
      spec:
        hubAcceptsClient: true 

      2

      Copy to Clipboard Toggle word wrap

      1

      Specify the name for the ManagedCluster resource.

      2

      Specify true to enable RHACM to manage the cluster.

   2. Create the ManagedCluster resource by running the following command:

      $ oc apply -f ibi-managed.yaml

      Copy to Clipboard Toggle word wrap

Verification

1. Check the status of the ImageClusterInstall in the hub cluster to monitor the progress of the target host installation by running the following command:

   $ oc get imageclusterinstall

   Copy to Clipboard Toggle word wrap

   Example output

   NAME       REQUIREMENTSMET           COMPLETED                     BAREMETALHOSTREF
   target-0   HostValidationSucceeded   ClusterInstallationSucceeded  ibi-bmh

   Copy to Clipboard Toggle word wrap

   Warning

   If the ImageClusterInstall resource is deleted, the IBI Operator reattaches the BareMetalHost resource and reboots the machine.

2. When the installation completes, you can retrieve the kubeconfig secret to log in to the managed cluster by running the following command:

   $ oc extract secret/<cluster_name>-admin-kubeconfig -n <cluster_namespace>  --to - > <directory>/<cluster_name>-kubeconfig

   Copy to Clipboard Toggle word wrap
   • <cluster_name> is the name of the cluster.
   • <cluster_namespace> is the namespace of the cluster.
   • <directory> is the directory in which to create the file.

Procedure

1. Create a working directory by running the following:

   $ mkdir ibi-config-iso-workdir 

   1

   Copy to Clipboard Toggle word wrap

   1

   Replace ibi-config-iso-workdir with the name of your working directory.

2. Create the installation manifest:

   1. Create a YAML file that defines the install-config manifest:

      Example install-config.yaml file

      apiVersion: v1
      metadata:
        name: sno-cluster-name
      baseDomain: host.example.com
      compute:
        - architecture: amd64
          hyperthreading: Enabled
          name: worker
          replicas: 0
      controlPlane:
        architecture: amd64
        hyperthreading: Enabled
        name: master
        replicas: 1
      networking:
        machineNetwork:
        - cidr: 192.168.200.0/24
      platform:
        none: {}
      fips: false
      cpuPartitioningMode: "AllNodes"
      pullSecret: '{"auths":{"<your_pull_secret>"}}}'
      sshKey: 'ssh-rsa <your_ssh_pub_key>'

      Copy to Clipboard Toggle word wrap

      Important

      If your cluster deployment requires a proxy configuration, you must do the following:

      • Create a seed image from a seed cluster featuring a proxy configuration. The proxy configurations do not have to match.
      • Configure the machineNetwork field in your installation manifest.
   2. Save the file in your working directory.

3. Optional. Create a configuration template in your working directory by running the following command:

   $ openshift-install image-based create config-template --dir ibi-config-iso-workdir/

   Copy to Clipboard Toggle word wrap

   Example output

   INFO Config-Template created in: ibi-config-iso-workdir

   Copy to Clipboard Toggle word wrap

   The command creates the image-based-config.yaml configuration template in your working directory:

   #
   # Note: This is a sample ImageBasedConfig file showing
   # which fields are available to aid you in creating your
   # own image-based-config.yaml file.
   #
   apiVersion: v1beta1
   kind: ImageBasedConfig
   metadata:
     name: example-image-based-config
   additionalNTPSources:
     - 0.rhel.pool.ntp.org
     - 1.rhel.pool.ntp.org
   hostname: change-to-hostname
   releaseRegistry: quay.io
   # networkConfig contains the network configuration for the host in NMState format.
   # See https://nmstate.io/examples.html for examples.
   networkConfig:
     interfaces:
       - name: eth0
         type: ethernet
         state: up
         mac-address: 00:00:00:00:00:00
         ipv4:
           enabled: true
           address:
             - ip: 192.168.122.2
               prefix-length: 23
           dhcp: false

   Copy to Clipboard Toggle word wrap

4. Edit your configuration file:

   Example image-based-config.yaml file

   #
   # Note: This is a sample ImageBasedConfig file showing
   # which fields are available to aid you in creating your
   # own image-based-config.yaml file.
   #
   apiVersion: v1beta1
   kind: ImageBasedConfig
   metadata:
     name: sno-cluster-name
   additionalNTPSources:
     - 0.rhel.pool.ntp.org
     - 1.rhel.pool.ntp.org
   hostname: host.example.com
   releaseRegistry: quay.io
   # networkConfig contains the network configuration for the host in NMState format.
   # See https://nmstate.io/examples.html for examples.
   networkConfig:
       interfaces:
         - name: ens1f0
           type: ethernet
           state: up
           ipv4:
             enabled: true
             dhcp: false
             auto-dns: false
             address:
               - ip: 192.168.200.25
                 prefix-length: 24
           ipv6:
             enabled: false
       dns-resolver:
         config:
           server:
             - 192.168.15.47
             - 192.168.15.48
       routes:
         config:
         - destination: 0.0.0.0/0
           metric: 150
           next-hop-address: 192.168.200.254
           next-hop-interface: ens1f0

   Copy to Clipboard Toggle word wrap

5. Create the configuration ISO in your working directory by running the following command:

   $ openshift-install image-based create config-image --dir ibi-config-iso-workdir/

   Copy to Clipboard Toggle word wrap

   Example output

   INFO Adding NMConnection file <ens1f0.nmconnection>
   INFO Consuming Install Config from target directory
   INFO Consuming Image-based Config ISO configuration from target directory
   INFO Config-Image created in: ibi-config-iso-workdir/auth

   Copy to Clipboard Toggle word wrap

   View the output in the working directory:

   Example output

   ibi-config-iso-workdir/
   ├── auth
   │   ├── kubeadmin-password
   │   └── kubeconfig
   └── imagebasedconfig.iso

   Copy to Clipboard Toggle word wrap

6. Attach the imagebasedconfig.iso to the preinstalled host using your preferred method and restart the host to complete the configuration process and deploy the cluster.

1. Export the kubeconfig environment variable to your kubeconfig file by running the following command:

   $ export KUBECONFIG=ibi-config-iso-workdir/auth/kubeconfig

   Copy to Clipboard Toggle word wrap

2. Verify that the cluster is responding by running the following command:

   $ oc get nodes

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                         STATUS   ROLES                  AGE     VERSION
   node/sno-cluster-name.host.example.com       Ready    control-plane,master   5h15m   v1.30.3

   Copy to Clipboard Toggle word wrap