Installing

Procedure

1. In the graphical installer under Installation Destination in the Storage Configuration subsection, select Custom → Done to open the dialog for configuring partitions and volumes. The Manual Partitioning window is displayed.
2. Under New Red Hat Enterprise Linux 9.x Installation, select Click here to create them automatically.
3. Select the root partition, /, reduce Desired Capacity so that the VG has sufficient capacity for your PVs, and then click Update Settings.

4. Complete your installation.

   Note

   For more options on partition configuration, read the guide linked in the Additional information section for Configuring Manual Partitioning.

5. As a root user, verify the VG capacity available on your system by running the following command:

   $ sudo vgs

   Example output:

   VG   #PV #LV #SN Attr   VSize    VFree
   rhel   1   2   0 wz--n- <127.00g 54.94g

Procedure

1. As a root user, enable the Red Hat build of MicroShift repositories by running the following command:

   $ sudo subscription-manager repos \
       --enable rhocp-4.14-for-rhel-9-$(uname -m)-rpms \
       --enable fast-datapath-for-rhel-9-$(uname -m)-rpms

2. Install Red Hat build of MicroShift by running the following command:

   $ sudo dnf install -y microshift

3. Download your installation pull secret from the Red Hat Hybrid Cloud Console to a temporary folder, for example, $HOME/openshift-pull-secret. This pull secret allows you to authenticate with the container registries that serve the container images used by Red Hat build of MicroShift.

4. To copy the pull secret to the /etc/crio folder of your RHEL machine, run the following command:

   $ sudo cp $HOME/openshift-pull-secret /etc/crio/openshift-pull-secret

5. Make the root user the owner of the /etc/crio/openshift-pull-secret file by running the following command:

   $ sudo chown root:root /etc/crio/openshift-pull-secret

6. Make the /etc/crio/openshift-pull-secret file readable and writeable by the root user only by running the following command:

   $ sudo chmod 600 /etc/crio/openshift-pull-secret

7. If your RHEL machine has a firewall enabled, you must configure a few mandatory firewall rules. For firewalld, run the following commands:

   $ sudo firewall-cmd --permanent --zone=trusted --add-source=10.42.0.0/16

   $ sudo firewall-cmd --permanent --zone=trusted --add-source=169.254.169.1

   $ sudo firewall-cmd --reload

Procedure

1. As a root user, start the MicroShift service by entering the following command:

   $ sudo systemctl start microshift

2. Optional: To configure your RHEL machine to start MicroShift when your machine starts, enter the following command:

   $ sudo systemctl enable microshift

3. Optional: To disable MicroShift from automatically starting when your machine starts, enter the following command:

   $ sudo systemctl disable microshift

   Note

   The first time that the MicroShift service starts, it downloads and initializes the container images for MicroShift. As a result, it can take several minutes for MicroShift to start the first time that the service is deployed. Boot time is reduced for subsequent starts of the MicroShift service.

Procedure

1. Enter the following command to stop the MicroShift service:

   $ sudo systemctl stop microshift

2. Workloads deployed on MicroShift might continue running even after the MicroShift service has been stopped. Enter the following command to display running workloads:

   $ sudo crictl ps -a

3. Enter the following commands to stop the deployed workloads:

   $ sudo systemctl stop kubepods.slice

Procedure

1. Access the list of container image references by using one of the following methods:

   • If the package is installed on the MicroShift host, get the location of the files by running the following command:

     $ rpm -ql microshift-release-info

     Example output

     /usr/share/microshift/release/release-x86_64.json

   • If the package is not installed on a MicroShift host, download and unpack the RPM package without installing it by running the following command:

     $ rpm2cpio microshift-release-info*.noarch.rpm | cpio -idmv

     Example output

     /usr/share/microshift/release/release-x86_64.json

2. Extract the list of container images into the microshift-container-refs.txt file by running the following commands:

   $ RELEASE_FILE=/usr/share/microshift/release/release-$(uname -m).json

   $ jq -r '.images | .[]' ${RELEASE_FILE} > microshift-container-refs.txt

Procedure

1. Install the skopeo tool used for copying the container images by running the following command:

   $ sudo dnf install -y skopeo

2. Set the environment variable that points to the pull secret file:

   $ PULL_SECRET_FILE=~/.pull-secret-mirror.json

3. Set the environment variable that points to the list of container images:

   $ IMAGE_LIST_FILE=~/microshift-container-refs.txt

4. Set the environment variable that points to the destination directory for storing the downloaded data:

   $ IMAGE_LOCAL_DIR=~/microshift-containers

5. Run the following script to download the container images to the ${IMAGE_LOCAL_DIR} directory:

   while read -r src_img ; do
      # Remove the source registry prefix
      dst_img=$(echo "${src_img}" | cut -d '/' -f 2-)
   
      # Run the image download command
      echo "Downloading '${src_img}' to '${IMAGE_LOCAL_DIR}'"
      mkdir -p "${IMAGE_LOCAL_DIR}/${dst_img}"
      skopeo copy --all --quiet \
         --preserve-digests \
         --authfile "${PULL_SECRET_FILE}" \
         docker://"${src_img}" dir://"${IMAGE_LOCAL_DIR}/${dst_img}"
   
   done < "${IMAGE_LIST_FILE}"

6. Transfer the image set to the target environment, such as air-gapped site. Then you can upload the image set into the mirror registry.

Procedure

1. Install the skopeo tool used for copying the container images by running the following command:

   $ sudo dnf install -y skopeo

2. Set the environment variables pointing to the pull secret file:

   $ IMAGE_PULL_FILE=~/.pull-secret-mirror.json

3. Set the environment variables pointing to the local container image directory:

   $ IMAGE_LOCAL_DIR=~/microshift-containers

4. Set the environment variables pointing to the mirror registry URL for uploading the container images:

   $ TARGET_REGISTRY=<registry_host>:<port> 1

   1

   Replace <registry_host>:<port> with the host name and port of your mirror registry server.

5. Run the following script to upload the container images to the ${TARGET_REGISTRY} mirror registry:

   image_tag=mirror-$(date +%y%m%d%H%M%S)
   image_cnt=1
      # Uses timestamp and counter as a tag on the target images to avoid
      # their overwrite by the 'latest' automatic tagging
   
   pushd "${IMAGE_LOCAL_DIR}" >/dev/null
   while read -r src_manifest ; do
      # Remove the manifest.json file name
      src_img=$(dirname "${src_manifest}")
      # Add the target registry prefix and remove SHA
      dst_img="${TARGET_REGISTRY}/${src_img}"
      dst_img=$(echo "${dst_img}" | awk -F'@' '{print $1}')
   
      # Run the image upload command
      echo "Uploading '${src_img}' to '${dst_img}'"
      skopeo copy --all --quiet \
         --preserve-digests \
         --authfile "${IMAGE_PULL_FILE}" \
         dir://"${IMAGE_LOCAL_DIR}/${src_img}" docker://"${dst_img}:${image_tag}-${image_cnt}"
      # Increment the counter
      (( image_cnt += 1 ))
   
   done < <(find . -type f -name manifest.json -printf '%P\n')
   popd >/dev/null

Procedure

1. Log into your MicroShift host.

2. Enable the SSL certificate trust on any host accessing the mirror registry by completing the following steps:

   1. Copy the rootCA.pem file from the mirror registry, for example, <registry_path>/quay-rootCA, to the MicroShift host at the /etc/pki/ca-trust/source/anchors directory.

   2. Enable the certificate in the system-wide trust store configuration by running the following command:

      $ sudo update-ca-trust

3. Create the /etc/containers/registries.conf.d/999-microshift-mirror.conf configuration file that maps the Red Hat registry host names to the mirror registry:

   Example mirror configuration file

   [[registry]]
       prefix = ""
       location = "<registry_host>:<port>" 1
       mirror-by-digest-only = true
       insecure = false
   
   [[registry]]
       prefix = ""
       location = "quay.io"
       mirror-by-digest-only = true
   [[registry.mirror]]
       location = "<registry_host>:<port>"
       insecure = false
   
   [[registry]]
       prefix = ""
       location = "registry.redhat.io"
       mirror-by-digest-only = true
   [[registry.mirror]]
       location = "<registry_host>:<port>"
       insecure = false
   
   [[registry]]
       prefix = ""
       location = "registry.access.redhat.com"
       mirror-by-digest-only = true
   [[registry.mirror]]
       location = "<registry_host>:<port>"
       insecure = false

   1

   Replace <registry_host>:<port> with the host name and port of your mirror registry server, for example, <microshift-quay:8443>.

4. Enable the MicroShift service by running the following command:

   $ sudo systemctl enable microshift

5. Reboot the host by running the following command:

   $ sudo reboot

Procedure

1. Create an Image Builder configuration file for adding the rhocp-4.14 RPM repository source required to pull MicroShift RPMs by running the following command:

   cat > rhocp-4.14.toml <<EOF
   id = "rhocp-4.14"
   name = "Red Hat OpenShift Container Platform 4.14 for RHEL 9"
   type = "yum-baseurl"
   url = "https://cdn.redhat.com/content/dist/layered/rhel9/$(uname -m)/rhocp/4.14/os"
   check_gpg = true
   check_ssl = true
   system = false
   rhsm = true
   EOF

2. Create an Image Builder configuration file for adding the fast-datapath RPM repository by running the following command:

   cat > fast-datapath.toml <<EOF
   id = "fast-datapath"
   name = "Fast Datapath for RHEL 9"
   type = "yum-baseurl"
   url = "https://cdn.redhat.com/content/dist/layered/rhel9/$(uname -m)/fast-datapath/os"
   check_gpg = true
   check_ssl = true
   system = false
   rhsm = true
   EOF

3. Add the sources to the Image Builder by running the following commands:

   $ sudo composer-cli sources add rhocp-4.14.toml

   $ sudo composer-cli sources add fast-datapath.toml

Procedure

1. Use the following example to create your blueprint:

   Image Builder blueprint example

   cat > minimal-microshift.toml <<EOF
   name = "minimal-microshift"
   
   description = ""
   version = "0.0.1"
   modules = []
   groups = []
   
   [[packages]]
   name = "microshift"
   version = "*"
   
   [customizations.services]
   enabled = ["microshift"]
   EOF

   Note

   The wildcard * in the commands uses the latest MicroShift RPMs. If you need a specific version, substitute the wildcard for the version you want. For example, insert 4.14.1 to download the MicroShift 4.14.1 RPMs.

2. Add the blueprint to the Image Builder by running the following command:

   $ sudo composer-cli blueprints push minimal-microshift.toml

Verification

1. Verify the Image Builder configuration listing only MicroShift packages by running the following command:

   $ sudo composer-cli blueprints depsolve minimal-microshift | grep microshift

   Example output

   blueprint: minimal-microshift v0.0.1
       microshift-greenboot-4.13.1-202305250827.p0.g4105d3b.assembly.4.13.1.el9.noarch
       microshift-networking-4.13.1-202305250827.p0.g4105d3b.assembly.4.13.1.el9.x86_64
       microshift-release-info-4.13.1-202305250827.p0.g4105d3b.assembly.4.13.1.el9.noarch
       microshift-4.13.1-202305250827.p0.g4105d3b.assembly.4.13.1.el9.x86_64
       microshift-selinux-4.13.1-202305250827.p0.g4105d3b.assembly.4.13.1.el9.noarch

2. Optional: Verify the Image Builder configuration listing all components to be installed by running the following command:

   $ sudo composer-cli blueprints depsolve minimal-microshift

Procedure

1. Start an ostree container image build by running the following command:

   $ BUILDID=$(sudo composer-cli compose start-ostree --ref "rhel/{op-system-version-major}/$(uname -m)/edge" <microshift_blueprint> edge-container | awk '/^Compose/ {print $2}') 1

   1

   Replace <microshift_blueprint> with the name of your blueprint.

   This command also returns the identification (ID) of the build for monitoring.

2. You can check the status of the build periodically by running the following command:

   $ sudo composer-cli compose status

   Example output of a running build

   ID                                     Status     Time                     Blueprint                 Version   Type               Size
   cc3377ec-4643-4483-b0e7-6b0ad0ae6332   RUNNING    Wed Jun 7 12:26:23 2023  microshift_blueprint      0.0.1     edge-container

   Example output of a completed build

   ID                                     Status     Time                      Blueprint              Version   Type               Size
   cc3377ec-4643-4483-b0e7-6b0ad0ae6332   FINISHED   Wed Jun 7 12:32:37 2023   microshift_blueprint   0.0.1     edge-container

   Note

   You can use the watch command to monitor your build if you are familiar with how to start and stop it.

3. Download the container image using the ID and get the image ready for use by running the following command:

   $ sudo composer-cli compose image ${BUILDID}

4. Change the ownership of the downloaded container image to the current user by running the following command:

   $ sudo chown $(whoami). ${BUILDID}-container.tar

5. Add read permissions for the current user to the image by running the following command:

   $ sudo chmod a+r ${BUILDID}-container.tar

6. Bootstrap a server on port 8085 for the ostree container image to be consumed by the ISO build by completing the following steps:

   1. Get the IMAGEID variable result by running the following command:

      $ IMAGEID=$(cat < "./${BUILDID}-container.tar" | sudo podman load | grep -o -P '(?<=sha256[@:])[a-z0-9]*')

   2. Use the IMAGEID variable result to execute the podman command step by running the following command:

      $ sudo podman run -d --name=minimal-microshift-server -p 8085:8080 ${IMAGEID}

      This command also returns the ID of the container saved in the IMAGEID variable for monitoring.

7. Generate the installer blueprint file by running the following command:

   cat > microshift-installer.toml <<EOF
   name = "microshift-installer"
   
   description = ""
   version = "0.0.0"
   modules = []
   groups = []
   packages = []
   EOF

1. Add the blueprint to the Image Builder by running the following command:

   $ sudo composer-cli blueprints push microshift-installer.toml

2. Start the ostree ISO build by running the following command:

   $ BUILDID=$(sudo composer-cli compose start-ostree --url http://localhost:8085/repo/ --ref "rhel/9/$(uname -m)/edge" microshift-installer edge-installer | awk '{print $2}')

   This command also returns the identification (ID) of the build for monitoring.

3. You can check the status of the build periodically by running the following command:

   $ sudo composer-cli compose status

   Example output for a running build

   ID                                     Status     Time                      Blueprint              Version   Type               Size
   c793c24f-ca2c-4c79-b5b7-ba36f5078e8d   RUNNING    Wed Jun 7 13:22:20 2023   microshift-installer   0.0.0     edge-installer

   Example output for a completed build

   ID                                     Status     Time                      Blueprint              Version   Type               Size
   c793c24f-ca2c-4c79-b5b7-ba36f5078e8d   FINISHED   Wed Jun 7 13:34:49 2023   microshift-installer   0.0.0     edge-installer

1. Download the ISO using the ID by running the following command:

   $ sudo composer-cli compose image ${BUILDID}

2. Change the ownership of the downloaded container image to the current user by running the following command:

   $ sudo chown $(whoami). ${BUILDID}-installer.iso

3. Add read permissions for the current user to the image by running the following command:

   $ sudo chmod a+r ${BUILDID}-installer.iso

Prerequisites

1. You have created an RHEL for Edge Installer (ISO) image containing your RHEL for Edge commit with Red Hat build of MicroShift.

   1. This requirement includes the steps of composing an RFE Container image, creating the RFE Installer blueprint, starting the RFE container, and composing the RFE Installer image.

2. Create a Kickstart file or use an existing one. In the Kickstart file, you must include:

   1. Detailed instructions about how to create a user.
   2. How to fetch and deploy the RHEL for Edge image.

Procedure

1. In the main section of the Kickstart file, update the setup of the filesystem such that it contains an LVM volume group called rhel with at least 10GB system root. Leave free space for the LVMS CSI driver to use for storing the data for your workloads.

   Example kickstart snippet for configuring the filesystem

   # Partition disk such that it contains an LVM volume group called `rhel` with a
   # 10GB+ system root but leaving free space for the LVMS CSI driver for storing data.
   #
   # For example, a 20GB disk would be partitioned in the following way:
   #
   # NAME          MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
   # sda             8:0    0  20G  0 disk
   # ├─sda1          8:1    0 200M  0 part /boot/efi
   # ├─sda1          8:1    0 800M  0 part /boot
   # └─sda2          8:2    0  19G  0 part
   #  └─rhel-root  253:0    0  10G  0 lvm  /sysroot
   #
   ostreesetup --nogpg --osname=rhel --remote=edge \
   --url=file:///run/install/repo/ostree/repo --ref=rhel/<RHEL VERSION NUMBER>/x86_64/edge
   zerombr
   clearpart --all --initlabel
   part /boot/efi --fstype=efi --size=200
   part /boot --fstype=xfs --asprimary --size=800
   # Uncomment this line to add a SWAP partition of the recommended size
   #part swap --fstype=swap --recommended
   part pv.01 --grow
   volgroup rhel pv.01
   logvol / --vgname=rhel --fstype=xfs --size=10000 --name=root
   # To add users, use a line such as the following
   user --name=<YOUR_USER_NAME> \
   --password=<YOUR_HASHED_PASSWORD> \
   --iscrypted --groups=<YOUR_USER_GROUPS>

2. In the %post section of the Kickstart file, add your pull secret and the mandatory firewall rules.

   Example Kickstart snippet for adding the pull secret and firewall rules

   %post --log=/var/log/anaconda/post-install.log --erroronfail
   
   # Add the pull secret to CRI-O and set root user-only read/write permissions
   cat > /etc/crio/openshift-pull-secret << EOF
   YOUR_OPENSHIFT_PULL_SECRET_HERE
   EOF
   chmod 600 /etc/crio/openshift-pull-secret
   
   # Configure the firewall with the mandatory rules for MicroShift
   firewall-offline-cmd --zone=trusted --add-source=10.42.0.0/16
   firewall-offline-cmd --zone=trusted --add-source=169.254.169.1
   
   %end

3. Install the mkksiso tool by running the following command:

   $ sudo yum install -y lorax

4. Update the Kickstart file in the ISO with your new Kickstart file by running the following command:

   $ sudo mkksiso <your_kickstart>.ks <your_installer>.iso <updated_installer>.iso

Procedure

1. Get the exact list of container image references used by the MicroShift version you are deploying. You can either install the microshift-release-info RPM package by following step 2 or download and unpack the RPM by following step 3.

2. To install the microshift-release-info RPM package:

   1. Install the microshift-release-info RPM package by running the following command:

      $ sudo dnf install -y microshift-release-info-<release_version>

      Replace <release_version> with the numerical value of the release you are deploying, using the entire version number, such as 4.14.0.

   2. List the contents of the /usr/share/microshift/release directory to verify the presence of the release information files by running the following command:

      $ ls /usr/share/microshift/release

      Example output

      release-x86_64.json
      release-aarch64.json

      If you installed the microshift-release-info RPM, you can proceed to step 4.

3. If you did not complete step 2, download and unpack the microshift-release-info RPM without installing it:

   1. Download the RPM package by running the following command:

      $ sudo dnf download microshift-release-info-<release_version>

      Replace <release_version> with the numerical value of the release you are deploying, using the entire version number, such as 4.14.0.

      Example rpm

      microshift-release-info-4.14.0.*.el9.noarch.rpm 1

      1

      The * represents the date and commit ID. Your output should contain both, for example -202311101230.p0.g7dc6a00.assembly.4.14.0.

   2. Unpack the RPM package without installing it by running the following command:

      $ rpm2cpio <my_microshift_release_info> | cpio -idmv 1
      ./usr/share/microshift/release/release-aarch64.json
      ./usr/share/microshift/release/release-x86_64.json

      1

      Replace <my_microshift_release_info> with the name of the RPM package from the previous step.

4. Define the location of your JSON file, which contains the container reference information, by running the following command:

   $ RELEASE_FILE=</path/to/your/release-$(uname -m).json>

   Replace </path/to/your/release-$(uname -m).json> with the full path to your JSON file. Be sure to use the file needed for your architecture.

5. Define the location of your TOML file, which contains instructions for building the image, by running the following command:

   $ BLUEPRINT_FILE=</path/to/your/blueprint.toml>

   Replace </path/to/your/blueprint.toml> with the full path to your JSON file.

6. Generate and then embed the container image references in your blueprint TOML file by running the following command:

   $  jq -r '.images | .[] | ("[[containers]]\nsource = \"" + . + "\"\n")' "${RELEASE_FILE}" >> "${BLUEPRINT_FILE}"

   Example resulting <my_blueprint.toml> fragment showing container references

   [[containers]]
   source = "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cfef91557f9a70cff5a90accba45841a37524e9b93f98a97b20f6b2b69e5db"
   
   [[containers]]
   source = "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:82cfef91557f9a70cff5a90accba45841a37524e9b93f98a97b20f6b2b69e5db"

7. You can manually embed any container image by adding it to the Image Builder blueprint using the following example:

   Example section for manually embedding container image to Image Builder

   [[containers]]
   source = "<my_image_pullspec_with_tag_or_digest>"

   Replace <my_image_pullspec_with_tag_or_digest> with the exact reference to a container image used by the MicroShift version you are deploying.

Procedure

1. Add a pull secret for authenticating to the registry by setting the auth_file_path in the [containers] section of the /etc/osbuild-worker/osbuild-worker.toml osbuilder worker configuration file:

   [containers]
   auth_file_path = "/etc/osbuild-worker/pull-secret.json"

2. Restart the osbuild-worker to apply configuration changes by restarting the host. Restarting the host ensures that all osbuild-worker services currently running are restarted.

1. System as expected: When all of the scripts in the required.d directory are successfully run, greenboot runs any scripts present in the /etc/greenboot/green.d directory.
2. System trouble: If any of the scripts in the required.d directory fail, greenboot runs any prerollback scripts present in the red.d directory, then restarts the system.

Procedure

1. Make the directory by running the following command:

   $ sudo mkdir -p /etc/systemd/journald.conf.d

2. Create the configuration file by running the following command:

   cat <<EOF | sudo tee /etc/systemd/journald.conf.d/microshift.conf &>/dev/null
   [Journal]
   Storage=persistent
   SystemMaxUse=1G
   RuntimeMaxUse=1G
   EOF

3. Edit the configuration file values for your size requirements.

Procedure

1. Log into the failing host as a root user.

2. Perform the debug report creation procedure by running the following command:

   $ microshift-sos-report

   Example output

   sosreport (version 4.5.1)
   
   This command will collect diagnostic and configuration information from
   this Red Hat Enterprise Linux system and installed applications.
   
   An archive containing the collected information will be generated in
   /var/tmp/sos.o0sznf_8 and may be provided to a Red Hat support
   representative.
   
   Any information provided to Red Hat will be treated in accordance with
   the published support policies at:
   
           Distribution Website : https://www.redhat.com/
           Commercial Support   : https://www.access.redhat.com/
   
   The generated archive may contain data considered sensitive and its
   content should be reviewed by the originating organization before being
   passed to any third party.
   
   No changes will be made to system configuration.
   
   
    Setting up archive ...
    Setting up plugins ...
    Running plugins. Please wait ...
   
     Starting 1/2   microshift      [Running: microshift]
     Starting 2/2   microshift_ovn  [Running: microshift microshift_ovn]
     Finishing plugins              [Running: microshift]
   
     Finished running plugins
   
   Found 1 total reports to obfuscate, processing up to 4 concurrently
   
   sosreport-microshift-rhel9-2023-03-31-axjbyxw :    Beginning obfuscation...
   sosreport-microshift-rhel9-2023-03-31-axjbyxw :    Obfuscation completed
   
   Successfully obfuscated 1 report(s)
   
   Creating compressed archive...
   
   A mapping of obfuscated elements is available at
   	/var/tmp/sosreport-microshift-rhel9-2023-03-31-axjbyxw-private_map
   
   Your sosreport has been generated and saved in:
   	/var/tmp/sosreport-microshift-rhel9-2023-03-31-axjbyxw-obfuscated.tar.xz
   
    Size	444.14KiB
    Owner	root
    sha256	922e5ff2db25014585b7c6c749d2c44c8492756d619df5e9838ce863f83d4269
   
   Please send this file to your support representative.