Composing, installing, and managing RHEL for Edge images

Procedure

1. Create a Containerfile with the following structure:

   FROM registry.redhat.io/rhel10/rhel-bootc:latest
   
   RUN dnf -y install [software] [dependencies] && dnf clean all
   
   ADD [application]
   ADD [configuration files]
   
   RUN [config scripts]

2. Build the <image> image by using the Containerfile in the current directory:

   $ podman build -t quay.io/<namespace>/<image>:<tag> .

Procedure

1. Create a Containerfile with the following content:

   FROM registry.io.redhat.com/rhel10/rhel-bootc:latest
   
   RUN dnf install -y  \
       clevis  \
       clevis-dracut  \
       clevis-luks  \
       greenboot  \
       greenboot-default-health-checks  \
       fdo-client  \
       fdo-owner-cli
   # (Optional) Extra packages often used in edge
   # RUN dnf install -y
   #     dracut-config-generic  \
   #     platform-python  \
   #     pinentry  \
   #     firewalld  \
   #     iptables  \
   #     NetworkManager-wifi  \
   #     NetworkManager-wwan  \
   #     wpa_supplicant  \
   #     traceroute  \
   #     rootfiles  \
   #     policycoreutils-python-utils  \
   #     setools-console  \
   #     rsync  \
   #     usbguard
   RUN systemctl enable NetworkManager.service  \
       greenboot-grub2-set-counter.service  \
       greenboot-grub2-set-success.service  \
       greenboot-healthcheck.service  \
       greenboot-rpm-ostree-grub2-check-fallback.service  \
       greenboot-status.service  \
       greenboot-task-runner.service
       redboot-auto-reboot.service  \
       redboot-task-runner.service"

2. Build your similar RHEL for Edge customized bootc image:

   $ podman build -t quay.io/<namespace>/<image>:<tag> .

3. Optional: Push the image:

   $ podman push quay.io/<namespace>/<image>:<tag>

Procedure

1. Log in to authenticate to registry.redhat.io:

   $ sudo podman login registry.redhat.io

2. Install the bootc-image-builder tool:

   $ sudo podman pull registry.redhat.io/rhel10/bootc-image-builder

Procedure

1. Extract the information of users and groups from the RHEL for Edge system.

   $ mkdir -p ./usr/lib
   $ ssh admin@192.168.100.50 'cat /lib/passwd' > ./usr/lib/passwd
   $ ssh admin@192.168.100.50 'cat /lib/group' > ./usr/lib/group

2. Include the missing RHEL for Edge packages in the bootc based system by specifying them in a Containerfile. Additionally, use the COPY command to include the group and passwd content that was extracted from the RHEL for Edge system. The following is an example:

   FROM registry.redhat.io/rhel9/rhel-bootc
   WORKDIR /tmp
   RUN dnf -y install ModemManager \
                      NetworkManager-wifi \
                      NetworkManager-wwan \
                      audit \
                      checkpolicy \
                      clevis \
                      clevis-dracut \
                      clevis-luks \
                      clevis-pin-tpm2 \
                      clevis-systemd \
                      containernetworking-plugins \
                      dnsmasq \
                      dracut-config-generic \
                      fdo-client \
                      fdo-owner-cli \
                      firewalld \
                      firewalld-filesystem \
                      greenboot \
                      greenboot-default-health-checks \
                      grubby \
                      ignition \
                      ignition-edge \
                      ipset \
                      iwl100-firmware \
                      iwl1000-firmware \
                      iwl105-firmware \
                      iwl135-firmware \
                      iwl2000-firmware \
                      iwl2030-firmware \
                      iwl3160-firmware \
                      iwl5000-firmware \
                      iwl5150-firmware \
                      iwl6050-firmware \
                      iwl7260-firmware \
                      libsecret \
                      pinentry \
                      policycoreutils-python-utils \
                      python3-distro \
                      python3-setools \
                      rsync \
                      setools-console \
                      tmux \
                      traceroute \
                      usbguard \
                      usbguard-selinux \
                      wireless-regdb \
                      wpa_supplicant
   
   COPY etc /etc
   # You can find the passwd and group content that were extracted from the RHEL for Edge system usr/lib/ in your current working directory. You can copy the content into the container image with the following step:
   COPY usr /usr

3. Build the bootc image and push it to the registry:

   $ podman build -f Containerfile -t quay.io/<namespace>/<image>:<tag> .
   $ podman push quay.io/<namespace>/<image>:<tag>

4. Run the bootc switch command to the newly created bootable container image.

   $ ssh admin@192.168.100.50
   $ sudo bootc switch quay.io/<namespace>/<image>:<tag>
   $ sudo reboot

1. Check the content of /lib/passwd.

   $ cat /lib/passwd

2. Check the content of /lib/group.

   $ cat /lib/group

1. The Device contacts the Manufacturing server
2. The Manufacturing server generates an Ownership Voucher and the Device Credentials for the Device.
3. The Ownership Voucher is transferred to the Owner onboarding server.

4. The Owner onboarding server transfers the Ownership Voucher to the Rendezvous server, which makes a mapping of the Ownership Voucher to the Owner.
5. The device client reads device credentials.
6. The device client connects to the network.
7. After connecting to the network, the Device client contacts the Rendezvous server.
8. The Rendezvous server sends the owner endpoint URL to the Device Client, and registers the device.
9. The Device client connects to the Owner onboarding server shared by the Rendezvous server.
10. The Device proves that it is the correct device by signing a statement with a device key.
11. The Owner onboarding server proves itself correct by signing a statement with the last key of the Owner Voucher.
12. The Owner onboarding server transfers the information of the Device to the Service Info API server.
13. The Service info API server sends the configuration for the Device.
14. The Device is onboarded.

Procedure

1. Create a Containerfile to include the FDO client and kernel arguments. For example:

   $ cat Containerfile
   FROM registry.redhat.io/rhel10/rhel-bootc:10
   #fdo
   RUN dnf install -y fdo-init fdo-client && \
   systemctl enable fdo-client-linuxapp.service
   
   RUN mkdir -p /usr/lib/bootc/kargs.d && \ cat <<`EOF` >> /usr/lib/bootc/kargs.d/console.toml kargs = ["inst.device=/dev/sda"] EOF

2. Build the <image> container image by using the Containerfile in the current directory:

   $ podman build -t quay.io/<namespace>/<image>:<tag> .

3. Install the resulting image on the target device and power it on.

   After powering on, the system performs the following steps automatically:

   • The FDO client reaches out to the Manufacturing Server to exchange initial credentials.
   • The device contacts the Rendezvous Server to verify its voucher and locate its intended owner.
   • The device establishes mutual trust with the Owner Server. The service-info API then pushes the final configuration, such as SSH keys, user accounts, and encrypted filesystems, to the device.

Procedure

1. Generate the keys and certificates in the /etc/fdo directory:

   $ for i in "diun" "manufacturer" "device-ca" "owner"; do fdo-admin-tool generate-key-and-cert $i; done
   $ ls keys
   device_ca_cert.pem device_ca_key.der diun_cert.pem diun_key.der manufacturer_cert.pem manufacturer_key.der owner_cert.pem owner_key.der

2. Check the key and certificates that were created in the /etc/fdo/keys directory:

   $ tree keys

   You can see the following output:

   - device_ca_cert.pem
   - device_ca_key.der
   - diun_cert.pem
   - diun_key.dre
   - manufacturer_cert.pem
   - manufacturer_key.der
   - owner_cert.pem
   - owner_key.pem

   See the fdo-admin-tool generate-key-and-cert --help command output for more details.

Procedure

1. Install the fdo-admin-cli package:

   # dnf install -y fdo-admin-cli

2. Check if the fdo-manufacturing-server RPM package is installed:

   $ rpm -qa | grep fdo-manufacturing-server --refresh

3. Check if the files were correctly installed:

   $ *ls /usr/share/doc/fdo*SSSS

   You can see the following output:

   Output:
   manufacturing-server.yml
   owner-onboarding-server.yml
   rendezvous-info.yml
   rendezvous-server.yml
   serviceinfo-api-server.yml

4. Optional: Check the content of each file, for example:

   $ cat /usr/share/doc/fdo/manufacturing-server.yml

5. Configure the Manufacturing server. You must provide the following information:

   • The Manufacturing server URL
   • The IP address or DNS name for the Rendezvous server

   • The path to the keys and certificates that you generated.

     You can find an example of a Manufacturing server configuration file in the /usr/share/doc/fdo/manufacturing-server.yml directory. The following is a manufacturing server.yml example that is created and saved in the /etc/fdo directory. It contains paths to the directories, certificates, keys that you created, the Rendezvous server IP address and the default port.

     session_store_driver:
       Directory:
         path: /etc/fdo/stores/manufacturing_sessions/
     ownership_voucher_store_driver:
       Directory:
         path: /etc/fdo/stores/owner_vouchers
     public_key_store_driver:
       Directory:
         path: /etc/fdo/stores/manufacturer_keys
     bind: "0.0.0.0:8080"
     protocols:
       plain_di: false
       diun:
         mfg_string_type: SerialNumber
         key_type: SECP384R1
         allowed_key_storage_types:
           - Tpm
           - FileSystem
         key_path: /etc/fdo/keys/diun_key.der
         cert_path: /etc/fdo/keys/diun_cert.pem
     rendezvous_info:
       - deviceport: 8082
         ip_address: 192.168.122.99
         ownerport: 8082
         protocol: http
     manufacturing:
       manufacturer_cert_path: /etc/fdo/keys/manufacturer_cert.pem
       device_cert_ca_private_key: /etc/fdo/keys/device_ca_key.der
       device_cert_ca_chain: /etc/fdo/keys/device_ca_cert.pem
       owner_cert_path: /etc/fdo/keys/owner_cert.pem
       manufacturer_private_key: /etc/fdo/keys/manufacturer_key.der

6. Start the Manufacturing server.

   1. Check if the systemd unit file are in the server:

      # systemctl list-unit-files | grep fdo | grep manufacturing
      fdo-manufacturing-server.service disabled disabled

   2. Enable and start the manufacturing server.

      # systemctl enable --now fdo-manufacturing-server.service

   3. Open the default ports in your firewall:

      # firewall-cmd --add-port=8080/tcp --permanent
      # systemctl restart firewalld

   4. Ensure that the service is listening on the 8080 port:

      # ss -ltn

7. Install RHEL for Edge onto your system by using the RHEL for Edge Simplified Installer image.

Procedure

1. Install the fdo-rendezvous-server RPM packages:

   # dnf install -y fdo-rendezvous-server

2. Create the rendezvous-server.yml configuration file, including the path to the manufacturer certificate. You can find an example in /usr/share/doc/fdo/rendezvous-server.yml. The following example shows a configuration file that is saved in /etc/fdo/rendezvous-server.yml.

   storage_driver:
     Directory:
       path: /etc/fdo/stores/rendezvous_registered
   session_store_driver:
     Directory:
       path: /etc/fdo/stores/rendezvous_sessions
   trusted_manufacturer_keys_path: /etc/fdo/keys/manufacturer_cert.pem
   max_wait_seconds: ~
   bind: "0.0.0.0:8082"

3. Check the Rendezvous server service status:

   # systemctl list-unit-files | grep fdo | grep rende
   fdo-rendezvous-server.service disabled disabled

   1. If the service is stopped and disabled, enable and start it:

      # systemctl enable --now fdo-rendezvous-server.service

4. Check that the server is listening on the default configured port 8082:

   # ss -ltn

5. Open the port if you have a firewall configured on this server:

   # firewall-cmd --add-port=8082/tcp --permanent
   # systemctl restart firewalld

Procedure

1. Install the required RPMs in this server:

   # dnf install -y fdo-owner-cli fdo-owner-onboarding-server

2. Prepare the owner-onboarding-server.yml configuration file and save it to the /etc/fdo/ directory. Include the path to the certificates you already copied and information about where to publish the Owner server service in this file.

   The following is an example available in /usr/share/doc/fdo/owner-onboarding-server.yml. You can find references to the Service Info API, such as the URL or the authentication token.

   ---
   ownership_voucher_store_driver:
     Directory:
       path: /etc/fdo/stores/owner_vouchers
   session_store_driver:
     Directory:
       path: /etc/fdo/stores/owner_onboarding_sessions
   trusted_device_keys_path: /etc/fdo/keys/device_ca_cert.pem
   owner_private_key_path: /etc/fdo/keys/owner_key.der
   owner_public_key_path: /etc/fdo/keys/owner_cert.pem
   bind: "0.0.0.0:8081"
   service_info_api_url: "http://localhost:8083/device_info"
   service_info_api_authentication:
     BearerToken:
       token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo=
   owner_addresses:
     - transport: http
       addresses:
         - ip_address: 192.168.122.149

3. Create and configure the Service Info API.

   1. Add the automated information for onboarding, such as user creation, files to be copied or created, commands to be executed, disk to be encrypted, and so on. Use the Service Info API configuration file example in /usr/share/doc/fdo/serviceinfo-api-server.yml as a template to create the configuration file under /etc/fdo/.

      ---
      service_info:
        initial_user:
          username: admin
          sshkeys:
          - "ssh-rsa AAAA...."
        files:
        - path: /root/resolv.conf
          source_path: /etc/resolv.conf
        commands:
        - command: touch
          args:
          - /root/test
          return_stdout: true
          return_stderr: true
        diskencryption_clevis:
        - disk_label: /dev/vda4
          binding:
            pin: tpm2
            config: "{}"
          reencrypt: true
        additional_serviceinfo: ~
      bind: "0.0.0.0:8083"
      device_specific_store_driver:
        Directory:
          path: /etc/fdo/stores/serviceinfo_api_devices
      service_info_auth_token: Kpt5P/5flBkaiNSvDYS3cEdBQXJn2Zv9n1D50431/lo=
      admin_auth_token: zJNoErq7aa0RusJ1w0tkTjdITdMCWYkndzVv7F0V42Q=

4. Check the status of the systemd units:

   # systemctl list-unit-files | grep fdo
   fdo-owner-onboarding-server.service        disabled        disabled
   fdo-serviceinfo-api-server.service         disabled        disabled

   1. If the service is stopped and disabled, enable and start it:

      # systemctl enable --now fdo-owner-onboarding-server.service
      # systemctl enable --now fdo-serviceinfo-api-server.service

      Note

      You must restart the systemd services every time you change the configuration files.

5. Check that the server is listening on the default configured port 8083:

   # ss -ltn

6. Open the port if you have a firewall configured on this server:

   # firewall-cmd --add-port=8081/tcp --permanent
   # firewall-cmd --add-port=8083/tcp --permanent
   # systemctl restart firewalld

Procedure

1. Start the installation process by booting the RHEL for Edge image on your device. You can install it from a CD-ROM or from a USB flash drive, for example.

2. Verify through the terminal that the device has reached the manufacturing service to perform the initial device credential exchange and has produced an ownership voucher.

   You can find the ownership voucher at the storage location configured by the ownership_voucher_store_driver: parameter at the manufacturing-sever.yml file.

   The directory should have an ownership_voucher file with a name in the GUID format which indicates that the correct device credentials were added to the device.

   The onboarding server uses the device credential to authenticate against the onboarding server. It then passes the configuration to the device. After the device receives the configuration from the onboarding server, it receives an SSH key and installs the operating system on the device. Finally, the system automatically reboots, encrypts it with a strong key stored at TPM.

Procedure

1. Create a Containerfile, for example:

   FROM registry.redhat.io/rhel10/rhel-bootc:latest
   RUN dnf install -y fdo-init fdo-client
   RUN systemctl enable fdo-client-linuxapp.service

2. Create your Kickstart file. The following Kickstart file is an example of a fully unattended Kickstart file configuration that contains user creation and partition instructions.

   [customizations.installer.kickstart]
   contents = """
   text --non-interactive
   zerombr
   clearpart --all --initlabel --disklabel=gpt
   autopart --noswap --type=lvm
   user --name=test --groups=wheel --plaintext --password=test
   sshkey --username=test "ssh-ed25519 AAA..."
   network --bootproto=dhcp --device=link --activate --onboot=on
   poweroff
   
   %post
   export MANUFACTURING_SERVER_URL="http://192.168……"
   export DIUN_PUB_KEY_INSECURE="true"
   /usr/libexec/fdo/fdo-manufacturing-client
   %end
   
   """

   In the export <MANUFACTURING_SERVER_URL> field, replace the manufacturing server URL with your own manufacturing server URL.

3. Save the Kickstart configuration in the .toml format to inject the Kickstart content. For example, config.toml.

4. Create the following folder:

   $ mkdir $(pwd)/output"

5. Run bootc-image-builder, and include the Kickstart file configuration that you want to add to the ISO build. The bootc-image-builder automatically adds the ostreecontainer command that installs the container image.

   $ sudo podman run \
       --rm \
       -it \
       --privileged \
       --pull=newer \
       --security-opt label=type:unconfined_t \
       -v /var/lib/containers/storage:/var/lib/containers/storage \
       -v $(pwd)/config.toml:/config.toml \
       -v $(pwd)/output:/output \
       registry.redhat.io/rhel10/bootc-image-builder:latest \
       --type iso \
       --config /config.toml \
     quay.io/<namespace>/<image>:<tag>

   You can find the resulting .iso image in the output folder.