Red Hat Summit Red Hat SummitSupportoConsoleSviluppatoriInizia una provaContatti

Questo contenuto non è disponibile nella lingua selezionata.

Chapter 14. Pre-caching images for single-node OpenShift deployments

In environments with limited bandwidth where you use the GitOps Zero Touch Provisioning (ZTP) solution to deploy a large number of clusters, you want to avoid downloading all the images that are required for bootstrapping and installing OpenShift Container Platform. The limited bandwidth at remote single-node OpenShift sites can cause long deployment times. The factory-precaching-cli tool allows you to pre-stage servers before shipping them to the remote site for ZTP provisioning.

The factory-precaching-cli tool does the following:

  • Downloads the RHCOS rootfs image that is required by the minimal ISO to boot.
  • Creates a partition from the installation disk labelled as data.
  • Formats the disk in xfs.
  • Creates a GUID Partition Table (GPT) data partition at the end of the disk, where the size of the partition is configurable by the tool.
  • Copies the container images required to install OpenShift Container Platform.
  • Copies the container images required by ZTP to install OpenShift Container Platform.
  • Optional: Copies Day-2 Operators to the partition.
Important

The factory-precaching-cli tool is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

14.1. Getting the factory-precaching-cli tool
Copia collegamento

The factory-precaching-cli tool Go binary is publicly available in the {rds-first} tools container image. The factory-precaching-cli tool Go binary in the container image is executed on the server running an RHCOS live image using podman. If you are working in a disconnected environment or have a private registry, you need to copy the image there so you can download the image to the server.

Procedure

  • Pull the factory-precaching-cli tool image by running the following command: 

    # podman pull quay.io/openshift-kni/telco-ran-tools:latest
    Copy to Clipboard Toggle word wrap

Verification

  • To check that the tool is available, query the current version of the factory-precaching-cli tool Go binary: 

    # podman run quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli -v
    Copy to Clipboard Toggle word wrap

    Example output

    factory-precaching-cli version 20221018.120852+main.feecf17
    Copy to Clipboard Toggle word wrap

14.2. Booting from a live operating system image
Copia collegamento

You can use the factory-precaching-cli tool with to boot servers where only one disk is available and external disk drive cannot be attached to the server.

Warning

RHCOS requires the disk to not be in use when the disk is about to be written with an RHCOS image.

Depending on the server hardware, you can mount the RHCOS live ISO on the blank server using one of the following methods:

  • Using the Dell RACADM tool on a Dell server.
  • Using the HPONCFG tool on a HP server.
  • Using the Redfish BMC API.
Note

It is recommended to automate the mounting procedure. To automate the procedure, you need to pull the required images and host them on a local HTTP server.

Prerequisites

  • You powered up the host.
  • You have network connectivity to the host.
Procedure

This example procedure uses the Redfish BMC API to mount the RHCOS live ISO.

  1. Mount the RHCOS live ISO:

    1. Check virtual media status: 

      $ curl --globoff -H "Content-Type: application/json" -H \
"Accept: application/json" -k -X GET --user ${username_password} \
https://$BMC_ADDRESS/redfish/v1/Managers/Self/VirtualMedia/1 | python -m json.tool
      Copy to Clipboard Toggle word wrap

    2. Mount the ISO file as a virtual media: 

      $ curl --globoff -L -w "%{http_code} %{url_effective}\\n" -ku ${username_password} -H "Content-Type: application/json" -H "Accept: application/json" -d '{"Image": "http://[$HTTPd_IP]/RHCOS-live.iso"}' -X POST https://$BMC_ADDRESS/redfish/v1/Managers/Self/VirtualMedia/1/Actions/VirtualMedia.InsertMedia
      Copy to Clipboard Toggle word wrap

    3. Set the boot order to boot from the virtual media once: 

      $ curl --globoff  -L -w "%{http_code} %{url_effective}\\n"  -ku ${username_password}  -H "Content-Type: application/json" -H "Accept: application/json" -d '{"Boot":{ "BootSourceOverrideEnabled": "Once", "BootSourceOverrideTarget": "Cd", "BootSourceOverrideMode": "UEFI"}}' -X PATCH https://$BMC_ADDRESS/redfish/v1/Systems/Self
      Copy to Clipboard Toggle word wrap
  2. Reboot and ensure that the server is booting from virtual media.

14.3. Partitioning the disk
Copia collegamento

To run the full pre-caching process, you have to boot from a live ISO and use the factory-precaching-cli tool from a container image to partition and pre-cache all the artifacts required.

A live ISO or RHCOS live ISO is required because the disk must not be in use when the operating system (RHCOS) is written to the device during the provisioning. Single-disk servers can also be enabled with this procedure.

Prerequisites

  • You have a disk that is not partitioned.
  • You have access to the quay.io/openshift-kni/telco-ran-tools:latest image.
  • You have enough storage to install OpenShift Container Platform and pre-cache the required images.

Procedure

  1. Verify that the disk is cleared: 

    # lsblk
    Copy to Clipboard Toggle word wrap

    Example output

    NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0     7:0    0  93.8G  0 loop /run/ephemeral
loop1     7:1    0 897.3M  1 loop /sysroot
sr0      11:0    1   999M  0 rom  /run/media/iso
nvme0n1 259:1    0   1.5T  0 disk
    Copy to Clipboard Toggle word wrap

  2. Erase any file system, RAID or partition table signatures from the device: 

    # wipefs -a /dev/nvme0n1
    Copy to Clipboard Toggle word wrap

    Example output

    /dev/nvme0n1: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 8 bytes were erased at offset 0x1749a955e00 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
    Copy to Clipboard Toggle word wrap

Important

The tool fails if the disk is not empty because it uses partition number 1 of the device for pre-caching the artifacts.

14.3.1. Creating the partition
Copia collegamento

Once the device is ready, you create a single partition and a GPT partition table. The partition is automatically labelled as data and created at the end of the device. Otherwise, the partition will be overridden by the coreos-installer.

Important

The coreos-installer requires the partition to be created at the end of the device and to be labelled as data. Both requirements are necessary to save the partition when writing the RHCOS image to the disk.

Prerequisites

  • The container must run as privileged due to formatting host devices.
  • You have to mount the /dev folder so that the process can be executed inside the container.

Procedure

In the following example, the size of the partition is 250 GiB due to allow pre-caching the DU profile for Day 2 Operators.

  1. Run the container as privileged and partition the disk: 

    # podman run -v /dev:/dev --privileged \
--rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli partition \
    1 
    
-d /dev/nvme0n1 \
    2 
    
-s 250
    3
    Copy to Clipboard Toggle word wrap
    1
    Specifies the partitioning function of the factory-precaching-cli tool.
    2
    Defines the root directory on the disk.
    3
    Defines the size of the disk in GB.

  2. Check the storage information: 

    # lsblk
    Copy to Clipboard Toggle word wrap

    Example output

    NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0  93.8G  0 loop /run/ephemeral
loop1         7:1    0 897.3M  1 loop /sysroot
sr0          11:0    1   999M  0 rom  /run/media/iso
nvme0n1     259:1    0   1.5T  0 disk
└─nvme0n1p1 259:3    0   250G  0 part
    Copy to Clipboard Toggle word wrap

Verification

You must verify that the following requirements are met:

  • The device has a GPT partition table
  • The partition uses the latest sectors of the device.
  • The partition is correctly labeled as data.

Query the disk status to verify that the disk is partitioned as expected: 

# gdisk -l /dev/nvme0n1
Copy to Clipboard Toggle word wrap

Example output

GPT fdisk (gdisk) version 1.0.3

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/nvme0n1: 3125627568 sectors, 1.5 TiB
Model: Dell Express Flash PM1725b 1.6TB SFF
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): CB5A9D44-9B3C-4174-A5C1-C64957910B61
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 3125627534
Partitions will be aligned on 2048-sector boundaries
Total free space is 2601338846 sectors (1.2 TiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1      2601338880      3125627534   250.0 GiB   8300  data
Copy to Clipboard Toggle word wrap

14.3.2. Mounting the partition
Copia collegamento

After verifying that the disk is partitioned correctly, you can mount the device into /mnt.

Important

It is recommended to mount the device into /mnt because that mounting point is used during GitOps ZTP preparation.

  1. Verify that the partition is formatted as xfs: 

    # lsblk -f /dev/nvme0n1
    Copy to Clipboard Toggle word wrap

    Example output

    NAME        FSTYPE LABEL UUID                                 MOUNTPOINT
nvme0n1
└─nvme0n1p1 xfs          1bee8ea4-d6cf-4339-b690-a76594794071
    Copy to Clipboard Toggle word wrap

  2. Mount the partition: 

    # mount /dev/nvme0n1p1 /mnt/
    Copy to Clipboard Toggle word wrap

Verification

  • Check that the partition is mounted: 

    # lsblk
    Copy to Clipboard Toggle word wrap

    Example output

    NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0         7:0    0  93.8G  0 loop /run/ephemeral
loop1         7:1    0 897.3M  1 loop /sysroot
sr0          11:0    1   999M  0 rom  /run/media/iso
nvme0n1     259:1    0   1.5T  0 disk
└─nvme0n1p1 259:2    0   250G  0 part /var/mnt
    1
    Copy to Clipboard Toggle word wrap

    1
    The mount point is /var/mnt because the /mnt folder in RHCOS is a link to /var/mnt.

14.4. Downloading the images
Copia collegamento

The factory-precaching-cli tool allows you to download the following images to your partitioned server:

  • OpenShift Container Platform images
  • Operator images that are included in the distributed unit (DU) profile for 5G RAN sites
  • Operator images from disconnected registries
Note

The list of available Operator images can vary in different OpenShift Container Platform releases.

14.4.1. Downloading with parallel workers
Copia collegamento

The factory-precaching-cli tool uses parallel workers to download multiple images simultaneously. You can configure the number of workers with the --parallel or -p option. The default number is set to 80% of the available CPUs to the server.

Note

Your login shell may be restricted to a subset of CPUs, which reduces the CPUs available to the container. To remove this restriction, you can precede your commands with taskset 0xffffffff, for example: 

# taskset 0xffffffff podman run --rm quay.io/openshift-kni/telco-ran-tools:latest factory-precaching-cli download --help
Copy to Clipboard Toggle word wrap

14.4.2. Preparing to download the OpenShift Container Platform images
Copia collegamento

To download OpenShift Container Platform container images, you need to know the multicluster engine version. When you use the --du-profile flag, you also need to specify the Red Hat Advanced Cluster Management (RHACM) version running in the hub cluster that is going to provision the single-node OpenShift.

Prerequisites

  • You have RHACM and the multicluster engine Operator installed.
  • You partitioned the storage device.
  • You have enough space for the images on the partitioned device.
  • You connected the bare-metal server to the Internet.
  • You have a valid pull secret.

Procedure

  1. Check the RHACM version and the multicluster engine version by running the following commands in the hub cluster: 

    $ oc get csv -A | grep -i advanced-cluster-management
    Copy to Clipboard Toggle word wrap

    Example output

    open-cluster-management                            advanced-cluster-management.v2.6.3           Advanced Cluster Management for Kubernetes   2.6.3                 advanced-cluster-management.v2.6.3                Succeeded
    Copy to Clipboard Toggle word wrap 

    $ oc get csv -A | grep -i multicluster-engine
    Copy to Clipboard Toggle word wrap

    Example output

    multicluster-engine                                cluster-group-upgrades-operator.v0.0.3       cluster-group-upgrades-operator              0.0.3                                                                   Pending
multicluster-engine                                multicluster-engine.v2.1.4                   multicluster engine for Kubernetes           2.1.4                 multicluster-engine.v2.0.3                        Succeeded
multicluster-engine                                openshift-gitops-operator.v1.5.7             Red Hat OpenShift GitOps                     1.5.7                 openshift-gitops-operator.v1.5.6-0.1664915551.p   Succeeded
multicluster-engine                                openshift-pipelines-operator-rh.v1.6.4       Red Hat OpenShift Pipelines                  1.6.4                 openshift-pipelines-operator-rh.v1.6.3            Succeeded
    Copy to Clipboard Toggle word wrap

  2. To access the container registry, copy a valid pull secret on the server to be installed:

    1. Create the .docker folder: 

      $ mkdir /root/.docker
      Copy to Clipboard Toggle word wrap

    2. Copy the valid pull in the config.json file to the previously created .docker/ folder: 

      $ cp config.json /root/.docker/config.json
      1
      Copy to Clipboard Toggle word wrap
      1
      /root/.docker/config.json is the default path where podman checks for the login credentials for the registry.
Note

If you use a different registry to pull the required artifacts, you need to copy the proper pull secret. If the local registry uses TLS, you need to include the certificates from the registry as well.

14.4.3. Downloading the OpenShift Container Platform images
Copia collegamento

The factory-precaching-cli tool allows you to pre-cache all the container images required to provision a specific OpenShift Container Platform release.

Procedure

  • Pre-cache the release by running the following command: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools -- \
   factory-precaching-cli download \
    1 
    
   -r 4.18.0 \
    2 
    
   --acm-version 2.6.3 \
    3 
    
   --mce-version 2.1.4 \
    4 
    
   -f /mnt \
    5 
    
   --img quay.io/custom/repository
    6
    Copy to Clipboard Toggle word wrap
    1
    Specifies the downloading function of the factory-precaching-cli tool.
    2
    Defines the OpenShift Container Platform release version.
    3
    Defines the RHACM version.
    4
    Defines the multicluster engine version.
    5
    Defines the folder where you want to download the images on the disk.
    6
    Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk.

    Example output

    Generated /mnt/imageset.yaml
Generating list of pre-cached artifacts...
Processing artifact [1/176]: ocp-v4.0-art-dev@sha256_6ac2b96bf4899c01a87366fd0feae9f57b1b61878e3b5823da0c3f34f707fbf5
Processing artifact [2/176]: ocp-v4.0-art-dev@sha256_f48b68d5960ba903a0d018a10544ae08db5802e21c2fa5615a14fc58b1c1657c
Processing artifact [3/176]: ocp-v4.0-art-dev@sha256_a480390e91b1c07e10091c3da2257180654f6b2a735a4ad4c3b69dbdb77bbc06
Processing artifact [4/176]: ocp-v4.0-art-dev@sha256_ecc5d8dbd77e326dba6594ff8c2d091eefbc4d90c963a9a85b0b2f0e6155f995
Processing artifact [5/176]: ocp-v4.0-art-dev@sha256_274b6d561558a2f54db08ea96df9892315bb773fc203b1dbcea418d20f4c7ad1
Processing artifact [6/176]: ocp-v4.0-art-dev@sha256_e142bf5020f5ca0d1bdda0026bf97f89b72d21a97c9cc2dc71bf85050e822bbf
...
Processing artifact [175/176]: ocp-v4.0-art-dev@sha256_16cd7eda26f0fb0fc965a589e1e96ff8577e560fcd14f06b5fda1643036ed6c8
Processing artifact [176/176]: ocp-v4.0-art-dev@sha256_cf4d862b4a4170d4f611b39d06c31c97658e309724f9788e155999ae51e7188f
...
Summary:

Release:                            4.18.0
Hub Version:                        2.6.3
ACM Version:                        2.6.3
MCE Version:                        2.1.4
Include DU Profile:                 No
Workers:                            83
    Copy to Clipboard Toggle word wrap

Verification

  • Check that all the images are compressed in the target folder of server: 

    $ ls -l /mnt
    1
    Copy to Clipboard Toggle word wrap
    1
    It is recommended that you pre-cache the images in the /mnt folder.

    Example output

    -rw-r--r--. 1 root root  136352323 Oct 31 15:19 ocp-v4.0-art-dev@sha256_edec37e7cd8b1611d0031d45e7958361c65e2005f145b471a8108f1b54316c07.tgz
-rw-r--r--. 1 root root  156092894 Oct 31 15:33 ocp-v4.0-art-dev@sha256_ee51b062b9c3c9f4fe77bd5b3cc9a3b12355d040119a1434425a824f137c61a9.tgz
-rw-r--r--. 1 root root  172297800 Oct 31 15:29 ocp-v4.0-art-dev@sha256_ef23d9057c367a36e4a5c4877d23ee097a731e1186ed28a26c8d21501cd82718.tgz
-rw-r--r--. 1 root root  171539614 Oct 31 15:23 ocp-v4.0-art-dev@sha256_f0497bb63ef6834a619d4208be9da459510df697596b891c0c633da144dbb025.tgz
-rw-r--r--. 1 root root  160399150 Oct 31 15:20 ocp-v4.0-art-dev@sha256_f0c339da117cde44c9aae8d0bd054bceb6f19fdb191928f6912a703182330ac2.tgz
-rw-r--r--. 1 root root  175962005 Oct 31 15:17 ocp-v4.0-art-dev@sha256_f19dd2e80fb41ef31d62bb8c08b339c50d193fdb10fc39cc15b353cbbfeb9b24.tgz
-rw-r--r--. 1 root root  174942008 Oct 31 15:33 ocp-v4.0-art-dev@sha256_f1dbb81fa1aa724e96dd2b296b855ff52a565fbef003d08030d63590ae6454df.tgz
-rw-r--r--. 1 root root  246693315 Oct 31 15:31 ocp-v4.0-art-dev@sha256_f44dcf2c94e4fd843cbbf9b11128df2ba856cd813786e42e3da1fdfb0f6ddd01.tgz
-rw-r--r--. 1 root root  170148293 Oct 31 15:00 ocp-v4.0-art-dev@sha256_f48b68d5960ba903a0d018a10544ae08db5802e21c2fa5615a14fc58b1c1657c.tgz
-rw-r--r--. 1 root root  168899617 Oct 31 15:16 ocp-v4.0-art-dev@sha256_f5099b0989120a8d08a963601214b5c5cb23417a707a8624b7eb52ab788a7f75.tgz
-rw-r--r--. 1 root root  176592362 Oct 31 15:05 ocp-v4.0-art-dev@sha256_f68c0e6f5e17b0b0f7ab2d4c39559ea89f900751e64b97cb42311a478338d9c3.tgz
-rw-r--r--. 1 root root  157937478 Oct 31 15:37 ocp-v4.0-art-dev@sha256_f7ba33a6a9db9cfc4b0ab0f368569e19b9fa08f4c01a0d5f6a243d61ab781bd8.tgz
-rw-r--r--. 1 root root  145535253 Oct 31 15:26 ocp-v4.0-art-dev@sha256_f8f098911d670287826e9499806553f7a1dd3e2b5332abbec740008c36e84de5.tgz
-rw-r--r--. 1 root root  158048761 Oct 31 15:40 ocp-v4.0-art-dev@sha256_f914228ddbb99120986262168a705903a9f49724ffa958bb4bf12b2ec1d7fb47.tgz
-rw-r--r--. 1 root root  167914526 Oct 31 15:37 ocp-v4.0-art-dev@sha256_fa3ca9401c7a9efda0502240aeb8d3ae2d239d38890454f17fe5158b62305010.tgz
-rw-r--r--. 1 root root  164432422 Oct 31 15:24 ocp-v4.0-art-dev@sha256_fc4783b446c70df30b3120685254b40ce13ba6a2b0bf8fb1645f116cf6a392f1.tgz
-rw-r--r--. 1 root root  306643814 Oct 31 15:11 troubleshoot@sha256_b86b8aea29a818a9c22944fd18243fa0347c7a2bf1ad8864113ff2bb2d8e0726.tgz
    Copy to Clipboard Toggle word wrap

14.4.4. Downloading the Operator images
Copia collegamento

You can also pre-cache Day-2 Operators used in the 5G Radio Access Network (RAN) Distributed Unit (DU) cluster configuration. The Day-2 Operators depend on the installed OpenShift Container Platform version.

Important

You need to include the RHACM hub and multicluster engine Operator versions by using the --acm-version and --mce-version flags so the factory-precaching-cli tool can pre-cache the appropriate containers images for RHACM and the multicluster engine Operator.

Procedure

  • Pre-cache the Operator images: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download \
    1 
    
   -r 4.18.0 \
    2 
    
   --acm-version 2.6.3 \
    3 
    
   --mce-version 2.1.4 \
    4 
    
   -f /mnt \
    5 
    
   --img quay.io/custom/repository
    6 
    
   --du-profile -s
    7
    Copy to Clipboard Toggle word wrap
    1
    Specifies the downloading function of the factory-precaching-cli tool.
    2
    Defines the OpenShift Container Platform release version.
    3
    Defines the RHACM version.
    4
    Defines the multicluster engine version.
    5
    Defines the folder where you want to download the images on the disk.
    6
    Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk.
    7
    Specifies pre-caching the Operators included in the DU configuration.

    Example output

    Generated /mnt/imageset.yaml
Generating list of pre-cached artifacts...
Processing artifact [1/379]: ocp-v4.0-art-dev@sha256_7753a8d9dd5974be8c90649aadd7c914a3d8a1f1e016774c7ac7c9422e9f9958
Processing artifact [2/379]: ose-kube-rbac-proxy@sha256_c27a7c01e5968aff16b6bb6670423f992d1a1de1a16e7e260d12908d3322431c
Processing artifact [3/379]: ocp-v4.0-art-dev@sha256_370e47a14c798ca3f8707a38b28cfc28114f492bb35fe1112e55d1eb51022c99
...
Processing artifact [378/379]: ose-local-storage-operator@sha256_0c81c2b79f79307305e51ce9d3837657cf9ba5866194e464b4d1b299f85034d0
Processing artifact [379/379]: multicluster-operators-channel-rhel8@sha256_c10f6bbb84fe36e05816e873a72188018856ad6aac6cc16271a1b3966f73ceb3
...
Summary:

Release:                            4.18.0
Hub Version:                        2.6.3
ACM Version:                        2.6.3
MCE Version:                        2.1.4
Include DU Profile:                 Yes
Workers:                            83
    Copy to Clipboard Toggle word wrap

14.4.5. Pre-caching custom images in disconnected environments
Copia collegamento

The --generate-imageset argument stops the factory-precaching-cli tool after the ImageSetConfiguration custom resource (CR) is generated. This allows you to customize the ImageSetConfiguration CR before downloading any images. After you customized the CR, you can use the --skip-imageset argument to download the images that you specified in the ImageSetConfiguration CR.

You can customize the ImageSetConfiguration CR in the following ways:

  • Add Operators and additional images
  • Remove Operators and additional images
  • Change Operator and catalog sources to local or disconnected registries

Procedure

  1. Pre-cache the images: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download \
    1 
    
   -r 4.18.0 \
    2 
    
   --acm-version 2.6.3 \
    3 
    
   --mce-version 2.1.4 \
    4 
    
   -f /mnt \
    5 
    
   --img quay.io/custom/repository
    6 
    
   --du-profile -s \
    7 
    
   --generate-imageset
    8
    Copy to Clipboard Toggle word wrap
    1
    Specifies the downloading function of the factory-precaching-cli tool.
    2
    Defines the OpenShift Container Platform release version.
    3
    Defines the RHACM version.
    4
    Defines the multicluster engine version.
    5
    Defines the folder where you want to download the images on the disk.
    6
    Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk.
    7
    Specifies pre-caching the Operators included in the DU configuration.
    8
    The --generate-imageset argument generates the ImageSetConfiguration CR only, which allows you to customize the CR.

    Example output

    Generated /mnt/imageset.yaml
    Copy to Clipboard Toggle word wrap

    Example ImageSetConfiguration CR

    apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
mirror:
  platform:
    channels:
    - name: stable-4.18
      minVersion: 4.18.0
    1 
    
      maxVersion: 4.18.0
  additionalImages:
    - name: quay.io/custom/repository
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.18
      packages:
        - name: advanced-cluster-management
    2 
    
          channels:
             - name: 'release-2.6'
               minVersion: 2.6.3
               maxVersion: 2.6.3
        - name: multicluster-engine
    3 
    
          channels:
             - name: 'stable-2.1'
               minVersion: 2.1.4
               maxVersion: 2.1.4
        - name: local-storage-operator
    4 
    
          channels:
            - name: 'stable'
        - name: ptp-operator
    5 
    
          channels:
            - name: 'stable'
        - name: sriov-network-operator
    6 
    
          channels:
            - name: 'stable'
        - name: cluster-logging
    7 
    
          channels:
            - name: 'stable'
        - name: lvms-operator
    8 
    
          channels:
            - name: 'stable-4.18'
        - name: amq7-interconnect-operator
    9 
    
          channels:
            - name: '1.10.x'
        - name: bare-metal-event-relay
    10 
    
          channels:
            - name: 'stable'
    - catalog: registry.redhat.io/redhat/certified-operator-index:v4.18
      packages:
        - name: sriov-fec
    11 
    
          channels:
            - name: 'stable'
    Copy to Clipboard Toggle word wrap

    1
    The platform versions match the versions passed to the tool.
    2 3
    The versions of RHACM and the multicluster engine Operator match the versions passed to the tool.
    4 5 6 7 8 9 10 11
    The CR contains all the specified DU Operators.

  2. Customize the catalog resource in the CR: 

    apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
mirror:
  platform:
[...]
  operators:
    - catalog: eko4.cloud.lab.eng.bos.redhat.com:8443/redhat/certified-operator-index:v4.18
      packages:
        - name: sriov-fec
          channels:
            - name: 'stable'
    Copy to Clipboard Toggle word wrap

    When you download images by using a local or disconnected registry, you have to first add certificates for the registries that you want to pull the content from.

  3. To avoid any errors, copy the registry certificate into your server: 

    # cp /tmp/eko4-ca.crt /etc/pki/ca-trust/source/anchors/.
    Copy to Clipboard Toggle word wrap

  4. Then, update the certificates trust store: 

    # update-ca-trust
    Copy to Clipboard Toggle word wrap

  5. Mount the host /etc/pki folder into the factory-cli image: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker -v /etc/pki:/etc/pki --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli download \
    1 
    
   -r 4.18.0 \
    2 
    
   --acm-version 2.6.3 \
    3 
    
   --mce-version 2.1.4 \
    4 
    
   -f /mnt \
    5 
    
   --img quay.io/custom/repository
    6 
    
   --du-profile -s \
    7 
    
   --skip-imageset
    8
    Copy to Clipboard Toggle word wrap
    1
    Specifies the downloading function of the factory-precaching-cli tool.
    2
    Defines the OpenShift Container Platform release version.
    3
    Defines the RHACM version.
    4
    Defines the multicluster engine version.
    5
    Defines the folder where you want to download the images on the disk.
    6
    Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk.
    7
    Specifies pre-caching the Operators included in the DU configuration.
    8
    The --skip-imageset argument allows you to download the images that you specified in your customized ImageSetConfiguration CR.

  6. Download the images without generating a new imageSetConfiguration CR: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download -r 4.18.0 \
--acm-version 2.6.3 --mce-version 2.1.4 -f /mnt \
--img quay.io/custom/repository \
--du-profile -s \
--skip-imageset
    Copy to Clipboard Toggle word wrap

14.5. Pre-caching images in GitOps ZTP
Copia collegamento

The SiteConfig manifest defines how an OpenShift cluster is to be installed and configured. In the GitOps Zero Touch Provisioning (ZTP) provisioning workflow, the factory-precaching-cli tool requires the following additional fields in the SiteConfig manifest:

  • clusters.ignitionConfigOverride
  • nodes.installerArgs
  • nodes.ignitionConfigOverride
Important

SiteConfig v1 is deprecated starting with OpenShift Container Platform version 4.18. Equivalent and improved functionality is now available through the SiteConfig Operator using the ClusterInstance custom resource. For more information, see Procedure to transition from SiteConfig CRs to the ClusterInstance API.

For more information about the SiteConfig Operator, see SiteConfig.

Example SiteConfig with additional fields

apiVersion: ran.openshift.io/v1
kind: SiteConfig
metadata:
  name: "example-5g-lab"
  namespace: "example-5g-lab"
spec:
  baseDomain: "example.domain.redhat.com"
  pullSecretRef:
    name: "assisted-deployment-pull-secret"
  clusterImageSetNameRef: "img4.9.10-x86-64-appsub"
1 

  sshPublicKey: "ssh-rsa ..."
  clusters:
  - clusterName: "sno-worker-0"
    clusterImageSetNameRef: "eko4-img4.11.5-x86-64-appsub"
2 

    clusterLabels:
      group-du-sno: ""
      common-411: true
      sites : "example-5g-lab"
      vendor: "OpenShift"
    clusterNetwork:
      - cidr: 10.128.0.0/14
        hostPrefix: 23
    machineNetwork:
      - cidr: 10.19.32.192/26
    serviceNetwork:
      - 172.30.0.0/16
    networkType: "OVNKubernetes"
    additionalNTPSources:
      - clock.corp.redhat.com
    ignitionConfigOverride:
      '{
        "ignition": {
          "version": "3.1.0"
        },
        "systemd": {
          "units": [
            {
              "name": "var-mnt.mount",
              "enabled": true,
              "contents": "[Unit]\nDescription=Mount partition with artifacts\nBefore=precache-images.service\nBindsTo=precache-images.service\nStopWhenUnneeded=true\n\n[Mount]\nWhat=/dev/disk/by-partlabel/data\nWhere=/var/mnt\nType=xfs\nTimeoutSec=30\n\n[Install]\nRequiredBy=precache-images.service"
            },
            {
              "name": "precache-images.service",
              "enabled": true,
              "contents": "[Unit]\nDescription=Extracts the precached images in discovery stage\nAfter=var-mnt.mount\nBefore=agent.service\n\n[Service]\nType=oneshot\nUser=root\nWorkingDirectory=/var/mnt\nExecStart=bash /usr/local/bin/extract-ai.sh\n#TimeoutStopSec=30\n\n[Install]\nWantedBy=multi-user.target default.target\nWantedBy=agent.service"
            }
          ]
        },
        "storage": {
          "files": [
            {
              "overwrite": true,
              "path": "/usr/local/bin/extract-ai.sh",
              "mode": 755,
              "user": {
                "name": "root"
              },
              "contents": {
                "source": "data:,%23%21%2Fbin%2Fbash%0A%0AFOLDER%3D%22%24%7BFOLDER%3A-%24%28pwd%29%7D%22%0AOCP_RELEASE_LIST%3D%22%24%7BOCP_RELEASE_LIST%3A-ai-images.txt%7D%22%0ABINARY_FOLDER%3D%2Fvar%2Fmnt%0A%0Apushd%20%24FOLDER%0A%0Atotal_copies%3D%24%28sort%20-u%20%24BINARY_FOLDER%2F%24OCP_RELEASE_LIST%20%7C%20wc%20-l%29%20%20%23%20Required%20to%20keep%20track%20of%20the%20pull%20task%20vs%20total%0Acurrent_copy%3D1%0A%0Awhile%20read%20-r%20line%3B%0Ado%0A%20%20uri%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%241%7D%27%29%0A%20%20%23tar%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%242%7D%27%29%0A%20%20podman%20image%20exists%20%24uri%0A%20%20if%20%5B%5B%20%24%3F%20-eq%200%20%5D%5D%3B%20then%0A%20%20%20%20%20%20echo%20%22Skipping%20existing%20image%20%24tar%22%0A%20%20%20%20%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20%20%20%20%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%0A%20%20%20%20%20%20continue%0A%20%20fi%0A%20%20tar%3D%24%28echo%20%22%24uri%22%20%7C%20%20rev%20%7C%20cut%20-d%20%22%2F%22%20-f1%20%7C%20rev%20%7C%20tr%20%22%3A%22%20%22_%22%29%0A%20%20tar%20zxvf%20%24%7Btar%7D.tgz%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-f%20%24%7Btar%7D.gz%3B%20fi%0A%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20skopeo%20copy%20dir%3A%2F%2F%24%28pwd%29%2F%24%7Btar%7D%20containers-storage%3A%24%7Buri%7D%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-rf%20%24%7Btar%7D%3B%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%3B%20fi%0Adone%20%3C%20%24%7BBINARY_FOLDER%7D%2F%24%7BOCP_RELEASE_LIST%7D%0A%0A%23%20workaround%20while%20https%3A%2F%2Fgithub.com%2Fopenshift%2Fassisted-service%2Fpull%2F3546%0A%23cp%20%2Fvar%2Fmnt%2Fmodified-rhcos-4.10.3-x86_64-metal.x86_64.raw.gz%20%2Fvar%2Ftmp%2F.%0A%0Aexit%200"
              }
            },
            {
              "overwrite": true,
              "path": "/usr/local/bin/agent-fix-bz1964591",
              "mode": 755,
              "user": {
                "name": "root"
              },
              "contents": {
                "source": "data:,%23%21%2Fusr%2Fbin%2Fsh%0A%0A%23%20This%20script%20is%20a%20workaround%20for%20bugzilla%201964591%20where%20symlinks%20inside%20%2Fvar%2Flib%2Fcontainers%2F%20get%0A%23%20corrupted%20under%20some%20circumstances.%0A%23%0A%23%20In%20order%20to%20let%20agent.service%20start%20correctly%20we%20are%20checking%20here%20whether%20the%20requested%0A%23%20container%20image%20exists%20and%20in%20case%20%22podman%20images%22%20returns%20an%20error%20we%20try%20removing%20the%20faulty%0A%23%20image.%0A%23%0A%23%20In%20such%20a%20scenario%20agent.service%20will%20detect%20the%20image%20is%20not%20present%20and%20pull%20it%20again.%20In%20case%0A%23%20the%20image%20is%20present%20and%20can%20be%20detected%20correctly%2C%20no%20any%20action%20is%20required.%0A%0AIMAGE%3D%24%28echo%20%241%20%7C%20sed%20%27s%2F%3A.%2A%2F%2F%27%29%0Apodman%20image%20exists%20%24IMAGE%20%7C%7C%20echo%20%22already%20loaded%22%20%7C%7C%20echo%20%22need%20to%20be%20pulled%22%0A%23podman%20images%20%7C%20grep%20%24IMAGE%20%7C%7C%20podman%20rmi%20--force%20%241%20%7C%7C%20true"
              }
            }
          ]
        }
      }'
    nodes:
      - hostName: "snonode.sno-worker-0.example.domain.redhat.com"
        role: "master"
        bmcAddress: "idrac-virtualmedia+https://10.19.28.53/redfish/v1/Systems/System.Embedded.1"
        bmcCredentialsName:
          name: "worker0-bmh-secret"
        bootMACAddress: "e4:43:4b:bd:90:46"
        bootMode: "UEFI"
        rootDeviceHints:
          deviceName: /dev/disk/by-path/pci-0000:01:00.0-scsi-0:2:0:0
        installerArgs: '["--save-partlabel", "data"]'
        ignitionConfigOverride: |
           {
            "ignition": {
              "version": "3.1.0"
            },
            "systemd": {
              "units": [
                {
                  "name": "var-mnt.mount",
                  "enabled": true,
                  "contents": "[Unit]\nDescription=Mount partition with artifacts\nBefore=precache-ocp-images.service\nBindsTo=precache-ocp-images.service\nStopWhenUnneeded=true\n\n[Mount]\nWhat=/dev/disk/by-partlabel/data\nWhere=/var/mnt\nType=xfs\nTimeoutSec=30\n\n[Install]\nRequiredBy=precache-ocp-images.service"
                },
                {
                  "name": "precache-ocp-images.service",
                  "enabled": true,
                  "contents": "[Unit]\nDescription=Extracts the precached OCP images into containers storage\nAfter=var-mnt.mount\nBefore=machine-config-daemon-pull.service nodeip-configuration.service\n\n[Service]\nType=oneshot\nUser=root\nWorkingDirectory=/var/mnt\nExecStart=bash /usr/local/bin/extract-ocp.sh\nTimeoutStopSec=60\n\n[Install]\nWantedBy=multi-user.target"
                }
              ]
            },
            "storage": {
              "files": [
                {
                  "overwrite": true,
                  "path": "/usr/local/bin/extract-ocp.sh",
                  "mode": 755,
                  "user": {
                    "name": "root"
                  },
                  "contents": {
                    "source": "data:,%23%21%2Fbin%2Fbash%0A%0AFOLDER%3D%22%24%7BFOLDER%3A-%24%28pwd%29%7D%22%0AOCP_RELEASE_LIST%3D%22%24%7BOCP_RELEASE_LIST%3A-ocp-images.txt%7D%22%0ABINARY_FOLDER%3D%2Fvar%2Fmnt%0A%0Apushd%20%24FOLDER%0A%0Atotal_copies%3D%24%28sort%20-u%20%24BINARY_FOLDER%2F%24OCP_RELEASE_LIST%20%7C%20wc%20-l%29%20%20%23%20Required%20to%20keep%20track%20of%20the%20pull%20task%20vs%20total%0Acurrent_copy%3D1%0A%0Awhile%20read%20-r%20line%3B%0Ado%0A%20%20uri%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%241%7D%27%29%0A%20%20%23tar%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%242%7D%27%29%0A%20%20podman%20image%20exists%20%24uri%0A%20%20if%20%5B%5B%20%24%3F%20-eq%200%20%5D%5D%3B%20then%0A%20%20%20%20%20%20echo%20%22Skipping%20existing%20image%20%24tar%22%0A%20%20%20%20%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20%20%20%20%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%0A%20%20%20%20%20%20continue%0A%20%20fi%0A%20%20tar%3D%24%28echo%20%22%24uri%22%20%7C%20%20rev%20%7C%20cut%20-d%20%22%2F%22%20-f1%20%7C%20rev%20%7C%20tr%20%22%3A%22%20%22_%22%29%0A%20%20tar%20zxvf%20%24%7Btar%7D.tgz%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-f%20%24%7Btar%7D.gz%3B%20fi%0A%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20skopeo%20copy%20dir%3A%2F%2F%24%28pwd%29%2F%24%7Btar%7D%20containers-storage%3A%24%7Buri%7D%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-rf%20%24%7Btar%7D%3B%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%3B%20fi%0Adone%20%3C%20%24%7BBINARY_FOLDER%7D%2F%24%7BOCP_RELEASE_LIST%7D%0A%0Aexit%200"
                  }
                }
              ]
            }
           }
        nodeNetwork:
          config:
            interfaces:
              - name: ens1f0
                type: ethernet
                state: up
                macAddress: "AA:BB:CC:11:22:33"
                ipv4:
                  enabled: true
                  dhcp: true
                ipv6:
                  enabled: false
          interfaces:
            - name: "ens1f0"
              macAddress: "AA:BB:CC:11:22:33"
Copy to Clipboard Toggle word wrap

1
Specifies the cluster image set used for deployment, unless you specify a different image set in the spec.clusters.clusterImageSetNameRef field.
2
Specifies the cluster image set used to deploy an individual cluster. If defined, it overrides the spec.clusterImageSetNameRef at the site level.

14.5.1. Understanding the clusters.ignitionConfigOverride field
Copia collegamento

The clusters.ignitionConfigOverride field adds a configuration in Ignition format during the GitOps ZTP discovery stage. The configuration includes systemd services in the ISO mounted in virtual media. This way, the scripts are part of the discovery RHCOS live ISO and they can be used to load the Assisted Installer (AI) images.

systemd services
The systemd services are var-mnt.mount and precache-images.services. The precache-images.service depends on the disk partition to be mounted in /var/mnt by the var-mnt.mount unit. The service calls a script called extract-ai.sh.
extract-ai.sh
The extract-ai.sh script extracts and loads the required images from the disk partition to the local container storage. When the script finishes successfully, you can use the images locally.
agent-fix-bz1964591
The agent-fix-bz1964591 script is a workaround for an AI issue. To prevent AI from removing the images, which can force the agent.service to pull the images again from the registry, the agent-fix-bz1964591 script checks if the requested container images exist.

14.5.2. Understanding the nodes.installerArgs field
Copia collegamento

The nodes.installerArgs field allows you to configure how the coreos-installer utility writes the RHCOS live ISO to disk. You need to indicate to save the disk partition labeled as data because the artifacts saved in the data partition are needed during the OpenShift Container Platform installation stage.

The extra parameters are passed directly to the coreos-installer utility that writes the live RHCOS to disk. On the next reboot, the operating system starts from the disk.

You can pass several options to the coreos-installer utility: 

OPTIONS:
...
    -u, --image-url <URL>
            Manually specify the image URL

    -f, --image-file <path>
            Manually specify a local image file

    -i, --ignition-file <path>
            Embed an Ignition config from a file

    -I, --ignition-url <URL>
            Embed an Ignition config from a URL
...
        --save-partlabel <lx>...
            Save partitions with this label glob

        --save-partindex <id>...
            Save partitions with this number or range
...
        --insecure-ignition
            Allow Ignition URL without HTTPS or hash
Copy to Clipboard Toggle word wrap

14.5.3. Understanding the nodes.ignitionConfigOverride field
Copia collegamento

Similarly to clusters.ignitionConfigOverride, the nodes.ignitionConfigOverride field allows the addition of configurations in Ignition format to the coreos-installer utility, but at the OpenShift Container Platform installation stage. When the RHCOS is written to disk, the extra configuration included in the GitOps ZTP discovery ISO is no longer available. During the discovery stage, the extra configuration is stored in the memory of the live OS.

Note

At this stage, the number of container images extracted and loaded is bigger than in the discovery stage. Depending on the OpenShift Container Platform release and whether you install the Day-2 Operators, the installation time can vary.

At the installation stage, the var-mnt.mount and precache-ocp.services systemd services are used.

precache-ocp.service

The precache-ocp.service depends on the disk partition to be mounted in /var/mnt by the var-mnt.mount unit. The precache-ocp.service service calls a script called extract-ocp.sh.

Important

To extract all the images before the OpenShift Container Platform installation, you must execute precache-ocp.service before executing the machine-config-daemon-pull.service and nodeip-configuration.service services.

extract-ocp.sh
The extract-ocp.sh script extracts and loads the required images from the disk partition to the local container storage.

When you commit the SiteConfig and optional PolicyGenerator or PolicyGenTemplate custom resources (CRs) to the Git repo that Argo CD is monitoring, you can start the GitOps ZTP workflow by syncing the CRs with the hub cluster.

14.6. Troubleshooting a "Rendered catalog is invalid" error
Copia collegamento

When you download images by using a local or disconnected registry, you might see the The rendered catalog is invalid error. This means that you are missing certificates of the new registry you want to pull content from.

Note

The factory-precaching-cli tool image is built on a UBI RHEL image. Certificate paths and locations are the same on RHCOS.

Example error

Generating list of pre-cached artifacts...
error: unable to run command oc-mirror -c /mnt/imageset.yaml file:///tmp/fp-cli-3218002584/mirror --ignore-history --dry-run: Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/publish
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/v2
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/charts
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/release-signatures
backend is not configured in /mnt/imageset.yaml, using stateless mode
backend is not configured in /mnt/imageset.yaml, using stateless mode
No metadata detected, creating new workspace
level=info msg=trying next host error=failed to do request: Head "https://eko4.cloud.lab.eng.bos.redhat.com:8443/v2/redhat/redhat-operator-index/manifests/v4.11": x509: certificate signed by unknown authority host=eko4.cloud.lab.eng.bos.redhat.com:8443

The rendered catalog is invalid.

Run "oc-mirror list operators --catalog CATALOG-NAME --package PACKAGE-NAME" for more information.

error: error rendering new refs: render reference "eko4.cloud.lab.eng.bos.redhat.com:8443/redhat/redhat-operator-index:v4.11": error resolving name : failed to do request: Head "https://eko4.cloud.lab.eng.bos.redhat.com:8443/v2/redhat/redhat-operator-index/manifests/v4.11": x509: certificate signed by unknown authority
Copy to Clipboard Toggle word wrap

Procedure

  1. Copy the registry certificate into your server: 

    # cp /tmp/eko4-ca.crt /etc/pki/ca-trust/source/anchors/.
    Copy to Clipboard Toggle word wrap

  2. Update the certificates truststore: 

    # update-ca-trust
    Copy to Clipboard Toggle word wrap

  3. Mount the host /etc/pki folder into the factory-cli image: 

    # podman run -v /mnt:/mnt -v /root/.docker:/root/.docker -v /etc/pki:/etc/pki --privileged -it --rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli download -r 4.18.0 --acm-version 2.5.4 \
   --mce-version 2.0.4 -f /mnt \--img quay.io/custom/repository
   --du-profile -s --skip-imageset
    Copy to Clipboard Toggle word wrap
Torna in cima
Red Hat logoGithubredditYoutubeTwitter

Formazione

Prova, acquista e vendi

Community

Informazioni sulla documentazione di Red Hat

Aiutiamo gli utenti Red Hat a innovarsi e raggiungere i propri obiettivi con i nostri prodotti e servizi grazie a contenuti di cui possono fidarsi. Esplora i nostri ultimi aggiornamenti.

Rendiamo l’open source più inclusivo

Red Hat si impegna a sostituire il linguaggio problematico nel codice, nella documentazione e nelle proprietà web. Per maggiori dettagli, visita il Blog di Red Hat.

Informazioni su Red Hat

Forniamo soluzioni consolidate che rendono più semplice per le aziende lavorare su piattaforme e ambienti diversi, dal datacenter centrale all'edge della rete.

Theme

© 2025 Red Hat