Chapter 15. Installing a cluster on AWS with compute nodes on AWS Wavelength Zones

Procedure

1. List the zones that are available in your AWS Region by running the following command:

   Example command for listing available AWS Wavelength Zones in an AWS Region

   $ aws --region "<value_of_AWS_Region>" ec2 describe-availability-zones \
       --query 'AvailabilityZones[].[{ZoneName: ZoneName, GroupName: GroupName, Status: OptInStatus}]' \
       --filters Name=zone-type,Values=wavelength-zone \
       --all-availability-zones

   Depending on the AWS Region, the list of available zones might be long. The command returns the following fields:

   ZoneName

   The name of the Wavelength Zones.

   GroupName

   The group that comprises the zone. To opt in to the Region, save the name.

   Status

   The status of the Wavelength Zones group. If the status is not-opted-in, you must opt in the GroupName as described in the next step.

2. Opt in to the zone group on your AWS account by running the following command:

   $ aws ec2 modify-availability-zone-group \
       --group-name "<value_of_GroupName>" \1
       --opt-in-status opted-in

   1

   Replace <value_of_GroupName> with the name of the group of the Wavelength Zones where you want to create subnets. As an example for Wavelength Zones, specify us-east-1-wl1 to use the zone us-east-1-wl1-nyc-wlz-1 (US East New York).

Procedure

1. Complete the OpenShift Container Platform subscription from the AWS Marketplace.

2. Record the AMI ID for your specific AWS Region. As part of the installation process, you must update the install-config.yaml file with this value before deploying the cluster.

   Sample install-config.yaml file with AWS Marketplace compute nodes

   apiVersion: v1
   baseDomain: example.com
   compute:
   - hyperthreading: Enabled
     name: worker
     platform:
       aws:
         amiID: ami-06c4d345f7c207239 1
         type: m5.4xlarge
     replicas: 3
   metadata:
     name: test-cluster
   platform:
     aws:
       region: us-east-2 2
   sshKey: ssh-ed25519 AAAA...
   pullSecret: '{"auths": ...}'

   1

   The AMI ID from your AWS Marketplace subscription.

   2

   Your AMI ID is associated with a specific AWS Region. When creating the installation configuration file, ensure that you select the same AWS Region that you specified when configuring your subscription.

Procedure

1. Navigate to the OpenShift Container Platform downloads page on the Red Hat Customer Portal.
2. Select the architecture from the Product Variant drop-down list.
3. Select the appropriate version from the Version drop-down list.
4. Click Download Now next to the OpenShift v4.15 Linux Client entry and save the file.

5. Unpack the archive:

   $ tar xvf <file>

6. Place the oc binary in a directory that is on your PATH.

   To check your PATH, execute the following command:

   $ echo $PATH

Procedure

1. Navigate to the OpenShift Container Platform downloads page on the Red Hat Customer Portal.
2. Select the appropriate version from the Version drop-down list.
3. Click Download Now next to the OpenShift v4.15 Windows Client entry and save the file.
4. Unzip the archive with a ZIP program.

5. Move the oc binary to a directory that is on your PATH.

   To check your PATH, open the command prompt and execute the following command:

   C:\> path

Procedure

1. Navigate to the OpenShift Container Platform downloads page on the Red Hat Customer Portal.
2. Select the appropriate version from the Version drop-down list.

3. Click Download Now next to the OpenShift v4.15 macOS Client entry and save the file.

   Note

   For macOS arm64, choose the OpenShift v4.15 macOS arm64 Client entry.

4. Unpack and unzip the archive.

5. Move the oc binary to a directory on your PATH.

   To check your PATH, open a terminal and execute the following command:

   $ echo $PATH

Procedure

1. Go to the Cluster Type page on the Red Hat Hybrid Cloud Console. If you have a Red Hat account, log in with your credentials. If you do not, create an account.
2. Select your infrastructure provider from the Run it yourself section of the page.
3. Select your host operating system and architecture from the dropdown menus under OpenShift Installer and click Download Installer.

4. Place the downloaded file in the directory where you want to store the installation configuration files.

   Important
   • The installation program creates several files on the computer that you use to install your cluster. You must keep the installation program and the files that the installation program creates after you finish installing the cluster. Both of the files are required to delete the cluster.
   • Deleting the files created by the installation program does not remove your cluster, even if the cluster failed during installation. To remove your cluster, complete the OpenShift Container Platform uninstallation procedures for your specific cloud provider.

5. Extract the installation program. For example, on a computer that uses a Linux operating system, run the following command:

   $ tar -xvf openshift-install-linux.tar.gz

6. Download your installation pull secret from Red Hat OpenShift Cluster Manager. This pull secret allows you to authenticate with the services that are provided by the included authorities, including Quay.io, which serves the container images for OpenShift Container Platform components.

Procedure

1. If you do not have an existing SSH key pair on your local machine to use for authentication onto your cluster nodes, create one. For example, on a computer that uses a Linux operating system, run the following command:

   $ ssh-keygen -t ed25519 -N '' -f <path>/<file_name> 1

   1

   Specify the path and file name, such as ~/.ssh/id_ed25519, of the new SSH key. If you have an existing key pair, ensure your public key is in the your ~/.ssh directory.

   Note

   If you plan to install an OpenShift Container Platform cluster that uses the RHEL cryptographic libraries that have been submitted to NIST for FIPS 140-2/140-3 Validation on only the x86_64, ppc64le, and s390x architectures, do not create a key that uses the ed25519 algorithm. Instead, create a key that uses the rsa or ecdsa algorithm.

2. View the public SSH key:

   $ cat <path>/<file_name>.pub

   For example, run the following to view the ~/.ssh/id_ed25519.pub public key:

   $ cat ~/.ssh/id_ed25519.pub

3. Add the SSH private key identity to the SSH agent for your local user, if it has not already been added. SSH agent management of the key is required for password-less SSH authentication onto your cluster nodes, or if you want to use the ./openshift-install gather command.

   Note

   On some distributions, default SSH private key identities such as ~/.ssh/id_rsa and ~/.ssh/id_dsa are managed automatically.

   1. If the ssh-agent process is not already running for your local user, start it as a background task:

      $ eval "$(ssh-agent -s)"

      Example output

      Agent pid 31874

      Note

      If your cluster is in FIPS mode, only use FIPS-compliant algorithms to generate the SSH key. The key must be either RSA or ECDSA.

4. Add your SSH private key to the ssh-agent:

   $ ssh-add <path>/<file_name> 1

   1

   Specify the path and file name for your SSH private key, such as ~/.ssh/id_ed25519

   Example output

   Identity added: /home/<you>/<path>/<file_name> (<computer_name>)

1. One vCPU is equivalent to one physical core when simultaneous multithreading (SMT), or Hyper-Threading, is not enabled. When enabled, use the following formula to calculate the corresponding ratio: (threads per core × cores) × sockets = vCPUs.
2. OpenShift Container Platform and Kubernetes are sensitive to disk performance, and faster storage is recommended, particularly for etcd on the control plane nodes which require a 10 ms p99 fsync duration. Note that on many cloud platforms, storage size and IOPS scale together, so you might need to over-allocate storage volume to obtain sufficient performance.
3. As with all user-provisioned installations, if you choose to use RHEL compute machines in your cluster, you take responsibility for all operating system life cycle management and maintenance, including performing system updates, applying patches, and completing all other required tasks. Use of RHEL 7 compute machines is deprecated and has been removed in OpenShift Container Platform 4.10 and later.

Procedure

1. Create the install-config.yaml file.

   1. Change to the directory that contains the installation program and run the following command:

      $ ./openshift-install create install-config --dir <installation_directory> 1

      1

      For <installation_directory>, specify the directory name to store the files that the installation program creates.

      Important

      Specify an empty directory. Some installation assets, like bootstrap X.509 certificates have short expiration intervals, so you must not reuse an installation directory. If you want to reuse individual files from another cluster installation, you can copy them into your directory. However, the file names for the installation assets might change between releases. Use caution when copying installation files from an earlier OpenShift Container Platform version.

   2. At the prompts, provide the configuration details for your cloud:

      1. Optional: Select an SSH key to use to access your cluster machines.

         Note

         For production OpenShift Container Platform clusters on which you want to perform installation debugging or disaster recovery, specify an SSH key that your ssh-agent process uses.

      2. Select aws as the platform to target.

      3. If you do not have an AWS profile stored on your computer, enter the AWS access key ID and secret access key for the user that you configured to run the installation program.

         Note

         The AWS access key ID and secret access key are stored in ~/.aws/credentials in the home directory of the current user on the installation host. You are prompted for the credentials by the installation program if the credentials for the exported profile are not present in the file. Any credentials that you provide to the installation program are stored in the file.

      4. Select the AWS Region to deploy the cluster to.
      5. Select the base domain for the Route 53 service that you configured for your cluster.
      6. Enter a descriptive name for your cluster.
      7. Paste the pull secret from Red Hat OpenShift Cluster Manager.

2. Optional: Back up the install-config.yaml file.

   Important

   The install-config.yaml file is consumed during the installation process. If you want to reuse the file, you must back it up now.

Procedure

1. Modify the install-config.yaml file by specifying Wavelength Zones names in the platform.aws.zones property of the edge compute pool.

   # ...
   platform:
     aws:
       region: <region_name> 1
   compute:
   - name: edge
     platform:
       aws:
         zones: 2
         - <wavelength_zone_name>
   #...

   1

   The AWS Region name.

   2

   The list of Wavelength Zones names that you use must exist in the same AWS Region specified in the platform.aws.region field.

   Example of a configuration to install a cluster in the us-west-2 AWS Region that extends edge nodes to Wavelength Zones in Los Angeles and Las Vegas locations

   apiVersion: v1
   baseDomain: example.com
   metadata:
     name: cluster-name
   platform:
     aws:
       region: us-west-2
   compute:
   - name: edge
     platform:
       aws:
         zones:
         - us-west-2-wl1-lax-wlz-1
         - us-west-2-wl1-las-wlz-1
   pullSecret: '{"auths": ...}'
   sshKey: 'ssh-ed25519 AAAA...'
   #...

2. Deploy your cluster.

Procedure

1. Create a JSON file that contains the parameter values that the CloudFormation template requires:

   [
     {
       "ParameterKey": "VpcCidr", 1
       "ParameterValue": "10.0.0.0/16" 2
     },
     {
       "ParameterKey": "AvailabilityZoneCount", 3
       "ParameterValue": "3" 4
     },
     {
       "ParameterKey": "SubnetBits", 5
       "ParameterValue": "12" 6
     }
   ]

   1

   The CIDR block for the VPC.

   2

   Specify a CIDR block in the format x.x.x.x/16-24.

   3

   The number of availability zones to deploy the VPC in.

   4

   Specify an integer between 1 and 3.

   5

   The size of each subnet in each availability zone.

   6

   Specify an integer between 5 and 13, where 5 is /27 and 13 is /19.

2. Go to the section of the documentation named "CloudFormation template for the VPC", and then copy the syntax from the provided template. Save the copied template syntax as a YAML file on your local system. This template describes the VPC that your cluster requires.

3. Launch the CloudFormation template to create a stack of AWS resources that represent the VPC by running the following command:

   Important

   You must enter the command on a single line.

   $ aws cloudformation create-stack --stack-name <name> \1
        --template-body file://<template>.yaml \2
        --parameters file://<parameters>.json  3

   1

   <name> is the name for the CloudFormation stack, such as cluster-vpc. You need the name of this stack if you remove the cluster.

   2

   <template> is the relative path to and name of the CloudFormation template YAML file that you saved.

   3

   <parameters> is the relative path and the name of the CloudFormation parameters JSON file.

   Example output

   arn:aws:cloudformation:us-east-1:123456789012:stack/cluster-vpc/dbedae40-2fd3-11eb-820e-12a48460849f

4. Confirm that the template components exist by running the following command:

   $ aws cloudformation describe-stacks --stack-name <name>

   After the StackStatus displays CREATE_COMPLETE, the output displays values for the following parameters. You must provide these parameter values to the other CloudFormation templates that you run to create your cluster.

   VpcId	The ID of your VPC.
   PublicSubnetIds	The IDs of the new public subnets.
   PrivateSubnetIds	The IDs of the new private subnets.
   PublicRouteTableId	The ID of the new public route table ID.

Procedure

1. Go to the next section of the documentation named "CloudFormation template for the VPC Carrier Gateway", and then copy the syntax from the CloudFormation template for VPC Carrier Gateway template. Save the copied template syntax as a YAML file on your local system. This template describes the VPC that your cluster requires.

2. Run the following command to deploy the CloudFormation template, which creates a stack of AWS resources that represent the VPC:

   $ aws cloudformation create-stack --stack-name <stack_name> \1
     --region ${CLUSTER_REGION} \
     --template-body file://<template>.yaml \2
     --parameters \//
       ParameterKey=VpcId,ParameterValue="${VpcId}" \3
       ParameterKey=ClusterName,ParameterValue="${ClusterName}" 4

   1

   <stack_name> is the name for the CloudFormation stack, such as clusterName-vpc-carrier-gw. You need the name of this stack if you remove the cluster.

   2

   <template> is the relative path and the name of the CloudFormation template YAML file that you saved.

   3

   <VpcId> is the VPC ID extracted from the CloudFormation stack output created in the section named "Creating a VPC in AWS".

   4

   <ClusterName> is a custom value that prefixes to resources that the CloudFormation stack creates. You can use the same name that is defined in the metadata.name section of the install-config.yaml configuration file.

   Example output

   arn:aws:cloudformation:us-east-1:123456789012:stack/<stack_name>/dbedae40-2fd3-11eb-820e-12a48460849f

Procedure

1. Go to the section of the documentation named "CloudFormation template for the VPC subnet", and copy the syntax from the template. Save the copied template syntax as a YAML file on your local system. This template describes the VPC that your cluster requires.

2. Run the following command to deploy the CloudFormation template, which creates a stack of AWS resources that represent the VPC:

   $ aws cloudformation create-stack --stack-name <stack_name> \ 1
     --region ${CLUSTER_REGION} \
     --template-body file://<template>.yaml \ 2
     --parameters \
       ParameterKey=VpcId,ParameterValue="${VPC_ID}" \ 3
       ParameterKey=ClusterName,ParameterValue="${CLUSTER_NAME}" \ 4
       ParameterKey=ZoneName,ParameterValue="${ZONE_NAME}" \ 5
       ParameterKey=PublicRouteTableId,ParameterValue="${ROUTE_TABLE_PUB}" \ 6
       ParameterKey=PublicSubnetCidr,ParameterValue="${SUBNET_CIDR_PUB}" \ 7
       ParameterKey=PrivateRouteTableId,ParameterValue="${ROUTE_TABLE_PVT}" \ 8
       ParameterKey=PrivateSubnetCidr,ParameterValue="${SUBNET_CIDR_PVT}" 9

   1

   <stack_name> is the name for the CloudFormation stack, such as cluster-wl-<wavelength_zone_shortname>. You need the name of this stack if you remove the cluster.

   2

   <template> is the relative path and the name of the CloudFormation template YAML file that you saved.

   3

   ${VPC_ID} is the VPC ID, which is the value VpcID in the output of the CloudFormation template for the VPC.

   4

   ${ZONE_NAME} is the value of Wavelength Zones name to create the subnets.

   5

   ${CLUSTER_NAME} is the value of ClusterName to be used as a prefix of the new AWS resource names.

   6

   ${ROUTE_TABLE_PUB} is the PublicRouteTableId extracted from the output of the VPC’s carrier gateway CloudFormation stack.

   7

   ${SUBNET_CIDR_PUB} is a valid CIDR block that is used to create the public subnet. This block must be part of the VPC CIDR block VpcCidr.

   8

   ${ROUTE_TABLE_PVT} is the PrivateRouteTableId extracted from the output of the VPC’s CloudFormation stack.

   9

   ${SUBNET_CIDR_PVT} is a valid CIDR block that is used to create the private subnet. This block must be part of the VPC CIDR block VpcCidr.

Procedure

1. Export the kubeadmin credentials:

   $ export KUBECONFIG=<installation_directory>/auth/kubeconfig 1

   1

   For <installation_directory>, specify the path to the directory that you stored the installation files in.

2. Verify you can run oc commands successfully using the exported configuration:

   $ oc whoami

   Example output

   system:admin

Procedure

1. Obtain the password for the kubeadmin user from the kubeadmin-password file on the installation host:

   $ cat <installation_directory>/auth/kubeadmin-password

   Note

   Alternatively, you can obtain the kubeadmin password from the <installation_directory>/.openshift_install.log log file on the installation host.

2. List the OpenShift Container Platform web console route:

   $ oc get routes -n openshift-console | grep 'console-openshift'

   Note

   Alternatively, you can obtain the OpenShift Container Platform route from the <installation_directory>/.openshift_install.log log file on the installation host.

   Example output

   console     console-openshift-console.apps.<cluster_name>.<base_domain>            console     https   reencrypt/Redirect   None

3. Navigate to the route detailed in the output of the preceding command in a web browser and log in as the kubeadmin user.

1. To check the machine sets created from the subnet you added to the install-config.yaml file, run the following command:

   $ oc get machineset -n openshift-machine-api

   Example output

   NAME                                         DESIRED   CURRENT   READY   AVAILABLE   AGE
   cluster-7xw5g-edge-us-east-1-wl1-nyc-wlz-1   1         1         1       1           3h4m
   cluster-7xw5g-worker-us-east-1a              1         1         1       1           3h4m
   cluster-7xw5g-worker-us-east-1b              1         1         1       1           3h4m
   cluster-7xw5g-worker-us-east-1c              1         1         1       1           3h4m

2. To check the machines that were created from the machine sets, run the following command:

   $ oc get machines -n openshift-machine-api

   Example output

   NAME                                        PHASE     TYPE          REGION      ZONE               AGE
   cluster-7xw5g-edge-us-east-1-wl1-nyc-wlz-1-wbclh  Running   c5d.2xlarge   us-east-1   us-east-1-wl1-nyc-wlz-1  3h
   cluster-7xw5g-master-0                            Running   m6i.xlarge    us-east-1   us-east-1a               3h4m
   cluster-7xw5g-master-1                            Running   m6i.xlarge    us-east-1   us-east-1b               3h4m
   cluster-7xw5g-master-2                            Running   m6i.xlarge    us-east-1   us-east-1c               3h4m
   cluster-7xw5g-worker-us-east-1a-rtp45             Running   m6i.xlarge    us-east-1   us-east-1a               3h
   cluster-7xw5g-worker-us-east-1b-glm7c             Running   m6i.xlarge    us-east-1   us-east-1b               3h
   cluster-7xw5g-worker-us-east-1c-qfvz4             Running   m6i.xlarge    us-east-1   us-east-1c               3h

3. To check nodes with edge roles, run the following command:

   $ oc get nodes -l node-role.kubernetes.io/edge

   Example output

   NAME                           STATUS   ROLES         AGE    VERSION
   ip-10-0-207-188.ec2.internal   Ready    edge,worker   172m   v1.25.2+d2e245f