Chapter 11. Deploying ROSA without AWS STS

Procedure

1. If you, as the customer, are utilizing AWS Organizations, then you must use an AWS account within your organization or create a new one.
2. To ensure that Red Hat can perform necessary actions, you must either create a service control policy (SCP) or ensure that none is applied to the AWS account.
3. Attach the SCP to the AWS account.
4. Follow the ROSA procedures for setting up the environment.

1. Perform the AWS prerequisites. To deploy a ROSA cluster, your AWS account must meet the prerequisite requirements.
2. Review the required AWS service quotas. To prepare for your cluster deployment, review the AWS service quotas that are required to run a ROSA cluster.
3. Configure your AWS account. Before you create a ROSA cluster, you must enable ROSA in your AWS account, install and configure the AWS CLI (aws) tool, and verify the AWS CLI tool configuration.
4. Install the ROSA and OpenShift CLI tools and verify the AWS servce quotas. Install and configure the ROSA CLI (rosa) and the OpenShift CLI (oc). You can verify if the required AWS resource quotas are available by using the ROSA CLI.
5. Create a ROSA cluster or Create a ROSA cluster using AWS PrivateLink. Use the ROSA CLI (rosa) to create a cluster. You can optionally create a ROSA cluster with AWS PrivateLink.
6. Access a cluster. You can configure an identity provider and grant cluster administrator privileges to the identity provider users as required. You can also access a newly deployed cluster quickly by configuring a cluster-admin user.
7. Revoke access to a ROSA cluster for a user. You can revoke access to a ROSA cluster from a user by using the ROSA CLI or the web console.
8. Delete a ROSA cluster. You can delete a ROSA cluster by using the ROSA CLI (rosa).

Procedure

1. Log in to the Amazon Web Services (AWS) account that you want to use.

   A dedicated AWS account is recommended to run production clusters. If you are using AWS Organizations, you can use an AWS account within your organization or create a new one.

   If you are using AWS Organizations and you need to have a service control policy (SCP) applied to the AWS account you plan to use, see AWS Prerequisites for details on the minimum required SCP.

   As part of the cluster creation process, rosa establishes an osdCcsAdmin IAM user. This user uses the IAM credentials you provide when configuring the AWS CLI.

   Note

   This user has Programmatic access enabled and the AdministratorAccess policy attached to it.

2. Enable the ROSA service in the AWS Console.

   1. Sign in to your AWS account.
   2. To enable ROSA, go to the ROSA service and select Enable OpenShift.

3. Install and configure the AWS CLI.

   1. Follow the AWS command-line interface documentation to install and configure the AWS CLI for your operating system.

      Specify the correct aws_access_key_id and aws_secret_access_key in the .aws/credentials file. See AWS Configuration basics in the AWS documentation.

   2. Set a default AWS region.

      Note

      It is recommended to set the default AWS region by using the environment variable.

      The ROSA service evaluates regions in the following priority order:

      1. The region specified when running the rosa command with the --region flag.
      2. The region set in the AWS_DEFAULT_REGION environment variable. See Environment variables to configure the AWS CLI in the AWS documentation.
      3. The default region set in your AWS configuration file. See Quick configuration with aws configure in the AWS documentation.

   3. Optional: Configure your AWS CLI settings and credentials by using an AWS named profile. rosa evaluates AWS named profiles in the following priority order:

      1. The profile specified when running the rosa command with the --profile flag.
      2. The profile set in the AWS_PROFILE environment variable. See Named profiles in the AWS documentation.

   4. Verify the AWS CLI is installed and configured correctly by running the following command to query the AWS API:

      $ aws sts get-caller-identity --output text

      Example output

      <aws_account_id>    arn:aws:iam::<aws_account_id>:user/<username>  <aws_user_id>

      After completing these steps, install ROSA.

Procedure

1. Install rosa, the Red Hat OpenShift Service on AWS command-line interface (CLI).

   1. Download the latest release of the ROSA CLI for your operating system.
   2. Optional: Rename the executable file you downloaded to rosa. This documentation uses rosa to refer to the executable file.

   3. Optional: Add rosa to your path.

      Example

      $ mv rosa /usr/local/bin/rosa

   4. Enter the following command to verify your installation:

      $ rosa

      Example output

      Command line tool for Red Hat OpenShift Service on AWS.
      For further documentation visit https://access.redhat.com/documentation/en-us/red_hat_openshift_service_on_aws
      
      Usage:
        rosa [command]
      
      Available Commands:
        completion  Generates completion scripts
        create      Create a resource from stdin
        delete      Delete a specific resource
        describe    Show details of a specific resource
        download    Download necessary tools for using your cluster
        edit        Edit a specific resource
        grant       Grant role to a specific resource
        help        Help about any command
        init        Applies templates to support Red Hat OpenShift Service on AWS
        install     Installs a resource into a cluster
        link        Link a ocm/user role from stdin
        list        List all resources of a specific type
        login       Log in to your Red Hat account
        logout      Log out
        logs        Show installation or uninstallation logs for a cluster
        revoke      Revoke role from a specific resource
        uninstall   Uninstalls a resource from a cluster
        unlink      UnLink a ocm/user role from stdin
        upgrade     Upgrade a resource
        verify      Verify resources are configured correctly for cluster install
        version     Prints the version of the tool
        whoami      Displays user account information
      
      Flags:
            --color string   Surround certain characters with escape sequences to display them in color on the terminal. Allowed options are [auto never always] (default "auto")
            --debug          Enable debug mode.
        -h, --help           help for rosa
      
      Use "rosa [command] --help" for more information about a command.

   5. Optional: Generate the command completion scripts for the ROSA CLI. The following example generates the Bash completion scripts for a Linux machine:

      $ rosa completion bash | sudo tee /etc/bash_completion.d/rosa

   6. Optional: Enable command completion for the ROSA CLI from your existing terminal. The following example enables Bash completion for rosa in an existing terminal on a Linux machine:

      $ source /etc/bash_completion.d/rosa

2. Log in to your Red Hat account with rosa.

   1. Enter the following command.

      $ rosa login

   2. Replace <my_offline_access_token> with your token.

      Example output

      To login to your Red Hat account, get an offline access token at https://console.redhat.com/openshift/token/rosa
      ? Copy the token and paste it here: <my-offline-access-token>

      Example output continued

      I: Logged in as 'rh-rosa-user' on 'https://api.openshift.com'

3. Enter the following command to verify that your AWS account has the necessary permissions.

   $ rosa verify permissions

   Example output

   I: Validating SCP policies...
   I: AWS SCP policies ok

   Note

   This command verifies permissions only for ROSA clusters that do not use the AWS Security Token Service (STS).

4. Verify that your AWS account has the necessary quota to deploy an Red Hat OpenShift Service on AWS cluster.

   $ rosa verify quota --region=us-west-2

   Example output

   I: Validating AWS quota...
   I: AWS quota ok

   Note

   Sometimes your AWS quota varies by region. If you receive any errors, try a different region.

   If you need to increase your quota, go to your AWS console, and request a quota increase for the service that failed.

   After both the permissions and quota checks pass, proceed to the next step.

5. Prepare your AWS account for cluster deployment:

   1. Run the following command to verify your Red Hat and AWS credentials are setup correctly. Check that your AWS Account ID, Default Region and ARN match what you expect. You can safely ignore the rows beginning with OCM for now.

      $ rosa whoami

      Example output

      AWS Account ID:               000000000000
      AWS Default Region:           us-east-2
      AWS ARN:                      arn:aws:iam::000000000000:user/hello
      OCM API:                      https://api.openshift.com
      OCM Account ID:               1DzGIdIhqEWyt8UUXQhSoWaaaaa
      OCM Account Name:             Your Name
      OCM Account Username:         you@domain.com
      OCM Account Email:            you@domain.com
      OCM Organization ID:          1HopHfA2hcmhup5gCr2uH5aaaaa
      OCM Organization Name:        Red Hat
      OCM Organization External ID: 0000000

   2. Initialize your AWS account. This step runs a CloudFormation template that prepares your AWS account for cluster deployment and management. This step typically takes 1-2 minutes to complete.

      $ rosa init

      Example output

      I: Logged in as 'rh-rosa-user' on 'https://api.openshift.com'
      I: Validating AWS credentials...
      I: AWS credentials are valid!
      I: Validating SCP policies...
      I: AWS SCP policies ok
      I: Validating AWS quota...
      I: AWS quota ok
      I: Ensuring cluster administrator user 'osdCcsAdmin'...
      I: Admin user 'osdCcsAdmin' created successfully!
      I: Verifying whether OpenShift command-line tool is available...
      E: OpenShift command-line tool is not installed.
      Run 'rosa download oc' to download the latest version, then add it to your PATH.

6. Install the OpenShift CLI (oc) from the ROSA CLI.

   1. Enter this command to download the latest version of the oc CLI:

      $ rosa download oc

   2. After downloading the oc CLI, unzip it and add it to your path.

   3. Enter this command to verify that the oc CLI is installed correctly:

      $ rosa verify oc

Procedure

1. You can create a cluster using the default settings or by specifying custom settings using the interactive mode. To view other options when creating a cluster, enter the rosa create cluster --help command.

   Creating a cluster can take up to 40 minutes.

   Note

   Multiple availability zones (AZ) are recommended for production workloads. The default is a single availability zone. Use --help for an example of how to set this option manually or use interactive mode to be prompted for this setting.

   • To create your cluster with the default cluster settings:

     $ rosa create cluster --cluster-name=<cluster_name>

     Example output

     I: Creating cluster with identifier '1de87g7c30g75qechgh7l5b2bha6r04e' and name 'rh-rosa-test-cluster1'
     I: To view list of clusters and their status, run `rosa list clusters`
     I: Cluster 'rh-rosa-test-cluster1' has been created.
     I: Once the cluster is 'Ready' you will need to add an Identity Provider and define the list of cluster administrators. See `rosa create idp --help` and `rosa create user --help` for more information.
     I: To determine when your cluster is Ready, run `rosa describe cluster rh-rosa-test-cluster1`.

   • To create a cluster using interactive prompts:

     $ rosa create cluster --interactive

   • To configure your networking IP ranges, you can use the following default ranges. For more information when using manual mode, use the rosa create cluster --help | grep cidr command. In interactive mode, you are prompted for the settings.

     • Node CIDR: 10.0.0.0/16
     • Service CIDR: 172.30.0.0/16
     • Pod CIDR: 10.128.0.0/14

2. Enter the following command to check the status of your cluster. During cluster creation, the State field from the output will transition from pending to installing, and finally to ready.

   $ rosa describe cluster --cluster=<cluster_name>

   Example output

   Name: rh-rosa-test-cluster1
   OpenShift Version: 4.6.8
   DNS: *.example.com
   ID: uniqueidnumber
   External ID: uniqueexternalidnumber
   AWS Account: 123456789101
   API URL: https://api.rh-rosa-test-cluster1.example.org:6443
   Console URL: https://console-openshift-console.apps.rh-rosa-test-cluster1.example.or
   Nodes: Master: 3, Infra: 2, Compute: 2
   Region: us-west-2
   Multi-AZ: false
   State: ready
   Channel Group: stable
   Private: No
   Created: Jan 15 2021 16:30:55 UTC
   Details Page: https://console.redhat.com/examplename/details/idnumber

   Note

   If installation fails or the State field does not change to ready after 40 minutes, check the installation troubleshooting documentation for more details.

3. Track the progress of the cluster creation by watching the OpenShift installer logs:

   $ rosa logs install --cluster=<cluster_name> --watch

Procedure

1. Enter the following command to revoke the dedicated-admin access of a user:

   $ rosa revoke user dedicated-admin --user=<idp_user_name> --cluster=<cluster_name>

2. Enter the following command to verify that your user no longer has dedicated-admin access. The output does not list the revoked user.

   $ oc get groups dedicated-admins

Procedure

1. Enter the following command to revoke the cluster-admin access of a user:

   $ rosa revoke user cluster-admins --user=myusername --cluster=mycluster

2. Enter the following command to verify that the user no longer has cluster-admin access. The output does not list the revoked user.

   $ oc get groups cluster-admins

Procedure

1. Obtain the cluster ID, the Amazon Resource Names (ARNs) for the cluster-specific Operator roles and the endpoint URL for the OIDC provider:

   $ rosa describe cluster --cluster=<cluster_name> 1

   1

   Replace <cluster_name> with the name of your cluster.

   Example output

   Name:                       mycluster
   ID:                         1s3v4x39lhs8sm49m90mi0822o34544a 1
   ...
   Operator IAM Roles: 2
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-credential-operator-cloud-crede
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-image-registry-installer-cloud-creden
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-ingress-operator-cloud-credentials
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cluster-csi-drivers-ebs-cloud-credent
    - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-network-config-controller-cloud
   State:                      ready
   Private:                    No
   Created:                    May 13 2022 11:26:15 UTC
   Details Page:               https://console.redhat.com/openshift/details/s/296kyEFwzoy1CREQicFRdZybrc0
   OIDC Endpoint URL:          https://oidc.op1.openshiftapps.com/<oidc_config_id> 3

   1

   Lists the cluster ID.

   2

   Specifies the ARNs for the cluster-specific Operator roles. For example, in the sample output the ARN for the role required by the Machine Config Operator is arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials.

   3

   Displays the endpoint URL for the cluster-specific OIDC provider.

   Important

   You require the cluster ID to delete the cluster-specific STS resources using the ROSA CLI (rosa) after the cluster is deleted.

2. Delete the cluster:

   • To delete the cluster by using Red Hat OpenShift Cluster Manager:

     1. Navigate to OpenShift Cluster Manager.
     2. Click the Options menu next to your cluster and select Delete cluster.
     3. Type the name of your cluster at the prompt and click Delete.

   • To delete the cluster using the ROSA CLI (rosa):

     1. Enter the following command to delete the cluster and watch the logs, replacing <cluster_name> with the name or ID of your cluster:

        $ rosa delete cluster --cluster=<cluster_name> --watch

        Important

        You must wait for the cluster deletion to complete before you remove the Operator roles and the OIDC provider. The cluster-specific Operator roles are required to clean-up the resources created by the OpenShift Operators. The Operators use the OIDC provider to authenticate.

3. Delete the OIDC provider that the cluster Operators use to authenticate:

   $ rosa delete oidc-provider -c <cluster_id> --mode auto 1

   1

   Replace <cluster_id> with the ID of the cluster.

   Note

   You can use the -y option to automatically answer yes to the prompts.

4. Optional. Delete the cluster-specific Operator IAM roles:

   Important

   The account-wide IAM roles can be used by other ROSA clusters in the same AWS account. Only remove the roles if they are not required by other clusters.

   $ rosa delete operator-roles -c <cluster_id> --mode auto 1

   1

   Replace <cluster_id> with the ID of the cluster.