Chapter 10. Deleting a ROSA cluster
This document provides steps to delete a Red Hat OpenShift Service on AWS (ROSA) cluster that uses the AWS Security Token Service (STS). After deleting your cluster, you can also delete the AWS Identity and Access Management (IAM) resources that are used by the cluster.
10.1. Prerequisites
If Red Hat OpenShift Service on AWS created a VPC, you must remove the following items from your cluster before you can successfully delete your cluster:
- Network configurations, such as VPN configurations and VPC peering connections
- Any additional services that were added to the VPC
If these configurations and services remain, the cluster does not delete properly.
10.2. Deleting a ROSA cluster and the cluster-specific IAM resources
You can delete a Red Hat OpenShift Service on AWS (ROSA) with AWS Security Token Service (STS) cluster by using the ROSA CLI (rosa
) or Red Hat OpenShift Cluster Manager.
After deleting the cluster, you can clean up the cluster-specific Identity and Access Management (IAM) resources in your AWS account by using the ROSA CLI (rosa
). The cluster-specific resources include the Operator roles and the OpenID Connect (OIDC) provider.
The cluster deletion must complete before you remove the IAM resources, because the resources are used in the cluster deletion and clean-up processes.
If add-ons are installed, the cluster deletion takes longer because add-ons are uninstalled before the cluster is deleted. The amount of time depends on the number and size of the add-ons.
If the cluster that created the VPC during the installation is deleted, the associated installation program-created VPC will also be deleted, resulting in the failure of all the clusters that are using the same VPC. Additionally, any resources created with the same tagSet
key-value pair of the resources created by the installation program and labeled with a value of owned
will also be deleted.
Prerequisites
- You have installed a ROSA cluster.
-
You have installed and configured the latest ROSA CLI (
rosa
) on your installation host.
Procedure
Obtain the cluster ID, the Amazon Resource Names (ARNs) for the cluster-specific Operator roles and the endpoint URL for the OIDC provider:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa describe cluster --cluster=<cluster_name>
$ rosa describe cluster --cluster=<cluster_name>
1 - 1
- Replace
<cluster_name>
with the name of your cluster.
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Name: mycluster ID: 1s3v4x39lhs8sm49m90mi0822o34544a ... Operator IAM Roles: - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-credential-operator-cloud-crede - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-image-registry-installer-cloud-creden - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cluster-csi-drivers-ebs-cloud-credent - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-network-config-controller-cloud State: ready Private: No Created: May 13 2022 11:26:15 UTC Details Page: https://console.redhat.com/openshift/details/s/296kyEFwzoy1CREQicFRdZybrc0 OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<oidc_config_id>
Name: mycluster ID: 1s3v4x39lhs8sm49m90mi0822o34544a
1 ... Operator IAM Roles:
2 - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-credential-operator-cloud-crede - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-image-registry-installer-cloud-creden - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-ingress-operator-cloud-credentials - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cluster-csi-drivers-ebs-cloud-credent - arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-cloud-network-config-controller-cloud State: ready Private: No Created: May 13 2022 11:26:15 UTC Details Page: https://console.redhat.com/openshift/details/s/296kyEFwzoy1CREQicFRdZybrc0 OIDC Endpoint URL: https://oidc.op1.openshiftapps.com/<oidc_config_id>
3 - 1
- Lists the cluster ID.
- 2
- Specifies the ARNs for the cluster-specific Operator roles. For example, in the sample output the ARN for the role required by the Machine Config Operator is
arn:aws:iam::<aws_account_id>:role/mycluster-x4q9-openshift-machine-api-aws-cloud-credentials
. - 3
- Displays the endpoint URL for the cluster-specific OIDC provider.
ImportantYou require the cluster ID to delete the cluster-specific STS resources using the ROSA CLI (
rosa
) after the cluster is deleted.Delete the cluster:
To delete the cluster by using Red Hat OpenShift Cluster Manager:
- Navigate to OpenShift Cluster Manager.
-
Click the Options menu
next to your cluster and select Delete cluster.
- Type the name of your cluster at the prompt and click Delete.
To delete the cluster using the ROSA CLI (
rosa
):Enter the following command to delete the cluster and watch the logs, replacing
<cluster_name>
with the name or ID of your cluster:Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete cluster --cluster=<cluster_name> --watch
$ rosa delete cluster --cluster=<cluster_name> --watch
ImportantYou must wait for the cluster deletion to complete before you remove the Operator roles and the OIDC provider. The cluster-specific Operator roles are required to clean-up the resources created by the OpenShift Operators. The Operators use the OIDC provider to authenticate.
Delete the OIDC provider that the cluster Operators use to authenticate:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete oidc-provider -c <cluster_id> --mode auto
$ rosa delete oidc-provider -c <cluster_id> --mode auto
1 - 1
- Replace
<cluster_id>
with the ID of the cluster.
NoteYou can use the
-y
option to automatically answer yes to the prompts.Optional. Delete the cluster-specific Operator IAM roles:
ImportantThe account-wide IAM roles can be used by other ROSA clusters in the same AWS account. Only remove the roles if they are not required by other clusters.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete operator-roles -c <cluster_id> --mode auto
$ rosa delete operator-roles -c <cluster_id> --mode auto
1 - 1
- Replace
<cluster_id>
with the ID of the cluster.
Troubleshooting
- If the cluster cannot be deleted because of missing IAM roles, see Additional Repairing a cluster that cannot be deleted.
If the cluster cannot be deleted for other reasons:
- Check that there are no Add-ons for your cluster pending in the Hybrid Cloud Console.
- Check that all AWS resources and dependencies have been deleted in the Amazon Web Console.
Additional resources
- For steps to delete the account-wide IAM roles and policies, see Deleting the account-wide IAM roles and policies.
- For steps to delete the OpenShift Cluster Manager and user IAM roles, see Unlinking and deleting the OpenShift Cluster Manager and user IAM roles.
10.3. Deleting the account-wide IAM resources
After you have deleted all Red Hat OpenShift Service on AWS (ROSA) with hosted control planes (HCP) clusters that depend on the account-wide AWS Identity and Access Management (IAM) resources, you can delete the account-wide resources.
If you no longer need to install a ROSA with HCP cluster by using Red Hat OpenShift Cluster Manager, you can also delete the OpenShift Cluster Manager and user IAM roles.
The account-wide IAM roles and policies might be used by other ROSA with HCP clusters in the same AWS account. Only remove the resources if they are not required by other clusters.
The OpenShift Cluster Manager and user IAM roles are required if you want to install, manage, and delete other Red Hat OpenShift Service on AWS clusters in the same AWS account by using OpenShift Cluster Manager. Only remove the roles if you no longer need to install Red Hat OpenShift Service on AWS clusters in your account by using OpenShift Cluster Manager. For more information about repairing your cluster if these roles are removed before deletion, see "Repairing a cluster that cannot be deleted" in Troubleshooting cluster deployments.
10.3.1. Deleting the account-wide IAM roles and policies
This section provides steps to delete the account-wide IAM roles and policies that you created for ROSA with STS ROSA with HCP deployments, along with the account-wide Operator policies. You can delete the account-wide AWS Identity and Access Management (IAM) roles and policies only after deleting all of the Red Hat OpenShift Service on AWS (ROSA) with AWS Security Token Services (STS) ROSA with HCP clusters that depend on them.
The account-wide IAM roles and policies might be used by other ROSA clusters Red Hat OpenShift Service on AWS in the same AWS account. Only remove the roles if they are not required by other clusters.
Prerequisites
- You have account-wide IAM roles that you want to delete.
-
You have installed and configured the latest ROSA CLI (
rosa
) on your installation host.
Procedure
Delete the account-wide roles:
List the account-wide roles in your AWS account by using the ROSA CLI (
rosa
):Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa list account-roles
$ rosa list account-roles
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION ManagedOpenShift-ControlPlane-Role Control plane arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role 4.10 ManagedOpenShift-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role 4.10 ManagedOpenShift-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role 4.10 ManagedOpenShift-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role 4.10
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION ManagedOpenShift-ControlPlane-Role Control plane arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-ControlPlane-Role 4.10 ManagedOpenShift-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Installer-Role 4.10 ManagedOpenShift-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Support-Role 4.10 ManagedOpenShift-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-Worker-Role 4.10
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ManagedOpenShift-HCP-ROSA-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Installer-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Support-Role 4.18 Yes ManagedOpenShift-HCP-ROSA-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Worker-Role 4.18 Yes
I: Fetching account roles
ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed
ManagedOpenShift-HCP-ROSA-Installer-Role Installer arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Installer-Role 4.18 Yes
ManagedOpenShift-HCP-ROSA-Support-Role Support arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Support-Role 4.18 Yes
ManagedOpenShift-HCP-ROSA-Worker-Role Worker arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-HCP-ROSA-Worker-Role 4.18 Yes
Delete the account-wide roles:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete account-roles --prefix <prefix> --mode auto
$ rosa delete account-roles --prefix <prefix> --mode auto
1 - 1
- You must include the
--<prefix>
argument. Replace<prefix>
with the prefix of the account-wide roles to delete. If you did not specify a custom prefix when you created the account-wide roles, specify the default prefix,ManagedOpenShift
.
ImportantThe account-wide IAM roles might be used by other ROSA clusters in the same AWS account. Only remove the roles if they are not required by other clusters.
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow W: There are no classic account roles to be deleted I: Deleting hosted CP account roles ? Delete the account role 'delete-rosa-HCP-ROSA-Installer-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Installer-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Support-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Support-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Worker-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Worker-Role' I: Successfully deleted the hosted CP account roles
W: There are no classic account roles to be deleted I: Deleting hosted CP account roles ? Delete the account role 'delete-rosa-HCP-ROSA-Installer-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Installer-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Support-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Support-Role' ? Delete the account role 'delete-rosa-HCP-ROSA-Worker-Role'? Yes I: Deleting account role 'delete-rosa-HCP-ROSA-Worker-Role' I: Successfully deleted the hosted CP account roles
- Delete the account-wide in-line and Operator policies:
Under the Policies page in the AWS IAM Console, filter the list of policies by the prefix that you specified when you created the account-wide roles and policies.
NoteIf you did not specify a custom prefix when you created the account-wide roles, search for the default prefix,
ManagedOpenShift
.Delete the account-wide in-line policies and Operator policies by using the AWS IAM Console. For more information about deleting IAM policies by using the AWS IAM Console, see Deleting IAM policies in the AWS documentation.
ImportantThe account-wide in-line and Operator IAM policies might be used by other ROSA clusters ROSA with HCP in the same AWS account. Only remove the roles if they are not required by other clusters.
10.3.2. Unlinking and deleting the OpenShift Cluster Manager and user IAM roles
When you install a ROSA with HCP cluster by using Red Hat OpenShift Cluster Manager, you also create OpenShift Cluster Manager and user Identity and Access Management (IAM) roles that link to your Red Hat organization. After deleting your cluster, you can unlink and delete the roles by using the ROSA CLI (rosa
).
The OpenShift Cluster Manager and user IAM roles are required if you want to use OpenShift Cluster Manager to install and manage other ROSA with HCP in the same AWS account. Only remove the roles if you no longer need to use the OpenShift Cluster Manager to install ROSA with HCP clusters.
Prerequisites
- You created OpenShift Cluster Manager and user IAM roles and linked them to your Red Hat organization.
-
You have installed and configured the latest ROSA CLI (
rosa
) on your installation host. - You have organization administrator privileges in your Red Hat organization.
Procedure
Unlink the OpenShift Cluster Manager IAM role from your Red Hat organization and delete the role:
List the OpenShift Cluster Manager IAM roles in your AWS account:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa list ocm-roles
$ rosa list ocm-roles
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Fetching ocm roles ROLE NAME ROLE ARN LINKED ADMIN AWS Managed ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> Yes Yes Yes
I: Fetching ocm roles ROLE NAME ROLE ARN LINKED ADMIN AWS Managed ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> Yes Yes Yes
If your OpenShift Cluster Manager IAM role is listed as linked in the output of the preceding command, unlink the role from your Red Hat organization by running the following command:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa unlink ocm-role --role-arn <arn>
$ rosa unlink ocm-role --role-arn <arn>
1 - 1
- Replace
<arn>
with the Amazon Resource Name (ARN) for your OpenShift Cluster Manager IAM role. The ARN is specified in the output of the preceding command. In the preceding example, the ARN is in the formatarn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>
.
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Unlinking OCM role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role from organization '<red_hat_organization_id>'? Yes I: Successfully unlinked role-arn 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' from organization account '<red_hat_organization_id>'
I: Unlinking OCM role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' role from organization '<red_hat_organization_id>'? Yes I: Successfully unlinked role-arn 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' from organization account '<red_hat_organization_id>'
Delete the OpenShift Cluster Manager IAM role and policies:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete ocm-role --role-arn <arn>
$ rosa delete ocm-role --role-arn <arn>
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Deleting OCM role ? OCM Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> ? Delete 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' ocm role? Yes ? OCM role deletion mode: auto I: Successfully deleted the OCM role
I: Deleting OCM role ? OCM Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id> ? Delete 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-OCM-Role-<red_hat_organization_external_id>' ocm role? Yes ? OCM role deletion mode: auto
1 I: Successfully deleted the OCM role
- 1
- Specifies the deletion mode. You can use
auto
mode to automatically delete the OpenShift Cluster Manager IAM role and policies. Inmanual
mode, the ROSA CLI generates theaws
commands needed to delete the role and policies.manual
mode enables you to review the details before running theaws
commands manually.
Unlink the user IAM role from your Red Hat organization and delete the role:
List the user IAM roles in your AWS account:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa list user-roles
$ rosa list user-roles
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Fetching user roles ROLE NAME ROLE ARN LINKED ManagedOpenShift-User-<ocm_user_name>-Role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role Yes
I: Fetching user roles ROLE NAME ROLE ARN LINKED ManagedOpenShift-User-<ocm_user_name>-Role arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role Yes
If your user IAM role is listed as linked in the output of the preceding command, unlink the role from your Red Hat organization:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa unlink user-role --role-arn <arn>
$ rosa unlink user-role --role-arn <arn>
1 - 1
- Replace
<arn>
with the Amazon Resource Name (ARN) for your user IAM role. The ARN is specified in the output of the preceding command. In the preceding example, the ARN is in the formatarn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role
.
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Unlinking user role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the current account '<ocm_user_account_id>'? Yes I: Successfully unlinked role ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' from account '<ocm_user_account_id>'
I: Unlinking user role ? Unlink the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the current account '<ocm_user_account_id>'? Yes I: Successfully unlinked role ARN 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' from account '<ocm_user_account_id>'
Delete the user IAM role:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow rosa delete user-role --role-arn <arn>
$ rosa delete user-role --role-arn <arn>
Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow I: Deleting user role ? User Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role ? Delete the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the AWS account? Yes ? User role deletion mode: auto I: Successfully deleted the user role
I: Deleting user role ? User Role ARN: arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role ? Delete the 'arn:aws:iam::<aws_account_id>:role/ManagedOpenShift-User-<ocm_user_name>-Role' role from the AWS account? Yes ? User role deletion mode: auto
1 I: Successfully deleted the user role
- 1
- Specifies the deletion mode. You can use
auto
mode to automatically delete the user IAM role. Inmanual
mode, the ROSA CLI generates theaws
command needed to delete the role.manual
mode enables you to review the details before running theaws
command manually.
10.4. Additional resources
- For information about the cluster delete protection feature, see Edit objects.
- For information about the AWS IAM resources for ROSA clusters that use STS, see About IAM resources.
- For information on cluster errors that are due to missing IAM roles, see Repairing a cluster that cannot be deleted.