Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Introduction to ROSA

Red Hat OpenShift Service on AWS 4

An overview of Red Hat OpenShift Service on AWS architecture

Red Hat OpenShift Documentation Team

Abstract

This document provides an overview of the platform and application architecture in Red Hat OpenShift Service on AWS (ROSA).

Table of Contents

Welcome to the official Red Hat OpenShift Service on AWS (ROSA) documentation, where you can learn about ROSA and start exploring its features.

Chapter 3. ROSA with HCP overview
Copy link

ROSA is a fully-managed turnkey application platform that allows you to focus on what matters most, delivering value to your customers by building and deploying applications. Red Hat and AWS SRE experts manage the underlying platform so you do not have to worry about infrastructure management. ROSA provides seamless integration with a wide range of AWS compute, database, analytics, machine learning, networking, mobile, AI and other services to further accelerate the building and delivering of differentiating experiences to your customers.

Red Hat OpenShift Service on AWS offers a reduced-cost solution to create a managed ROSA cluster with a focus on efficiency and security. You can quickly create a new cluster and deploy applications in minutes.

You subscribe to the service directly from your AWS account. After you create clusters, you can operate your clusters with the OpenShift web console, the ROSA CLI, or through Red Hat OpenShift Cluster Manager.

You receive OpenShift updates with new feature releases and a shared, common source for alignment with OpenShift Container Platform. ROSA supports the same versions of OpenShift as Red Hat OpenShift Container Platform to achieve version consistency.

Red Hat OpenShift Service on AWS
View larger image
Red Hat OpenShift Service on AWS

ROSA uses AWS Security Token Service (STS) with AWS IAM to obtain credentials to manage infrastructure in your AWS account. AWS STS is a global web service that creates temporary credentials for IAM users/roles or federated users/roles. ROSA uses this to assign short-term, limited-privilege, security credentials. These credentials are associated with IAM roles that are specific to each component that makes AWS API calls. This method aligns with the principals of least privilege and secure practices in cloud service resource management. The ROSA command-line interface (CLI) tool manages the STS credentials that are assigned for unique tasks and takes action on AWS resources as part of OpenShift functionality. For a more detailed explanation, see AWS STS and ROSA with HCP explained.

3.1. Key features of ROSA with HCP
Copy link

  • Cluster node scaling: ROSA with HCP requires a minimum of only two nodes, making it ideal for smaller projects while still being able to scale to support larger projects and enterprises. Easily add or remove compute nodes to match resource demand. Autoscaling allows you to automatically adjust the size of the cluster based on the current workload. See About autoscaling nodes on a cluster for more details.
  • Fully managed underlying control plane infrastructure: Control plane components, such as the API server and etcd database, are hosted in a Red Hat-owned AWS account.
  • Rapid provisioning time: Provisioning time is approximately 10 minutes.
  • Continued cluster operation during upgrades: Customers can upgrade the control plane and machine pools separately, ensuring the cluster remains operational during the upgrade process.
  • Native AWS service: Access and use Red Hat OpenShift on-demand with a self-service onboarding experience through the AWS management console.
  • Flexible, consumption-based pricing: Scale to your business needs and pay as you go with flexible pricing and an on-demand hourly or annual billing model.
  • Single bill for Red Hat OpenShift and AWS usage: Customers will receive a single bill from AWS for both Red Hat OpenShift and AWS consumption.
  • Fully integrated support experience: Management, maintenance, and upgrades are performed by Red Hat site reliability engineers (SREs) with joint Red Hat and Amazon support and a 99.95% service-level agreement (SLA). See the ROSA support documentation for more details.
  • AWS service integration: AWS has a robust portfolio of cloud services, such as compute, storage, networking, database, analytics, Virtualization and AI. All of these services are directly accessible through ROSA. This makes it easier to build, operate, and scale globally and on-demand through a familiar management interface.
  • Maximum availability: Deploy clusters across multiple availability zones in supported regions to maximize availability and maintain high availability for your most demanding mission-critical applications and data.
  • Optimized clusters: Choose from memory-optimized, compute-optimized, general purpose, or accelerated EC2 instance types with clusters to meet your needs.
  • Global availability: Refer to the product regional availability page to see where ROSA is available globally.

3.2. Billing and pricing
Copy link

Red Hat OpenShift Service on AWS is billed directly to your Amazon Web Services (AWS) account. ROSA pricing is consumption based, with annual commitments or three-year commitments for greater discounting. The total cost of ROSA consists of two components:

  • ROSA service fees
  • AWS infrastructure fees

Visit the Red Hat OpenShift Service on AWS Pricing page on the AWS website for more details.

3.3. Getting started with ROSA with HCP
Copy link

Use the following sections to find content to help you learn about and use ROSA with HCP.

3.3.1. Architect
Copy link

Expand
Learn about ROSA with HCPPlan ROSA with HCP deploymentAdditional resources

Architecture overview

Back up and restore ifdef::openshift-rosa-hcp[]

ROSA with HCP life cycle endif::openshift-rosa-hcp[]

ROSA with HCP architecture

Understanding process and security

ROSA with HCP service definition

Updates lifecycle

 

Getting support

  

3.3.2. Cluster Administrator
Copy link

Expand
Learn about ROSA with HCPDeploy ROSA with HCPManage ROSA with HCPAdditional resources

ROSA with HCP architecture

Installing ROSA with HCP

Logging

Getting Support

OpenShift Interactive Learning Portal

Storage

About Red Hat OpenShift Service on AWS monitoring

ROSA with HCP life cycle

ROSA responsibility matrix

Back up and restore

About IAM resources

ROSA roadmap

About availability

Upgrading

3.3.3. Developer
Copy link

Expand
Learn about application development in ROSA with HCPDeploy applicationsAdditional resources

Red Hat Developers site

Building applications overview

Getting support

Red Hat OpenShift Dev Spaces (formerly Red Hat CodeReady Workspaces)

Operators overview

ROSA roadmap

 

Images

 
 

Developer-focused CLI

 

3.3.4. Before creating your first ROSA cluster
Copy link

For additional information about ROSA installation, see a qucik introdcution to the process in Installing Red Hat OpenShift Service on AWS (ROSA) interactive walkthrough.

Chapter 4. AWS STS and ROSA with HCP explained
Copy link

Red Hat OpenShift Service on AWS uses an AWS (Amazon Web Services) Security Token Service (STS) for AWS Identity Access Management (IAM) to obtain the necessary credentials to interact with resources in your AWS account.

4.1. AWS STS credential method
Copy link

As part of ROSA with HCP, Red Hat must be granted the necessary permissions to manage infrastructure resources in your AWS account. ROSA with HCP IAM STS policies grants the cluster’s automation software limited, short-term access to resources in your AWS account.

The STS method uses predefined roles and policies to grant temporary, least-privilege permissions to IAM roles. The credentials typically expire an hour after being requested. Once expired, they are no longer recognized by AWS and no longer have account access to make API requests with them. For more information, see the AWS documentation.

AWS IAM STS roles must be created for each ROSA cluster. The ROSA command-line interface (CLI) (rosa) manages the STS roles and helps you attach the ROSA-specific, AWS managed policies to each role. The CLI provides the commands and files to create the roles, attach the AWS-managed policies, and an option to allow the CLI to automatically create the roles and attach the policies. Alternatively, the ROSA CLI can also provide you with the content to prepare the roles and attach the ROSA-specific AWS managed policies.

4.2. AWS STS security
Copy link

Security features for AWS STS include:

  • An explicit and limited set of policies that the user creates ahead of time.

    • The user can review every requested permission needed by the platform.
  • The service cannot do anything outside of those permissions.
  • There is no need to rotate or revoke credentials. Whenever the service needs to perform an action, it obtains credentials that expire in one hour or less.
  • Credential expiration reduces the risks of credentials leaking and being reused.
  • The ROSA-specific AWS managed policies are tightly scoped to only allow actions on ROSA-specific AWS resources in your account, within the limits of the AWS API.

ROSA policies grant cluster software components with least-privilege permissions with short-term security credentials to specific and segregated IAM roles. The credentials are associated with IAM roles specific to each component and cluster that makes AWS API calls. This method aligns with principles of least-privilege and secure practices in cloud service resource management.

4.3. Components of ROSA with HCP
Copy link

  • AWS infrastructure - The infrastructure required for the cluster including the Amazon EC2 instances, Amazon EBS storage, and networking components. See AWS compute types to see the supported instance types for compute nodes and provisioned AWS infrastructure for more information on cloud resource configuration.
  • AWS STS - A method for granting short-term, dynamic tokens to provide users the necessary permissions to temporarily interact with your AWS account resources.
  • OpenID Connect (OIDC) - A mechanism for cluster Operators to authenticate with AWS, assume the cluster roles through a trust policy, and obtain temporary credentials from AWS IAM STS to make the required API calls.

  • Roles and policies - The roles and policies used by ROSA with HCP can be divided into account-wide roles and policies and Operator roles and policies.

    The policies determine the allowed actions for each of the roles. See About IAM resources for more details about the individual roles and policies. See Required IAM roles and resources for more details on preparing these resources in your cluster.

    Note

    Certain policies are used by the cluster Operator roles, listed below. The Operator roles are created in a second step because they are dependent on an existing cluster name and cannot be created at the same time as the account-wide roles.

    • The Operator roles are:

      • <operator_role_prefix>-openshift-cluster-csi-drivers-ebs-cloud-credentials
      • <operator_role_prefix>-openshift-cloud-network-config-controller-cloud-credentials
      • <operator_role_prefix>-openshift-machine-api-aws-cloud-credentials
      • <operator_role_prefix>-openshift-cloud-credential-operator-cloud-credentials
      • <operator_role_prefix>-openshift-image-registry-installer-cloud-credentials
      • <operator_role_prefix>-openshift-ingress-operator-cloud-credentials
    • Trust policies are created for each account-wide role and each Operator role.

4.4. Deploying a ROSA with HCP cluster
Copy link

Deploying a ROSA with HCP cluster follows the following general steps:

  1. You create the account-wide roles.
  2. You create the Operator roles.
  3. Red Hat uses AWS IAM STS to send the required permissions to AWS that allow AWS to create and attach the corresponding AWS-managed Operator policies.
  4. You create the OIDC provider.
  5. You create the cluster.

During the cluster creation process, the ROSA CLI creates the required JSON files for you and outputs the commands you need. If desired, the ROSA CLI can also run the commands for you.

The ROSA CLI can automatically create the roles for you, or you can manually create them by using the --mode manual or --mode auto flags. For further details about deployment, see Creating a cluster with customizations.

4.5. ROSA with HCP workflow
Copy link

The user creates the required account-wide roles. During role creation, a trust policy, known as a cross-account trust policy, is created which allows a Red Hat-owned role to assume the roles. Trust policies are also created for the EC2 service, which allows workloads on EC2 instances to assume roles and obtain credentials. AWS assigns a corresponding permissions policy to each role.

After both the account-wide operator roles and policies are created, the user can create a cluster. These operator roles are assigned to the corresponding permission policies that were created earlier and a trust policy with an OIDC provider. The operator roles differ from the account-wide roles in that they ultimately represent the in-cluster pods that need access to AWS resources. Because a user cannot attach IAM roles to pods, they must create a trust policy with an OIDC provider so that the Operator, and therefore the pods, can access the roles they need.

cloud experts sts explained creation flow hcp
View larger image
cloud experts sts explained creation flow hcp

When a new role is needed, the workload currently using the Red Hat role will assume the role in the AWS account, obtain temporary credentials from AWS STS, and begin performing the actions using API calls within the user’s AWS account as permitted by the assumed role’s permissions policy. The credentials are temporary and have a maximum duration of one hour.

cloud experts sts explained highlevel
View larger image
cloud experts sts explained highlevel

Operators use the following process to obtain the requisite credentials to perform their tasks. Each Operator is assigned an Operator role, a permissions policy, and a trust policy with an OIDC provider. The Operator will assume the role by passing a JSON web token that contains the role and a token file (web_identity_token_file) to the OIDC provider, which then authenticates the signed key with a public key. The public key is created during cluster creation and stored in an S3 bucket. The Operator then confirms that the subject in the signed token file matches the role in the role trust policy which ensures that the OIDC provider can only obtain the allowed role. The OIDC provider then returns the temporary credentials to the Operator so that the Operator can make AWS API calls. For a visual representation, see the following diagram:

cloud experts sts explained oidc op roles hcp
View larger image
cloud experts sts explained oidc op roles hcp

Chapter 5. Architecture models
Copy link

Red Hat OpenShift Service on AWS (ROSA) has the following cluster topologies:

  • Hosted control plane (HCP) - The control plane is hosted in a Red Hat account and the worker nodes are deployed in the customer’s AWS account.
  • Classic - The control plane and the worker nodes are deployed in the customer’s AWS account.

5.1. Comparing ROSA with HCP and ROSA Classic
Copy link

Expand
Table 5.1. ROSA architectures comparison table
 
Hosted Control Plane (HCP)Classic

Control plane hosting

Control plane components, such as the API server etcd database, are hosted in a Red Hat-owned AWS account.

Control plane components, such as the API server etcd database, are hosted in a customer-owned AWS account.

Virtual Private Cloud (VPC)

Worker nodes communicate with the control plane over AWS PrivateLink.

Worker nodes and control plane nodes are deployed in the customer’s VPC.

Multi-zone deployment

The control plane is always deployed across multiple availability zones (AZs).

The control plane can be deployed within a single AZ or across multiple AZs.

Machine pools

Each machine pool is deployed in a single AZ (private subnet).

Machine pools can be deployed in single AZ or across multiple AZs.

Infrastructure nodes

Does not use any dedicated infrastructure nodes to host platform components, such as ingress and image registry.

Uses 2 (single-AZ) or 3 (multi-AZ) dedicated infrastructure nodes to host platform components.

OpenShift capabilities

Platform monitoring, image registry, and the ingress controller are deployed in the worker nodes.

Platform monitoring, image registry, and the ingress controller are deployed in the dedicated infrastructure nodes.

Cluster upgrades

The control plane and each machine pool can be upgraded separately.

The entire cluster must be upgraded at the same time.

Minimum EC2 footprint

2 EC2 instances are needed to create a cluster.

7 (single-AZ) or 9 (multi-AZ) EC2 instances are needed to create a cluster.

Additional resources

5.2. ROSA with HCP architecture
Copy link

In Red Hat OpenShift Service on AWS, the ROSA service hosts a highly-available, single-tenant OpenShift control plane. The hosted control plane is deployed across 3 availability zones with 2 API server instances and 3 etcd instances.

You can create a ROSA with HCP cluster with or without an internet-facing API server, with the latter considered a “private” cluster and the former considered a “public” cluster. Private API servers are only accessible from your VPC subnets. You access the hosted control plane through an AWS PrivateLink endpoint regardless of API privacy.

The worker nodes are deployed in your AWS account and run on your VPC private subnets. You can add additional private subnets from one or more availability zones to ensure high availability. Worker nodes are shared by OpenShift components and applications. OpenShift components such as the ingress controller, image registry, and monitoring are deployed on the worker nodes hosted on your VPC.

Figure 5.1. ROSA with HCP architecture

ROSA with HCP architecture
View larger image
ROSA with HCP architecture

With ROSA with HCP, you can create your clusters on public or private networks. The following images depict the architecture of both public and private networks.

Figure 5.2. ROSA with HCP deployed on a public network

ROSA with HCP deployed on a public network
View larger image
ROSA with HCP deployed on a public network

Figure 5.3. ROSA with HCP deployed on a private network

ROSA with HCP deployed on a private network
View larger image
ROSA with HCP deployed on a private network

Chapter 6. Policies and service definition
Copy link

Availability and disaster avoidance are extremely important aspects of any application platform. Although Red Hat OpenShift Service on AWS (ROSA) provides many protections against failures at several levels, customer-deployed applications must be appropriately configured for high availability. To account for outages that might occur with cloud providers, additional options are available such as deploying a cluster across multiple availability zones and maintaining multiple clusters with failover mechanisms.

6.1.1. Potential points of failure
Copy link

Red Hat OpenShift Service on AWS (ROSA) provides many features and options for protecting your workloads against downtime, but applications must be architected appropriately to take advantage of these features.

ROSA can help further protect you against many common Kubernetes issues given Red Hat site reliability engineering (SRE) support and the option to deploy machine pools into more than one availability zone, but there are several ways in which a container or infrastructure can still fail. By understanding potential points of failure, you can understand risks and appropriately architect both your applications and your clusters to be as resilient as necessary at each specific level.

Important

Worker nodes are not guaranteed longevity, and may be replaced at any time as part of the normal operation and management of OpenShift. For more details about the node lifecycle, refer to additional resources.

Note

An outage can occur at several different levels of infrastructure and cluster components.

6.1.1.1. Container or pod failure
Copy link

By design, pods are meant to exist for a short time. Appropriately scaling services so that multiple instances of your application pods are running can protect against issues with any individual pod or container. The OpenShift node scheduler can also make sure these workloads are distributed across different worker nodes to further improve resiliency.

When accounting for possible pod failures, it is also important to understand how storage is attached to your applications. Single persistent volumes attached to single pods cannot leverage the full benefits of pod scaling, whereas replicated databases, database services, or shared storage can.

To avoid disruption to your applications during planned maintenance, such as upgrades, it is important to define a Pod Disruption Budget. These are part of the Kubernetes API and can be managed with oc commands such as other object types. They allow for the specification of safety constraints on pods during operations, such as draining a node for maintenance.

6.1.1.2. Worker node failure
Copy link

Worker nodes are the virtual machines (VMs) that contain your application pods. By default, a ROSA cluster has a minimum of two worker nodes for a cluster. In the event of a worker node failure, pods are relocated to functioning worker nodes, as long as there is enough capacity, until any issue with an existing node is resolved or the node is replaced. More worker nodes means more protection against single-node outages, and ensures proper cluster capacity for rescheduled pods in the event of a node failure.

Note

When accounting for possible node failures, it is also important to understand how storage is affected. EFS volumes are not affected by node failure. However, EBS volumes are not accessible if they are connected to a node that fails.

6.1.1.3. Zone failure
Copy link

A zone failure from AWS affects all virtual components, such as worker nodes, block or shared storage, and load balancers that are specific to a single availability zone. To protect against a zone failure, ROSA provides the option for machine pools that are deployed across three availability zones. Existing stateless workloads are then redistributed to unaffected zones in the event of an outage, as long as there is enough capacity.

6.1.1.4. Storage failure
Copy link

If you have deployed a stateful application, then storage is a critical component and must be accounted for when thinking about high availability. A single block storage PV is unable to withstand outages even at the pod level. The best ways to maintain availability of storage are to use replicated storage solutions, shared storage that is unaffected by outages, or a database service that is independent of the cluster.

Additional resources

This documentation outlines Red Hat, Amazon Web Services (AWS), and customer responsibilities for the Red Hat OpenShift Service on AWS managed service.

While Red Hat and Amazon Web Services (AWS) manage the Red Hat OpenShift Service on AWS services, the customer shares certain responsibilities. The Red Hat OpenShift Service on AWS services are accessed remotely, hosted on public cloud resources, created in customer-owned AWS accounts, and have underlying platform and data security that is owned by Red Hat.

Important

If the cluster-admin role is added to a user, see the responsibilities and exclusion notes in the Red Hat Enterprise Agreement Appendix 4 (Online Subscription Services).

Expand
ResourceIncident and operations managementChange managementAccess and identity authorizationSecurity and regulation complianceDisaster recovery

Customer data

Customer

Customer

Customer

Customer

Customer

Customer applications

Customer

Customer

Customer

Customer

Customer

Developer services

Customer

Customer

Customer

Customer

Customer

Platform monitoring

Red Hat

Red Hat

Red Hat

Red Hat

Red Hat

Logging

Red Hat

Red Hat and Customer

Red Hat and Customer

Red Hat and Customer

Red Hat

Application networking

Red Hat and Customer

Red Hat and Customer

Red Hat and Customer

Red Hat

Red Hat

Cluster networking

Red Hat [1]

Red Hat and Customer [2]

Red Hat and Customer

Red Hat [1]

Red Hat [1]

Virtual networking management

Red Hat and Customer

Red Hat and Customer

Red Hat and Customer

Red Hat and Customer

Red Hat and Customer

Virtual compute management (control plane, infrastructure and worker nodes)

Red Hat

Red Hat

Red Hat

Red Hat

Red Hat

Cluster version

Red Hat

Red Hat and Customer

Red Hat

Red Hat

Red Hat

Capacity management

Red Hat

Red Hat and Customer

Red Hat

Red Hat

Red Hat

Virtual storage management

Red Hat

Red Hat

Red Hat

Red Hat

Red Hat

AWS software (public AWS services)

AWS

AWS

AWS

AWS

AWS

Hardware/AWS global infrastructure

AWS

AWS

AWS

AWS

AWS

  1. If the customer chooses to use their own CNI plugin, the responsibility shifts to the customer.
  2. The customer must configure their firewall to grant access to the required OpenShift and AWS domains and ports before the cluster is provisioned. For more information, see "AWS firewall prerequisites".

6.2.3. Review and action cluster notifications
Copy link

Cluster notifications (sometimes referred to as service logs) are messages about the status, health, or performance of your cluster.

Cluster notifications are the primary way that Red Hat Site Reliability Engineering (SRE) communicates with you about the health of your managed cluster. Red Hat SRE may also use cluster notifications to prompt you to perform an action in order to resolve or prevent an issue with your cluster.

Cluster owners and administrators must regularly review and action cluster notifications to ensure clusters remain healthy and supported.

You can view cluster notifications in the Red Hat Hybrid Cloud Console, in the Cluster history tab for your cluster. By default, only the cluster owner receives cluster notifications as emails. If other users need to receive cluster notification emails, add each user as a notification contact for your cluster.

6.2.3.1. Cluster notification policy
Copy link

Cluster notifications are designed to keep you informed about the health of your cluster and high impact events that affect it.

Most cluster notifications are generated and sent automatically to ensure that you are immediately informed of problems or important changes to the state of your cluster.

In certain situations, Red Hat Site Reliability Engineering (SRE) creates and sends cluster notifications to provide additional context and guidance for a complex issue.

Cluster notifications are not sent for low-impact events, low-risk security updates, routine operations and maintenance, or minor, transient issues that are quickly resolved by Red Hat SRE.

Red Hat services automatically send notifications when:

  • Remote health monitoring or environment verification checks detect an issue in your cluster, for example, when a worker node has low disk space.
  • Significant cluster life cycle events occur, for example, when scheduled maintenance or upgrades begin, or cluster operations are impacted by an event, but do not require customer intervention.
  • Significant cluster management changes occur, for example, when cluster ownership or administrative control is transferred from one user to another.
  • Your cluster subscription is changed or updated, for example, when Red Hat makes updates to subscription terms or features available to your cluster.

SRE creates and sends notifications when:

  • An incident results in a degradation or outage that impacts your cluster’s availability or performance, for example, your cloud provider has a regional outage. SRE sends subsequent notifications to inform you of incident resolution progress, and when the incident is resolved.
  • A security vulnerability, security breach, or unusual activity is detected on your cluster.
  • Red Hat detects that changes you have made are creating or may result in cluster instability.
  • Red Hat detects that your workloads are causing performance degradation or instability in your cluster.

6.2.4. Incident and operations management
Copy link

Red Hat is responsible for overseeing the service components required for default platform networking. AWS is responsible for protecting the hardware infrastructure that runs all of the services offered in the AWS Cloud. The customer is responsible for incident and operations management of customer application data and any custom networking the customer has configured for the cluster network or virtual network.

Expand
ResourceService responsibilitiesCustomer responsibilities

Application networking

Red Hat

  • Monitor native OpenShift router service, and respond to alerts.
  • Monitor health of application routes, and the endpoints behind them.
  • Report outages to Red Hat and AWS.

Cluster networking

Red Hat

  • Monitor, alert, and address incidents related to cluster DNS, network plugin connectivity between cluster components, and the default Ingress Controller.
  • Monitor and address incidents related to optional Ingress Controllers, additional Operators installed through the OperatorHub, and network plugins replacing the default OpenShift CNI plugins.

Virtual networking management

Red Hat

  • Monitor AWS load balancers, Amazon VPC subnets, and AWS service components necessary for default platform networking. Respond to alerts.
  • Monitor health of AWS load balancer endpoints.
  • Monitor network traffic that is optionally configured through Amazon VPC-to-VPC connection, AWS VPN connection, or AWS Direct Connect for potential issues or security threats.

Virtual storage management

Red Hat

  • Monitor Amazon EBS volumes attached to cluster nodes and Amazon S3 buckets used for the ROSA service’s built-in container image registry. Respond to alerts.
  • Monitor health of application data.
  • If customer managed AWS KMS keys are used, create and control the key lifecycle and key policies for Amazon EBS encryption.

Platform monitoring

Red Hat

  • Maintain a centralized monitoring and alerting system for all ROSA cluster components, site reliability engineer (SRE) services, and underlying AWS accounts.
 

Incident management

Red Hat

  • Raise and manage known incidents.
  • Share root cause analysis (RCA) drafts with the customer.
  • Raise known incidents through a support case.

Infrastructure and data resiliency

Red Hat

  • There is no Red Hat-provided backup method available for ROSA clusters with STS.
  • Red Hat does not commit to any Recovery Point Objective (RPO) or Recovery Time Objective (RTO).
  • Take regular backups of data and deploy multi-AZ clusters with workloads that follow Kubernetes best practices to ensure high availability within a region.
  • If an entire cloud region is unavailable, install a new cluster in a different region and restore apps using backup data.

Cluster capacity

Red Hat

  • Manage the capacity of all control plane and infrastructure nodes on the cluster.
  • Evaluate cluster capacity during upgrades and in response to cluster alerts.
 

AWS software (public AWS services)

AWS

  • Monitor health of AWS resources in the customer account.
  • Use IAM tools to apply the appropriate permissions to AWS resources in the customer account.

Hardware/AWS global infrastructure

AWS

  • Configure, manage, and monitor customer applications and data to ensure application and data security controls are properly enforced.
6.2.4.1. Platform monitoring
Copy link

Platform audit logs are securely forwarded to a centralized security information and event monitoring (SIEM) system, where they may trigger configured alerts to the Red Hat SRE team and are also subject to manual review. Audit logs are retained in the SIEM system for one year. Audit logs for a given cluster are not deleted at the time the cluster is deleted.

6.2.4.2. Incident management
Copy link

An incident is an event that results in a degradation or outage of one or more Red Hat services.

An incident can be raised by a customer or a Customer Experience and Engagement (CEE) member through a support case, directly by the centralized monitoring and alerting system, or directly by a member of the SRE team.

Depending on the impact on the service and customer, the incident is categorized in terms of severity.

When managing a new incident, Red Hat uses the following general workflow:

  1. An SRE first responder is alerted to a new incident and begins an initial investigation.
  2. After the initial investigation, the incident is assigned an incident lead, who coordinates the recovery efforts.
  3. The incident lead manages all communication and coordination around recovery, including any relevant notifications and support case updates.
  4. When the incident is resolved a brief summary of the incident and resolution are provided in the customer-initiated support ticket. This summary helps the customers understand the incident and its resolution in more detail.

If customers require more information in addition to what is provided in the support ticket, they can request the following workflow:

  1. The customer must make a request for the additional information within 5 business days of the incident resolution.
  2. Depending on the severity of the incident, Red Hat may provide customers with a root cause summary, or a root cause analysis (RCA) in the support ticket. The additional information will be provided within 7 business days for root cause summary and 30 business days for root cause analysis from the incident resolution.

Red Hat also assists with customer incidents raised through support cases. Red Hat can assist with activities including but not limited to:

  • Forensic gathering, including isolating virtual compute
  • Guiding compute image collection
  • Providing collected audit logs
6.2.4.3. Cluster capacity
Copy link

The impact of a cluster upgrade on capacity is evaluated as part of the upgrade testing process to ensure that capacity is not negatively impacted by new additions to the cluster. During a cluster upgrade, additional worker nodes are added to make sure that total cluster capacity is maintained during the upgrade process.

Capacity evaluations by the Red Hat SRE staff also happen in response to alerts from the cluster, after usage thresholds are exceeded for a certain period of time. Such alerts can also result in a notification to the customer.

6.2.5. Change management
Copy link

This section describes the policies about how cluster and configuration changes, patches, and releases are managed.

Red Hat is responsible for enabling changes to the cluster infrastructure and services that the customer will control, as well as maintaining versions for the control plane nodes, infrastructure nodes and services, and worker nodes. AWS is responsible for protecting the hardware infrastructure that runs all of the services offered in the AWS Cloud. The customer is responsible for initiating infrastructure change requests and installing and maintaining optional services and networking configurations on the cluster, as well as all changes to customer data and customer applications.

6.2.5.1. Customer-initiated changes
Copy link

You can initiate changes using self-service capabilities such as cluster deployment, worker node scaling, or cluster deletion.

Change history is captured in the Cluster History section in the OpenShift Cluster Manager Overview tab, and is available for you to view. The change history includes, but is not limited to, logs from the following changes:

  • Adding or removing identity providers
  • Adding or removing users to or from the dedicated-admins group
  • Scaling the cluster compute nodes
  • Scaling the cluster load balancer
  • Scaling the cluster persistent storage
  • Upgrading the cluster

You can implement a maintenance exclusion by avoiding changes in OpenShift Cluster Manager for the following components:

  • Deleting a cluster
  • Adding, modifying, or removing identity providers
  • Adding, modifying, or removing a user from an elevated group
  • Installing or removing add-ons
  • Modifying cluster networking configurations
  • Adding, modifying, or removing machine pools
  • Enabling or disabling user workload monitoring
  • Initiating an upgrade
Important

To enforce the maintenance exclusion, ensure machine pool autoscaling or automatic upgrade policies have been disabled. After the maintenance exclusion has been lifted, proceed with enabling machine pool autoscaling or automatic upgrade policies as desired.

6.2.5.2. Red Hat-initiated changes
Copy link

Red Hat site reliability engineering (SRE) manages the infrastructure, code, and configuration of Red Hat OpenShift Service on AWS using a GitOps workflow and fully automated CI/CD pipelines. This process ensures that Red Hat can safely introduce service improvements on a continuous basis without negatively impacting customers.

Every proposed change undergoes a series of automated verifications immediately upon check-in. Changes are then deployed to a staging environment where they undergo automated integration testing. Finally, changes are deployed to the production environment. Each step is fully automated.

An authorized Red Hat SRE reviewer must approve advancement to each step. The reviewer cannot be the same individual who proposed the change. All changes and approvals are fully auditable as part of the GitOps workflow.

Some changes are released to production incrementally, using feature flags to control availability of new features to specified clusters or customers, such as private or public previews.

6.2.5.3. Patch management
Copy link

OpenShift Container Platform software and the underlying immutable Red Hat CoreOS (RHCOS) operating system image are patched for bugs and vulnerabilities in regular z-stream upgrades. Read more about RHCOS architecture in the OpenShift Container Platform documentation.

6.2.5.4. Release management
Copy link

Red Hat does not automatically upgrade your clusters. You can schedule to upgrade the clusters at regular intervals (recurring upgrade) or just once (individual upgrade) using the OpenShift Cluster Manager web console. Red Hat might forcefully upgrade a cluster to a new z-stream version only if the cluster is affected by a critical impact CVE.

Note

Because the required permissions can change between y-stream releases, the AWS managed policies are automatically updated before an upgrade can be performed.

You can review the history of all cluster upgrade events in the OpenShift Cluster Manager web console. For more information about releases, see the Life Cycle policy.

The following table defines the responsibilities for cluster resources.

Expand
ResourceService responsibilitiesCustomer responsibilities

Logging

Red Hat

  • Centrally aggregate and monitor platform audit logs.
  • Provide and maintain a logging Operator to enable the customer to deploy a logging stack for default application logging.
  • Provide audit logs upon customer request.
  • Install the optional default application logging Operator on the cluster.
  • Install, configure, and maintain any optional application logging solutions, such as logging sidecar containers or third-party logging applications.
  • Tune size and frequency of application logs being produced by customer applications if they are affecting the stability of the logging stack or the cluster.
  • Request platform audit logs through a support case for researching specific incidents.

Application networking

Red Hat

  • Set up public load balancers. Provide the ability to set up private load balancers and up to one additional load balancer when required.
  • Set up native OpenShift router service. Provide the ability to set the router as private and add up to one additional router shard.
  • Install, configure, and maintain OVN-Kubernetes components for default internal pod traffic.
  • Provide the ability for the customer to manage NetworkPolicy and EgressNetworkPolicy (firewall) objects.
  • Configure non-default pod network permissions for project and pod networks, pod ingress, and pod egress using NetworkPolicy objects.
  • Use OpenShift Cluster Manager to request a private load balancer for default application routes.
  • Use OpenShift Cluster Manager to configure up to one additional public or private router shard and corresponding load balancer.
  • Request and configure any additional service load balancers for specific services.
  • Configure any necessary DNS forwarding rules.

Cluster networking

Red Hat

  • Set up cluster management components, such as public or private service endpoints and necessary integration with Amazon VPC components.
  • Set up internal networking components required for internal cluster communication between worker clusters and control planes.
  • Configure your firewall to grant access to the required OpenShift and AWS domains and ports before the cluster is provisioned. For more information, see "AWS firewall prerequisites".
  • Provide optional non-default IP address ranges for machine CIDR, service CIDR, and pod CIDR if needed through OpenShift Cluster Manager when the cluster is provisioned.
  • Request that the API service endpoint be made public or private on cluster creation or after cluster creation through OpenShift Cluster Manager.
  • Create additional Ingress Controllers to publish additional application routes.
  • Install, configure, and upgrade optional CNI plugins if clusters are installed without the default OpenShift CNI plugins.

Virtual networking management

Red Hat

  • Set up and configure Amazon VPC components required to provision the cluster, such as subnets, load balancers, internet gateways, and NAT gateways.
  • Provide the ability for the customer to manage AWS VPN connectivity with on-premise resources, Amazon VPC-to-VPC connectivity, and AWS Direct Connect as required through OpenShift Cluster Manager.
  • Enable customers to create and deploy AWS load balancers for use with service load balancers.
  • Set up and maintain optional Amazon VPC components, such as Amazon VPC-to-VPC connection, AWS VPN connection, or AWS Direct Connect.
  • Request and configure any additional service load balancers for specific services.

Virtual compute management

Red Hat

  • Set up and configure the ROSA control plane and data plane to use Amazon EC2 instances for cluster compute.
  • Monitor and manage the deployment of Amazon EC2 control plane and infrastructure nodes on the cluster.
  • Monitor and manage Amazon EC2 worker nodes by creating a machine pool using the OpenShift Cluster Manager or the ROSA CLI (rosa).
  • Manage changes to customer-deployed applications and application data.

Cluster version

Red Hat

  • Enable upgrade scheduling process.
  • Monitor upgrade progress and remedy any issues encountered.
  • Publish change logs and release notes for patch release upgrades.
  • Either set up automatic upgrades or schedule patch release upgrades immediately or for the future.
  • Acknowledge and schedule minor version upgrades.
  • Test customer applications on patch releases to ensure compatibility.

Capacity management

Red Hat

  • Monitor the use of the control plane.
  • Scale and resize control plane to maintain quality of service.
  • Monitor worker node utilization and, if appropriate, enables the auto-scaling feature.
  • Determine the scaling strategy of the cluster. See the additional resources for more information on machine pools.
  • Use the provided OpenShift Cluster Manager controls to add or remove additional worker nodes as required.
  • Respond to Red Hat notifications regarding cluster resource requirements.

Virtual storage management

Red Hat

  • Set up and configure Amazon EBS to provision local node storage and persistent volume storage for the cluster.
  • Set up and configure the built-in image registry to use Amazon S3 bucket storage. [1]
  • Regularly prune image registry resources in Amazon S3 to optimize Amazon S3 usage and cluster performance. [2]
  • Optionally configure the Amazon EBS CSI driver or the Amazon EFS CSI driver to provision persistent volumes on the cluster.

AWS software (public AWS services)

AWS

Compute: Provide the Amazon EC2 service, used for ROSA relevant resources.

Storage: Provide Amazon EBS, used by ROSA to provision local node storage and persistent volume storage for the cluster.

Storage: Provide Amazon S3, used for the ROSA built-in image registry.

Networking: Provide the following AWS Cloud services, used by ROSA to satisfy virtual networking infrastructure needs:

  • Amazon VPC
  • Elastic Load Balancing
  • AWS IAM
  • AWS STS

Networking: Provide the following AWS services, which customers can optionally integrate with ROSA:

  • AWS VPN
  • AWS Direct Connect
  • AWS PrivateLink
  • AWS Transit Gateway
  • Sign requests using an access key ID and secret access key associated with an IAM principal or STS temporary security credentials.
  • Specify VPC subnets for the cluster to use during cluster creation.
  • Optionally configure a customer-managed VPC for use with ROSA clusters (required for PrivateLink and HCP clusters).

Hardware/AWS global infrastructure

AWS

  • For information regarding management controls for AWS data centers, see Our Controls on the AWS Cloud Security page.
  • For information regarding change management best practices, see Guidance for Change Management on AWS in the AWS Solutions Library.
  • Implement change management best practices for customer applications and data hosted on the AWS Cloud.
  1. For more information on authentication flow for AWS STS, see Authentication flow for AWS STS.
  2. For more information on pruning images, see Automatically Pruning Images.

6.2.6. Security and regulation compliance
Copy link

The following table outlines the the responsibilities in regards to security and regulation compliance:

Expand
ResourceService responsibilitiesCustomer responsibilities

Logging

Red Hat

  • Send cluster audit logs to a Red Hat SIEM to analyze for security events. Retain audit logs for a defined period of time to support forensic analysis.
  • Analyze application logs for security events.
  • Send application logs to an external endpoint through logging sidecar containers or third-party logging applications if longer retention is required than is offered by the default logging stack.

Virtual networking management

Red Hat

  • Monitor virtual networking components for potential issues and security threats.
  • Use public AWS tools for additional monitoring and protection.
  • Monitor optional configured virtual networking components for potential issues and security threats.
  • Configure any necessary firewall rules or customer data center protections as required.

Virtual storage management

Red Hat

  • Monitor virtual storage components for potential issues and security threats.
  • Use public AWS tools for additional monitoring and protection.
  • Configure the ROSA service to encrypt control plane, infrastructure, and worker node volume data by default using the AWS managed Key Management Service (KMS) key that Amazon EBS provides.
  • Configure the ROSA service to encrypt customer persistent volumes that use the default storage class with the AWS managed KMS key that Amazon EBS provides.
  • Provide the ability for the customer to use a customer managed AWS KMS key to encrypt persistent volumes.
  • Configure the container image registry to encrypt image registry data at rest using server-side encryption with Amazon S3 managed keys (SSE-3).
  • Provide the ability for the customer to create a public or private Amazon S3 image registry to protect their container images from unauthorized user access.
  • Provision Amazon EBS volumes.
  • Manage Amazon EBS volume storage to ensure enough storage is available to mount as a volume in ROSA.
  • Create the persistent volume claim and generate a persistent volume though OpenShift Cluster Manager.

Virtual compute management

Red Hat

  • Monitor virtual compute components for potential issues and security threats.
  • Use public AWS tools for additional monitoring and protection.
  • Monitor optional configured virtual networking components for potential issues and security threats.
  • Configure any necessary firewall rules or customer data center protections as required.

AWS software (public AWS services)

AWS

Compute: Secure Amazon EC2, used for ROSA used for ROSA control plane and worker nodes. For more information, see Infrastructure security in Amazon EC2 in the Amazon EC2 User Guide.

Storage: Secure Amazon Elastic Block Store (EBS), used for ROSA control plane and worker node volumes, as well as Kubernetes persistent volumes. For more information, see Data protection in Amazon EC2 in the Amazon EC2 User Guide.

Storage: Provide AWS KMS, which ROSA uses to encrypt control plane, worker node volumes and persistent volumes. For more information, see Amazon EBS encryption in the Amazon EC2 User Guide.

Storage: Secure Amazon S3, used for the ROSA service’s built-in container image registry. For more information, see Amazon S3 security in the S3 User Guide.

Networking: Provide security capabilities and services to increase privacy and control network access on AWS global infrastructure, including network firewalls built into Amazon VPC, private or dedicated network connections, and automatic encryption of all traffic on the AWS global and regional networks between AWS secured facilities. For more information, see the AWS Shared Responsibility Model and Infrastructure security in the Introduction to AWS Security whitepaper.

  • Ensure security best practices and the principle of least privilege are followed to protect data on the Amazon EC2 instance. For more information, see Infrastructure security in Amazon EC2 and Data protection in Amazon EC2.
  • Monitor optional configured virtual networking components for potential issues and security threats.
  • Configure any necessary firewall rules or customer data center protections as required.
  • Create an optional customer managed KMS key and encrypt the Amazon EBS persistent volume using the KMS key.
  • Monitor the customer data in virtual storage for potential issues and security threats. For more information, see the shared responsibility model.

Hardware/AWS global infrastructure

AWS

  • Provide the AWS global infrastructure that ROSA uses to deliver service functionality. For more information regarding AWS security controls, see Security of the AWS Infrastructure in the AWS whitepaper.
  • Provide documentation for the customer to manage compliance needs and check their security state in AWS using tools such as AWS Artifact and AWS Security Hub. For more information, see Compliance validation for ROSA in the ROSA User Guide.
  • Configure, manage, and monitor customer applications and data to ensure application and data security controls are properly enforced.
  • Use IAM tools to apply the appropriate permissions to AWS resources in the customer account.

6.2.7. Disaster recovery
Copy link

Disaster recovery includes data and configuration backup, replicating data and configuration to the disaster recovery environment, and failover on disaster events.

Red Hat OpenShift Service on AWS (ROSA) provides disaster recovery for failures that occur at the pod, node, and availability zone levels.

All disaster recovery requires that the customer use best practices for deploying highly available applications, storage, and cluster architecture, such as multiple machine pools across multiple availability zones, to account for the level of desired availability.

One cluster with a single machine pool will not provide disaster avoidance or recovery in the event of an availability zone or region outage. Multiple clusters with single machine pools with customer-maintained failover can account for outages at the zone or at the regional level.

One cluster with multiple machine pools across multiple availability zones will not provide disaster avoidance or recovery in the event of a full region outage. Multiple clusters in several regions with multiple machine pools in more than one availability-zone with customer-maintained failover can account for outages at the regional level.

Expand
ResourceService responsibilitiesCustomer responsibilities

Virtual networking management

Red Hat

  • Recreate affected virtual network components that are necessary for the platform to function.
  • Configure virtual networking connections with more than one tunnel where possible for protection against outages as recommended by the public cloud provider.
  • Maintain failover DNS and load balancing if using a global load balancer with multiple clusters.

Virtual Storage management

Red Hat

  • Back up customer applications and application data.

Virtual compute management

Red Hat - Provide the ability for the customer to manually or automatically replace failed worker nodes.

  • Replace failed Amazon EC2 worker nodes by editing the machine pool configuration through OpenShift Cluster Manager or the ROSA CLI.

AWS software (public AWS services)

AWS

Compute: Provide Amazon EC2 features that support data resiliency such as Amazon EBS snapshots and Amazon EC2 Auto Scaling. For more information, see Resilience in Amazon EC2 in the EC2 User Guide.

Storage: Provide the ability for the ROSA service and customers to back up the Amazon EBS volume on the cluster through Amazon EBS volume snapshots.

Storage: For information about Amazon S3 features that support data resiliency, see Resilience in Amazon S3.

Networking: For information about Amazon VPC features that support data resiliency, see Resilience in Amazon Virtual Private Cloud in the Amazon VPC User Guide.

  • Configure ROSA clusters with multiple machine pools across multiple availability zones to improve fault tolerance and cluster availability.
  • Provision persistent volumes using the Amazon EBS CSI driver to enable volume snapshots.
  • Create CSI volume snapshots of Amazon EBS persistent volumes.

Hardware/AWS global infrastructure

AWS

  • Provide AWS global infrastructure that allows ROSA to scale nodes across Availability Zones. This functionality enables ROSA to orchestrate automatic failover between zones without interruption.
  • For more information about disaster recovery best practices, see Disaster recovery options in the cloud in the AWS Well-Architected Framework.
  • Configure ROSA clusters with multiple machine pools across multiple availability zones to improve fault tolerance and cluster availability.

The customer is responsible for the applications, workloads, and data that they deploy to Red Hat OpenShift Service on AWS. However, Red Hat and AWS provide various tools to help the customer manage data and applications on the platform.

Expand
ResourceRed Hat and AWSCustomer responsibilities

Customer data

Red Hat

  • Maintain platform-level standards for data encryption as defined by industry security and compliance standards.
  • Provide OpenShift components to help manage application data, such as secrets.
  • Enable integration with data services such as Amazon RDS to store and manage data outside of the cluster and/or AWS.

AWS

  • Provide Amazon RDS to allow customers to store and manage data outside of the cluster and/or AWS.
  • Maintain responsibility for all customer data stored on the platform and how customer applications consume and expose this data.

Customer applications

Red Hat

  • Provision clusters with OpenShift components installed so that customers can access the OpenShift and Kubernetes APIs to deploy and manage containerized applications.
  • Create clusters with image pull secrets so that customer deployments can pull images from the Red Hat Container Catalog registry.
  • Provide access to OpenShift APIs that a customer can use to set up Operators to add community, third-party, and Red Hat services to the cluster.
  • Provide storage classes and plugins to support persistent volumes for use with customer applications.
  • Provide a container image registry so customers can securely store application container images on the cluster to deploy and manage applications.

AWS

  • Provide Amazon EBS to support persistent volumes for use with customer applications.
  • Provide Amazon S3 to support Red Hat provisioning of the container image registry.
  • Maintain responsibility for customer and third-party applications, data, and their complete lifecycle.
  • If a customer adds Red Hat, community, third-party, their own, or other services to the cluster by using Operators or external images, the customer is responsible for these services and for working with the appropriate provider, including Red Hat, to troubleshoot any issues.
  • Use the provided tools and features to configure and deploy; keep up to date; set up resource requests and limits; size the cluster to have enough resources to run apps; set up permissions; integrate with other services; manage any image streams or templates that the customer deploys; externally serve; save, back up, and restore data; and otherwise manage their highly available and resilient workloads.
  • Maintain responsibility for monitoring the applications run on Red Hat OpenShift Service on AWS, including installing and operating software to gather metrics, create alerts, and protect secrets in the application.

This documentation outlines the service definition for the Red Hat OpenShift Service on AWS managed service.

6.3.1. Account management
Copy link

This section provides information about the service definition for Red Hat OpenShift Service on AWS account management.

6.3.1.1. Billing and pricing
Copy link

Red Hat OpenShift Service on AWS is billed directly to your Amazon Web Services (AWS) account. ROSA pricing is consumption based, with annual commitments or three-year commitments for greater discounting. The total cost of ROSA consists of two components:

  • ROSA service fees
  • AWS infrastructure fees

Visit the Red Hat OpenShift Service on AWS Pricing page on the AWS website for more details.

6.3.1.2. Cluster self-service
Copy link

Customers can self-service their clusters, including, but not limited to:

  • Create a cluster
  • Delete a cluster
  • Add or remove an identity provider
  • Add or remove a user from an elevated group
  • Configure cluster privacy
  • Add or remove machine pools and configure autoscaling
  • Define upgrade policies

You can perform these self-service tasks using the Red Hat OpenShift Service on AWS (ROSA) CLI, rosa.

6.3.1.3. Instance types
Copy link

All ROSA with HCP clusters require a minimum of 2 worker nodes. Shutting down the underlying (EC2 instance) infrastructure through the cloud provider console is unsupported and can lead to data loss and other risks.

Note

Approximately one vCPU core and 1 GiB of memory are reserved on each worker node and removed from allocatable resources. This reservation of resources is necessary to run processes required by the underlying platform. These processes include system daemons such as udev, kubelet, and container runtime among others. The reserved resources also account for kernel reservations.

OpenShift/ROSA core systems such as audit log aggregation, metrics collection, DNS, image registry, CNI/OVN-Kubernetes, and others might consume additional allocatable resources to maintain the stability and maintainability of the cluster. The additional resources consumed might vary based on usage.

For additional information, see the Kubernetes documentation.

6.3.1.4. Regions and availability zones
Copy link

The following AWS regions are currently available for ROSA with HCP.

Note

Regions in China are not supported, regardless of their support on OpenShift Container Platform.

Note

For GovCloud (US) regions, you must submit an Access request for Red Hat OpenShift Service on AWS (ROSA) FedRAMP.

Expand
Table 6.1. AWS regions
RegionLocationMinimum ROSA version requiredAWS opt-in required

us-east-1

N. Virginia

4.14

No

us-east-2

Ohio

4.14

No

us-west-2

Oregon

4.14

No

af-south-1

Cape Town

4.14

Yes

ap-east-1

Hong Kong

4.14

Yes

ap-south-2

Hyderabad

4.14

Yes

ap-southeast-3

Jakarta

4.14

Yes

ap-southeast-4

Melbourne

4.14

Yes

ap-southeast-5

Malaysia

4.16.34; 4.17.15

Yes

ap-south-1

Mumbai

4.14

No

ap-northeast-3

Osaka

4.14

No

ap-northeast-2

Seoul

4.14

No

ap-southeast-1

Singapore

4.14

No

ap-southeast-2

Sydney

4.14

No

ap-northeast-1

Tokyo

4.14

No

ca-central-1

Central Canada

4.14

No

eu-central-1

Frankfurt

4.14

No

eu-north-1

Stockholm

4.14

No

eu-west-1

Ireland

4.14

No

eu-west-2

London

4.14

No

eu-south-1

Milan

4.14

Yes

eu-west-3

Paris

4.14

No

eu-south-2

Spain

4.14

Yes

eu-central-2

Zurich

4.14

Yes

me-south-1

Bahrain

4.14

Yes

me-central-1

UAE

4.14

Yes

sa-east-1

São Paulo

4.14

No

il-central-1

Tel Aviv

4.15

Yes

ca-west-1

Calgary

4.14

Yes

Clusters can only be deployed in regions with at least 3 availability zones. For more information, see the Regions and Availability Zones section in the AWS documentation.

Each new ROSA with HCP cluster is installed within a preexisting Virtual Private Cloud (VPC) in a single region, with the option to deploy up to the total number of availability zones for the given region. This provides cluster-level network and resource isolation, and enables cloud-provider VPC settings, such as VPN connections and VPC Peering. Persistent volumes (PVs) are backed by Amazon Elastic Block Storage (Amazon EBS), and are specific to the availability zone in which they are provisioned. Persistent volume claims (PVCs) do not bind to a volume until the associated pod resource is assigned into a specific availability zone to prevent unschedulable pods. Availability zone-specific resources are only usable by resources in the same availability zone.

Warning

The region cannot be changed after a cluster has been deployed.

6.3.1.5. Local Zones
Copy link

Red Hat OpenShift Service on AWS does not support the use of AWS Local Zones.

6.3.1.6. Service Level Agreement (SLA)
Copy link

Any SLAs for the service itself are defined in Appendix 4 of the Red Hat Enterprise Agreement Appendix 4 (Online Subscription Services).

6.3.1.7. Limited support status
Copy link

When a cluster transitions to a Limited Support status, Red Hat no longer proactively monitors the cluster, the SLA is no longer applicable, and credits requested against the SLA are denied. It does not mean that you no longer have product support. In some cases, the cluster can return to a fully-supported status if you remediate the violating factors. However, in other cases, you might have to delete and recreate the cluster.

A cluster might move to a Limited Support status for many reasons, including the following scenarios:

If you remove or replace any native Red Hat OpenShift Service on AWS components or any other component that is installed and managed by Red Hat
If cluster administrator permissions were used, Red Hat is not responsible for any of your or your authorized users’ actions, including those that affect infrastructure services, service availability, or data loss. If Red Hat detects any such actions, the cluster might transition to a Limited Support status. Red Hat notifies you of the status change and you should either revert the action or create a support case to explore remediation steps that might require you to delete and recreate the cluster.

If you have questions about a specific action that might cause a cluster to move to a Limited Support status or need further assistance, open a support ticket.

6.3.1.8. Support
Copy link

Red Hat OpenShift Service on AWS includes Red Hat Premium Support, which can be accessed by using the Red Hat Customer Portal.

See the Red Hat Production Support Terms of Service for support response times.

AWS support is subject to a customer’s existing support contract with AWS.

6.3.2. Logging
Copy link

Red Hat OpenShift Service on AWS provides optional integrated log forwarding to Amazon (AWS) CloudWatch.

6.3.2.1. Cluster audit logging
Copy link

Cluster audit logs are available through AWS CloudWatch, if the integration is enabled. If the integration is not enabled, you can request the audit logs by opening a support case.

6.3.2.2. Application logging
Copy link

Application logs sent to STDOUT are collected by Fluentd and forwarded to AWS CloudWatch through the cluster logging stack, if it is installed.

6.3.3. Monitoring
Copy link

This section provides information about the service definition for Red Hat OpenShift Service on AWS monitoring.

6.3.3.1. Cluster metrics
Copy link

Red Hat OpenShift Service on AWS clusters come with an integrated Prometheus stack for cluster monitoring including CPU, memory, and network-based metrics. This is accessible through the web console. These metrics also allow for horizontal pod autoscaling based on CPU or memory metrics provided by a ROSA user.

6.3.3.2. Cluster notifications
Copy link

Cluster notifications (sometimes referred to as service logs) are messages about the status, health, or performance of your cluster.

Cluster notifications are the primary way that Red Hat Site Reliability Engineering (SRE) communicates with you about the health of your managed cluster. Red Hat SRE may also use cluster notifications to prompt you to perform an action in order to resolve or prevent an issue with your cluster.

Cluster owners and administrators must regularly review and action cluster notifications to ensure clusters remain healthy and supported.

You can view cluster notifications in the Red Hat Hybrid Cloud Console, in the Cluster history tab for your cluster. By default, only the cluster owner receives cluster notifications as emails. If other users need to receive cluster notification emails, add each user as a notification contact for your cluster.

6.3.4. Networking
Copy link

This section provides information about the service definition for ROSA networking.

6.3.4.1. Custom domains for applications
Copy link
Warning

Starting with Red Hat OpenShift Service on AWS 4.14, the Custom Domain Operator is deprecated. To manage Ingress in ROSA 4.14 or later, use the Ingress Operator.

To use a custom hostname for a route, you must update your DNS provider by creating a canonical name (CNAME) record. Your CNAME record should map the OpenShift canonical router hostname to your custom domain. The OpenShift canonical router hostname is shown on the Route Details page after a route is created. Alternatively, a wildcard CNAME record can be created once to route all subdomains for a given hostname to the cluster’s router.

6.3.4.2. Domain validated certificates
Copy link

ROSA includes TLS security certificates needed for both internal and external services on the cluster. For external routes, there are two separate TLS wildcard certificates that are provided and installed on each cluster: one is for the web console and route default hostnames, and the other is for the API endpoint. Let’s Encrypt is the certificate authority used for certificates. Routes within the cluster, such as the internal API endpoint, use TLS certificates signed by the cluster’s built-in certificate authority and require the CA bundle available in every pod for trusting the TLS certificate.

6.3.4.3. Custom certificate authorities for builds
Copy link

ROSA supports the use of custom certificate authorities to be trusted by builds when pulling images from an image registry.

6.3.4.4. Load balancers
Copy link

Red Hat OpenShift Service on AWS only deploys load balancers from the default ingress controller. All other load balancers can be optionally deployed by a customer for secondary ingress controllers or service load balancers.

6.3.4.5. Cluster ingress
Copy link

Project administrators can add route annotations for many different purposes, including ingress control through IP allow-listing.

Ingress policies can also be changed by using NetworkPolicy objects, which leverage the ovs-networkpolicy plugin. This allows for full control over the ingress network policy down to the pod level, including between pods on the same cluster and even in the same namespace.

All cluster ingress traffic will go through the defined load balancers. Direct access to all nodes is blocked by cloud configuration.

6.3.4.6. Cluster egress
Copy link

Pod egress traffic control through EgressNetworkPolicy objects can be used to prevent or limit outbound traffic in ROSA with hosted control planes (HCP).

6.3.4.7. Cloud network configuration
Copy link

Red Hat OpenShift Service on AWS allows for the configuration of a private network connection through AWS-managed technologies, such as:

  • VPN connections
  • VPC peering
  • Transit Gateway
  • Direct Connect
Important

Red Hat site reliability engineers (SREs) do not monitor private network connections. Monitoring of these connections is the responsibility of the customer.

6.3.4.8. DNS forwarding
Copy link

For ROSA clusters that have a private cloud network configuration, a customer can specify internal DNS servers available on that private connection that should be queried for explicitly provided domains.

6.3.4.9. Network verification
Copy link

Network verification checks run automatically when you deploy a ROSA cluster into an existing Virtual Private Cloud (VPC) or create an additional machine pool with a subnet that is new to your cluster. The checks validate your network configuration and highlight errors, enabling you to resolve configuration issues prior to deployment.

You can also run the network verification checks manually to validate the configuration for an existing cluster.

6.3.5. Storage
Copy link

This section provides information about the service definition for Red Hat OpenShift Service on AWS storage.

6.3.5.1. Encrypted-at-rest OS and node storage
Copy link

Worker nodes use encrypted-at-rest Amazon Elastic Block Store (Amazon EBS) storage.

6.3.5.2. Encrypted-at-rest PV
Copy link

EBS volumes that are used for PVs are encrypted-at-rest by default.

6.3.5.3. Block storage (RWO)
Copy link

Persistent volumes (PVs) are backed by Amazon Elastic Block Store (Amazon EBS), which is Read-Write-Once.

PVs can be attached only to a single node at a time and are specific to the availability zone in which they were provisioned. However, PVs can be attached to any node in the availability zone.

Each cloud provider has its own limits for how many PVs can be attached to a single node. See AWS instance type limits for details.

6.3.5.4. Shared Storage (RWX)
Copy link

The AWS CSI Driver can be used to provide RWX support for Red Hat OpenShift Service on AWS. A community Operator is provided to simplify setup. See Amazon Elastic File Storage Setup for Red Hat OpenShift Service on AWS for details.

6.3.6. Platform
Copy link

This section provides information about the service definition for the Red Hat OpenShift Service on AWS platform.

6.3.6.1. Autoscaling
Copy link

Node autoscaling is available on ROSA with HCP. You can configure the autoscaler option to automatically scale the number of machines in a cluster.

6.3.6.2. Multiple availability zone
Copy link

Control plane components are always deployed across multiple availability zones, regardless of a customer’s worker node configuration.

6.3.6.3. Node labels
Copy link

Custom node labels are created by Red Hat during node creation and cannot be changed on ROSA with HCP clusters at this time. However, custom labels are supported when creating new machine pools.

6.3.6.4. Node lifecycle
Copy link

Worker nodes are not guaranteed longevity, and may be replaced at any time as part of the normal operation and management of OpenShift.

A worker node might be replaced in the following circumstances:

  • Machine health checks are deployed and configured to ensure that a worker node with a NotReady status is replaced to ensure smooth operation of the cluster.
  • AWS EC2 instances may be terminated when AWS detects irreparable failure of the underlying hardware that hosts the instance.
  • During upgrades, a new, upgraded node is first created and joined to the cluster. Once this new node has been successfully integrated into the cluster via the previously described automated health checks, an older node is then removed from the cluster.

For all containerized workloads running on a Kubernetes based system, it is best practice to configure applications to be resilient of node replacements.

6.3.6.5. Cluster backup policy
Copy link

Red Hat recommends object-level backup solutions for ROSA clusters. OpenShift API for Data Protection (OADP) is included in OpenShift but not enabled by default. Customers can configure OADP on their clusters to achieve object-level backup and restore capabilities.

Red Hat does not back up customer applications or application data. Customers are solely responsible for applications and their data, and must put their own backup and restore capabilities in place.

Warning

Customers are solely responsible for backing up and restoring their applications and application data. For more information about customer responsibilities, see "Shared responsibility matrix".

6.3.6.6. OpenShift version
Copy link

ROSA with HCP is run as a service. Red Hat SRE team will force upgrade when end of life (EOL) is reached. Upgrade scheduling to the latest version is available.

6.3.6.7. Upgrades
Copy link

Upgrades can be scheduled using the ROSA CLI, rosa, or through OpenShift Cluster Manager.

See the Red Hat OpenShift Service on AWS Life Cycle for more information on the upgrade policy and procedures.

6.3.6.8. Windows Containers
Copy link

Red Hat OpenShift support for Windows Containers is not available on Red Hat OpenShift Service on AWS at this time. Alternatively, it is supported to run Windows based virtual machines on OpenShift Virtualization running on a ROSA cluster.

6.3.6.9. Container engine
Copy link

ROSA with HCP runs on OpenShift 4 and uses CRI-O as the only available container engine (container runtime interface).

6.3.6.10. Operating system
Copy link

ROSA with HCP runs on OpenShift 4 and uses Red Hat CoreOS (RHCOS) as the operating system for all cluster nodes.

6.3.6.11. Red Hat Operator support
Copy link

Red Hat workloads typically refer to Red Hat-provided Operators made available through Operator Hub. Red Hat workloads are not managed by the Red Hat SRE team, and must be deployed on worker nodes. These Operators may require additional Red Hat subscriptions, and may incur additional cloud infrastructure costs. Examples of these Red Hat-provided Operators are:

  • Red Hat Quay
  • Red Hat Advanced Cluster Management
  • Red Hat Advanced Cluster Security
  • Red Hat OpenShift Service Mesh
  • OpenShift Serverless
  • Red Hat OpenShift Logging
  • Red Hat OpenShift Pipelines
  • OpenShift Virtualization
6.3.6.12. Kubernetes Operator support
Copy link

All Operators listed in the OperatorHub marketplace should be available for installation. These Operators are considered customer workloads, and are not monitored nor managed by by Red Hat SRE. Operators authored by Red Hat are supported by Red Hat.

6.3.7. Security
Copy link

This section provides information about the service definition for Red Hat OpenShift Service on AWS security.

6.3.7.1. Authentication provider
Copy link

Authentication for the cluster can be configured using either OpenShift Cluster Manager or cluster creation process or using the ROSA CLI, rosa. ROSA is not an identity provider, and all access to the cluster must be managed by the customer as part of their integrated solution. The use of multiple identity providers provisioned at the same time is supported. The following identity providers are supported:

  • GitHub or GitHub Enterprise
  • GitLab
  • Google
  • LDAP
  • OpenID Connect
  • htpasswd
6.3.7.2. Privileged containers
Copy link

Privileged containers are available for users with the cluster-admin role. Usage of privileged containers as cluster-admin is subject to the responsibilities and exclusion notes in the Red Hat Enterprise Agreement Appendix 4 (Online Subscription Services).

6.3.7.3. Customer administrator user
Copy link

In addition to normal users, Red Hat OpenShift Service on AWS provides access to a ROSA with HCP-specific group called dedicated-admin. Any users on the cluster that are members of the dedicated-admin group:

  • Have administrator access to all customer-created projects on the cluster.
  • Can manage resource quotas and limits on the cluster.
  • Can add and manage NetworkPolicy objects.
  • Are able to view information about specific nodes and PVs in the cluster, including scheduler information.
  • Can access the reserved dedicated-admin project on the cluster, which allows for the creation of service accounts with elevated privileges and also gives the ability to update default limits and quotas for projects on the cluster.
  • Can install Operators from OperatorHub and perform all verbs in all *.operators.coreos.com API groups.
6.3.7.4. Cluster administration role
Copy link

The administrator of Red Hat OpenShift Service on AWS has default access to the cluster-admin role for your organization’s cluster. While logged into an account with the cluster-admin role, users have increased permissions to run privileged security contexts.

6.3.7.5. Project self-service
Copy link

By default, all users have the ability to create, update, and delete their projects. This can be restricted if a member of the dedicated-admin group removes the self-provisioner role from authenticated users: 

$ oc adm policy remove-cluster-role-from-group self-provisioner system:authenticated:oauth
Copy to Clipboard Toggle word wrap

Restrictions can be reverted by applying: 

$ oc adm policy add-cluster-role-to-group self-provisioner system:authenticated:oauth
Copy to Clipboard Toggle word wrap
6.3.7.6. Regulatory compliance
Copy link

See the Compliance table in Understanding process and security for ROSA for the latest compliance information.

6.3.7.7. Network security
Copy link

With Red Hat OpenShift Service on AWS, AWS provides a standard DDoS protection on all load balancers, called AWS Shield. This provides 95% protection against most commonly used level 3 and 4 attacks on all the public facing load balancers used for ROSA. A 10-second timeout is added for HTTP requests coming to the haproxy router to receive a response or the connection is closed to provide additional protection.

6.3.7.8. etcd encryption
Copy link

In Red Hat OpenShift Service on AWS, the control plane storage is encrypted at rest by default, including encryption of the etcd volumes. This storage-level encryption is provided through the storage layer of the cloud provider.

Customers can also opt to encrypt the etcd database at build time or provide their own custom AWS KMS keys for the purpose of encrypting the etcd database.

Etcd encryption will encrypt the following Kubernetes API server and OpenShift API server resources:

  • Secrets
  • Config maps
  • Routes
  • OAuth access tokens
  • OAuth authorize tokens

ROSA with HCP offers the following worker node instance types and sizes:

Note

Currently, ROSA with HCP supports a maximum of 500 worker nodes.

6.4.1. AWS x86-based instance types
Copy link

Example 6.1. General purpose

  • m5.xlarge (4 vCPU, 16 GiB)
  • m5.2xlarge (8 vCPU, 32 GiB)
  • m5.4xlarge (16 vCPU, 64 GiB)
  • m5.8xlarge (32 vCPU, 128 GiB)
  • m5.12xlarge (48 vCPU, 192 GiB)
  • m5.16xlarge (64 vCPU, 256 GiB)
  • m5.24xlarge (96 vCPU, 384 GiB)
  • m5.metal (96† vCPU, 384 GiB)
  • m5a.xlarge (4 vCPU, 16 GiB)
  • m5a.2xlarge (8 vCPU, 32 GiB)
  • m5a.4xlarge (16 vCPU, 64 GiB)
  • m5a.8xlarge (32 vCPU, 128 GiB)
  • m5a.12xlarge (48 vCPU, 192 GiB)
  • m5a.16xlarge (64 vCPU, 256 GiB)
  • m5a.24xlarge (96 vCPU, 384 GiB)
  • m5dn.metal (96 vCPU, 384 GiB)
  • m5zn.metal (48 vCPU, 192 GiB)
  • m5d.metal (96† vCPU, 384 GiB)
  • m5n.metal (96 vCPU, 384 GiB)
  • m6a.xlarge (4 vCPU, 16 GiB)
  • m6a.2xlarge (8 vCPU, 32 GiB)
  • m6a.4xlarge (16 vCPU, 64 GiB)
  • m6a.8xlarge (32 vCPU, 128 GiB)
  • m6a.12xlarge (48 vCPU, 192 GiB)
  • m6a.16xlarge (64 vCPU, 256 GiB)
  • m6a.24xlarge (96 vCPU, 384 GiB)
  • m6a.32xlarge (128 vCPU, 512 GiB)
  • m6a.48xlarge (192 vCPU, 768 GiB)
  • m6a.metal (192 vCPU, 768 GiB)
  • m6i.xlarge (4 vCPU, 16 GiB)
  • m6i.2xlarge (8 vCPU, 32 GiB)
  • m6i.4xlarge (16 vCPU, 64 GiB)
  • m6i.8xlarge (32 vCPU, 128 GiB)
  • m6i.12xlarge (48 vCPU, 192 GiB)
  • m6i.16xlarge (64 vCPU, 256 GiB)
  • m6i.24xlarge (96 vCPU, 384 GiB)
  • m6i.32xlarge (128 vCPU, 512 GiB)
  • m6i.metal (128 vCPU, 512 GiB)
  • m6id.xlarge (4 vCPU, 16 GiB)
  • m6id.2xlarge (8 vCPU, 32 GiB)
  • m6id.4xlarge (16 vCPU, 64 GiB)
  • m6id.8xlarge (32 vCPU, 128 GiB)
  • m6id.12xlarge (48 vCPU, 192 GiB)
  • m6id.16xlarge (64 vCPU, 256 GiB)
  • m6id.24xlarge (96 vCPU, 384 GiB)
  • m6id.32xlarge (128 vCPU, 512 GiB)
  • m6id.metal (128 vCPU, 512 GiB)
  • m6idn.xlarge (4 vCPU, 16 GiB)
  • m6idn.2xlarge (8 vCPU, 32 GiB)
  • m6idn.4xlarge (16 vCPU, 64 GiB)
  • m6idn.8xlarge (32 vCPU, 128 GiB)
  • m6idn.12xlarge (48 vCPU, 192 GiB)
  • m6idn.16xlarge (64 vCPU, 256 GiB)
  • m6idn.24xlarge (96 vCPU, 384 GiB)
  • m6idn.32xlarge (128 vCPU, 512 GiB)
  • m6in.xlarge (4 vCPU, 16 GiB)
  • m6in.2xlarge (8 vCPU, 32 GiB)
  • m6in.4xlarge (16 vCPU, 64 GiB)
  • m6in.8xlarge (32 vCPU, 128 GiB)
  • m6in.12xlarge (48 vCPU, 192 GiB)
  • m6in.16xlarge (64 vCPU, 256 GiB)
  • m6in.24xlarge (96 vCPU, 384 GiB)
  • m6in.32xlarge (128 vCPU, 512 GiB)
  • m7a.xlarge (4 vCPU, 16 GiB)
  • m7a.2xlarge (8 vCPU, 32 GiB)
  • m7a.4xlarge (16 vCPU, 64 GiB)
  • m7a.8xlarge (32 vCPU, 128 GiB)
  • m7a.12xlarge (48 vCPU, 192 GiB)
  • m7a.16xlarge (64 vCPU, 256 GiB)
  • m7a.24xlarge (96 vCPU, 384 GiB)
  • m7a.32xlarge (128 vCPU, 512 GiB)
  • m7a.48xlarge (192 vCPU, 768 GiB)
  • m7a.metal-48xl (192 vCPU, 768 GiB)
  • m7i-flex.2xlarge (8 vCPU, 32 GiB)
  • m7i-flex.4xlarge (16 vCPU, 64 GiB)
  • m7i-flex.8xlarge (32 vCPU, 128 GiB)
  • m7i-flex.xlarge (4 vCPU, 16 GiB)
  • m7i.xlarge (4 vCPU, 16 GiB)
  • m7i.2xlarge (8 vCPU, 32 GiB)
  • m7i.4xlarge (16 vCPU, 64 GiB)
  • m7i.8xlarge (32 vCPU, 128 GiB)
  • m7i.12xlarge (48 vCPU, 192 GiB)
  • m7i.16xlarge (64 vCPU, 256 GiB)
  • m7i.24xlarge (96 vCPU, 384 GiB)
  • m7i.48xlarge (192 vCPU, 768 GiB)
  • m7i.metal-24xl (96 vCPU, 384 GiB)
  • m7i.metal-48xl (192 vCPU, 768 GiB)

† These instance types offer 96 logical processors on 48 physical cores. They run on single servers with two physical Intel sockets.

Example 6.2. Burstable general purpose

  • t3.xlarge (4 vCPU, 16 GiB)
  • t3.2xlarge (8 vCPU, 32 GiB)
  • t3a.xlarge (4 vCPU, 16 GiB)
  • t3a.2xlarge (8 vCPU, 32 GiB)

Example 6.3. Memory intensive

  • u7i-6tb.112xlarge (448 vCPU, 6,144 GiB)
  • u7i-8tb.112xlarge (448 vCPU, 6,144 GiB)
  • u7i-12tb.224xlarge (896 vCPU, 12,288 GiB)
  • u7in-16tb.224xlarge (896 vCPU, 16,384 GiB)
  • u7in-24tb.224xlarge (896 vCPU, 24,576 GiB)
  • u7in-32tb.224xlarge (896 vCPU, 32,768 GiB)
  • u7inh-32tb.480xlarge (1920 vCPU, 32,768 GiB)
  • x1.16xlarge (64 vCPU, 976 GiB)
  • x1.32xlarge (128 vCPU, 1,952 GiB)
  • x1e.xlarge (4 vCPU, 122 GiB)
  • x1e.2xlarge (8 vCPU, 244 GiB)
  • x1e.4xlarge (16 vCPU, 488 GiB)
  • x1e.8xlarge (32 vCPU, 976 GiB)
  • x1e.16xlarge (64 vCPU, 1,952 GiB)
  • x1e.32xlarge (128 vCPU, 3,904 GiB)
  • x2idn.16xlarge (64 vCPU, 1,024 GiB)
  • x2idn.24xlarge (96 vCPU, 1,536 GiB)
  • x2idn.32xlarge (128 vCPU, 2,048 GiB)
  • x2iedn.xlarge (4 vCPU, 128 GiB)
  • x2iedn.2xlarge (8 vCPU, 256 GiB)
  • x2iedn.4xlarge (16 vCPU, 512 GiB)
  • x2iedn.8xlarge (32 vCPU, 1,024 GiB)
  • x2iedn.16xlarge (64 vCPU, 2,048 GiB)
  • x2iedn.24xlarge (96 vCPU, 3,072 GiB)
  • x2iedn.32xlarge (128 vCPU, 4,096 GiB)
  • x2iezn.2xlarge (8 vCPU, 256 GiB)
  • x2iezn.4xlarge (16vCPU, 512 GiB)
  • x2iezn.6xlarge (24vCPU, 768 GiB)
  • x2iezn.8xlarge (32vCPU, 1,024 GiB)
  • x2iezn.12xlarge (48vCPU, 1,536 GiB)
  • x2iezn.metal (48 vCPU, 1,536 GiB)
  • x2idn.metal (128vCPU, 2,048 GiB)
  • x2iedn.metal (128vCPU, 4,096 GiB)

Example 6.4. Memory optimized

  • r4.xlarge (4 vCPU, 30.5 GiB)
  • r4.2xlarge (8 vCPU, 61 GiB)
  • r4.4xlarge (16 vCPU, 122 GiB)
  • r4.8xlarge (32 vCPU, 244 GiB)
  • r4.16xlarge (64 vCPU, 488 GiB)
  • r5.xlarge (4 vCPU, 32 GiB)
  • r5.2xlarge (8 vCPU, 64 GiB)
  • r5.4xlarge (16 vCPU, 128 GiB)
  • r5.8xlarge (32 vCPU, 256 GiB)
  • r5.12xlarge (48 vCPU, 384 GiB)
  • r5.16xlarge (64 vCPU, 512 GiB)
  • r5.24xlarge (96 vCPU, 768 GiB)
  • r5.metal (96† vCPU, 768 GiB)
  • r5a.xlarge (4 vCPU, 32 GiB)
  • r5a.2xlarge (8 vCPU, 64 GiB)
  • r5a.4xlarge (16 vCPU, 128 GiB)
  • r5a.8xlarge (32 vCPU, 256 GiB)
  • r5a.12xlarge (48 vCPU, 384 GiB)
  • r5a.16xlarge (64 vCPU, 512 GiB)
  • r5a.24xlarge (96 vCPU, 768 GiB)
  • r5ad.xlarge (4 vCPU, 32 GiB)
  • r5ad.2xlarge (8 vCPU, 64 GiB)
  • r5ad.4xlarge (16 vCPU, 128 GiB)
  • r5ad.8xlarge (32 vCPU, 256 GiB)
  • r5ad.12xlarge (48 vCPU, 384 GiB)
  • r5ad.16xlarge (64 vCPU, 512 GiB)
  • r5ad.24xlarge (96 vCPU, 768 GiB)
  • r5b.xlarge (4 vCPU, 32 GiB)
  • r5b.2xlarge (8 vCPU, 364 GiB)
  • r5b.4xlarge (16 vCPU, 3,128 GiB)
  • r5b.8xlarge (32 vCPU, 3,256 GiB)
  • r5b.12xlarge (48 vCPU, 3,384 GiB)
  • r5b.16xlarge (64 vCPU, 3,512 GiB)
  • r5b.24xlarge (96 vCPU, 3,768 GiB)
  • r5b.metal (96 768 GiB)
  • r5d.xlarge (4 vCPU, 32 GiB)
  • r5d.2xlarge (8 vCPU, 64 GiB)
  • r5d.4xlarge (16 vCPU, 128 GiB)
  • r5d.8xlarge (32 vCPU, 256 GiB)
  • r5d.12xlarge (48 vCPU, 384 GiB)
  • r5d.16xlarge (64 vCPU, 512 GiB)
  • r5d.24xlarge (96 vCPU, 768 GiB)
  • r5d.metal (96† vCPU, 768 GiB)
  • r5n.xlarge (4 vCPU, 32 GiB)
  • r5n.2xlarge (8 vCPU, 64 GiB)
  • r5n.4xlarge (16 vCPU, 128 GiB)
  • r5n.8xlarge (32 vCPU, 256 GiB)
  • r5n.12xlarge (48 vCPU, 384 GiB)
  • r5n.16xlarge (64 vCPU, 512 GiB)
  • r5n.24xlarge (96 vCPU, 768 GiB)
  • r5n.metal (96 vCPU, 768 GiB)
  • r5dn.xlarge (4 vCPU, 32 GiB)
  • r5dn.2xlarge (8 vCPU, 64 GiB)
  • r5dn.4xlarge (16 vCPU, 128 GiB)
  • r5dn.8xlarge (32 vCPU, 256 GiB)
  • r5dn.12xlarge (48 vCPU, 384 GiB)
  • r5dn.16xlarge (64 vCPU, 512 GiB)
  • r5dn.24xlarge (96 vCPU, 768 GiB)
  • r5dn.metal (96 vCPU, 768 GiB)
  • r6a.xlarge (4 vCPU, 32 GiB)
  • r6a.2xlarge (8 vCPU, 64 GiB)
  • r6a.4xlarge (16 vCPU, 128 GiB)
  • r6a.8xlarge (32 vCPU, 256 GiB)
  • r6a.12xlarge (48 vCPU, 384 GiB)
  • r6a.16xlarge (64 vCPU, 512 GiB)
  • r6a.24xlarge (96 vCPU, 768 GiB)
  • r6a.32xlarge (128 vCPU, 1,024 GiB)
  • r6a.48xlarge (192 vCPU, 1,536 GiB)
  • r6a.metal (192 vCPU, 1,536 GiB)
  • r6i.xlarge (4 vCPU, 32 GiB)
  • r6i.2xlarge (8 vCPU, 64 GiB)
  • r6i.4xlarge (16 vCPU, 128 GiB)
  • r6i.8xlarge (32 vCPU, 256 GiB)
  • r6i.12xlarge (48 vCPU, 384 GiB)
  • r6i.16xlarge (64 vCPU, 512 GiB)
  • r6i.24xlarge (96 vCPU, 768 GiB)
  • r6i.32xlarge (128 vCPU, 1,024 GiB)
  • r6i.metal (128 vCPU, 1,024 GiB)
  • r6id.xlarge (4 vCPU, 32 GiB)
  • r6id.2xlarge (8 vCPU, 64 GiB)
  • r6id.4xlarge (16 vCPU, 128 GiB)
  • r6id.8xlarge (32 vCPU, 256 GiB)
  • r6id.12xlarge (48 vCPU, 384 GiB)
  • r6id.16xlarge (64 vCPU, 512 GiB)
  • r6id.24xlarge (96 vCPU, 768 GiB)
  • r6id.32xlarge (128 vCPU, 1,024 GiB)
  • r6id.metal (128 vCPU, 1,024 GiB)
  • r6idn.12xlarge (48 vCPU, 384 GiB)
  • r6idn.16xlarge (64 vCPU, 512 GiB)
  • r6idn.24xlarge (96 vCPU, 768 GiB)
  • r6idn.2xlarge (8 vCPU, 64 GiB)
  • r6idn.32xlarge (128 vCPU, 1,024 GiB)
  • r6idn.4xlarge (16 vCPU, 128 GiB)
  • r6idn.8xlarge (32 vCPU, 256 GiB)
  • r6idn.xlarge (4 vCPU, 32 GiB)
  • r6in.12xlarge (48 vCPU, 384 GiB)
  • r6in.16xlarge (64 vCPU, 512 GiB)
  • r6in.24xlarge (96 vCPU, 768 GiB)
  • r6in.2xlarge (8 vCPU, 64 GiB)
  • r6in.32xlarge (128 vCPU, 1,024 GiB)
  • r6in.4xlarge (16 vCPU, 128 GiB)
  • r6in.8xlarge (32 vCPU, 256 GiB)
  • r6in.xlarge (4 vCPU, 32 GiB)
  • r7a.xlarge (4 vCPU, 32 GiB)
  • r7a.2xlarge (8 vCPU, 64 GiB)
  • r7a.4xlarge (16 vCPU, 128 GiB)
  • r7a.8xlarge (32 vCPU, 256 GiB)
  • r7a.12xlarge (48 vCPU, 384 GiB)
  • r7a.16xlarge (64 vCPU, 512 GiB)
  • r7a.24xlarge (96 vCPU, 768 GiB)
  • r7a.32xlarge (128 vCPU, 1024 GiB)
  • r7a.48xlarge (192 vCPU, 1536 GiB)
  • r7a.metal-48xl (192 vCPU, 1536 GiB)
  • r7i.xlarge (4 vCPU, 32 GiB)
  • r7i.2xlarge (8 vCPU, 64 GiB)
  • r7i.4xlarge (16 vCPU, 128 GiB)
  • r7i.8xlarge (32 vCPU, 256 GiB)
  • r7i.12xlarge (48 vCPU, 384 GiB)
  • r7i.16xlarge (64 vCPU, 512 GiB)
  • r7i.24xlarge (96 vCPU, 768 GiB)
  • r7i.metal-24xl (96 vCPU, 768 GiB)
  • r7iz.xlarge (4 vCPU, 32 GiB)
  • r7iz.2xlarge (8 vCPU, 64 GiB)
  • r7iz.4xlarge (16 vCPU, 128 GiB)
  • r7iz.8xlarge (32 vCPU, 256 GiB)
  • r7iz.12xlarge (48 vCPU, 384 GiB)
  • r7iz.16xlarge (64 vCPU, 512 GiB)
  • r7iz.32xlarge (128 vCPU, 1024 GiB)
  • r7iz.metal-16xl (64 vCPU, 512 GiB)
  • r7iz.metal-32xl (128 vCPU, 1,024 GiB)
  • z1d.xlarge (4 vCPU, 32 GiB)
  • z1d.2xlarge (8 vCPU, 64 GiB)
  • z1d.3xlarge (12 vCPU, 96 GiB)
  • z1d.6xlarge (24 vCPU, 192 GiB)
  • z1d.12xlarge (48 vCPU, 384 GiB)
  • z1d.metal (48‡ vCPU, 384 GiB)

† These instance types offer 96 logical processors on 48 physical cores. They run on single servers with two physical Intel sockets.

‡ This instance type offers 48 logical processors on 24 physical cores.

Example 6.5. Accelerated computing

  • p3.2xlarge (8 vCPU, 61 GiB)
  • p3.8xlarge (32 vCPU, 244 GiB)
  • p3.16xlarge (64 vCPU, 488 GiB)
  • p3dn.24xlarge (96 vCPU, 768 GiB)
  • p4d.24xlarge (96 vCPU, 1,152 GiB)
  • p4de.24xlarge (96 vCPU, 1,152 GiB)
  • p5.48xlarge (192 vCPU, 2,048 GiB)
  • p5e.48xlarge (192 vCPU, 2,048 GiB)
  • p5en.48xlarge (192 vCPU, 2,048 GiB)
  • g4ad.xlarge (4 vCPU, 16 GiB)
  • g4ad.2xlarge (8 vCPU, 32 GiB)
  • g4ad.4xlarge (16 vCPU, 64 GiB)
  • g4ad.8xlarge (32 vCPU, 128 GiB)
  • g4ad.16xlarge (64 vCPU, 256 GiB)
  • g4dn.xlarge (4 vCPU, 16 GiB)
  • g4dn.2xlarge (8 vCPU, 32 GiB)
  • g4dn.4xlarge (16 vCPU, 64 GiB)
  • g4dn.8xlarge (32 vCPU, 128 GiB)
  • g4dn.12xlarge (48 vCPU, 192 GiB)
  • g4dn.16xlarge (64 vCPU, 256 GiB)
  • g4dn.metal (96 vCPU, 384 GiB)
  • g5.xlarge (4 vCPU, 16 GiB)
  • g5.2xlarge (8 vCPU, 32 GiB)
  • g5.4xlarge (16 vCPU, 64 GiB)
  • g5.8xlarge (32 vCPU, 128 GiB)
  • g5.16xlarge (64 vCPU, 256 GiB)
  • g5.12xlarge (48 vCPU, 192 GiB)
  • g5.24xlarge (96 vCPU, 384 GiB)
  • g5.48xlarge (192 vCPU, 768 GiB)
  • dl1.24xlarge (96 vCPU, 768 GiB)†
  • g6.xlarge (4 vCPU, 16 GiB)
  • g6.2xlarge (8 vCPU, 32 GiB)
  • g6.4xlarge (16 vCPU, 64 GiB)
  • g6.8xlarge (32 vCPU, 128 GiB)
  • g6.12xlarge (48 vCPU, 192 GiB)
  • g6.16xlarge (64 vCPU, 256 GiB)
  • g6.24xlarge (96 vCPU, 384 GiB)
  • g6.48xlarge (192 vCPU, 768 GiB)
  • g6e.xlarge (4 vCPU, 32 GiB)
  • g6e.2xlarge (8 vCPU, 64 GiB)
  • g6e.4xlarge (16 vCPU, 128 GiB)
  • g6e.8xlarge (32 vCPU, 256 GiB)
  • g6e.12xlarge (48 vCPU, 384 GiB)
  • g6e.16xlarge (64 vCPU, 512 GiB)
  • g6e.24xlarge (96 vCPU, 768 GiB)
  • g6e.48xlarge (192 vCPU, 1,536 GiB)
  • gr6.4xlarge (16 vCPU, 128 GiB)
  • gr6.8xlarge (32 vCPU, 256 GiB)
  • p6-b200.48xlarge (192 vCPU, 2,048 GiB)

† Intel specific; not covered by Nvidia

Support for the GPU instance type software stack is provided by AWS. Ensure that your AWS service quotas can accommodate the desired GPU instance types.

Example 6.6. Accelerated computing - AWS Trainium and Inferentia

Warning

For more information about AWS Trainium and Inferentia instance types, see Inferentia & Trainium instances on ROSA.

  • trn1.2xlarge (8 vCPU, 32 GiB)
  • trn1.32xlarge (128 vCPU, 512 GiB)
  • trn1n.32xlarge (128 vCPU, 512 GiB)
  • trn2.48xlarge (192 vCPU, 2048 GiB)
  • trn2u.48xlarge (192 vCPU, 2048 GiB)
  • inf1.xlarge (4 vCPU, 8 GiB)
  • inf1.2xlarge (8 vCPU, 16 GiB)
  • inf1.6xlarge (24 vCPU, 48 GiB)
  • inf1.24xlarge (96 vCPU, 192 GiB)
  • inf2.xlarge (4 vCPU, 16 GiB)
  • inf2.8xlarge (32 vCPU, 128 GiB)
  • inf2.24xlarge (96 vCPU, 384 GiB)
  • inf2.48xlarge (192 vCPU, 768 GiB)

Example 6.7. Compute optimized

  • c5.xlarge (4 vCPU, 8 GiB)
  • c5.2xlarge (8 vCPU, 16 GiB)
  • c5.4xlarge (16 vCPU, 32 GiB)
  • c5.9xlarge (36 vCPU, 72 GiB)
  • c5.12xlarge (48 vCPU, 96 GiB)
  • c5.18xlarge (72 vCPU, 144 GiB)
  • c5.24xlarge (96 vCPU, 192 GiB)
  • c5.metal (96 vCPU, 192 GiB)
  • c5d.xlarge (4 vCPU, 8 GiB)
  • c5d.2xlarge (8 vCPU, 16 GiB)
  • c5d.4xlarge (16 vCPU, 32 GiB)
  • c5d.9xlarge (36 vCPU, 72 GiB)
  • c5d.12xlarge (48 vCPU, 96 GiB)
  • c5d.18xlarge (72 vCPU, 144 GiB)
  • c5d.24xlarge (96 vCPU, 192 GiB)
  • c5d.metal (96 vCPU, 192 GiB)
  • c5a.xlarge (4 vCPU, 8 GiB)
  • c5a.2xlarge (8 vCPU, 16 GiB)
  • c5a.4xlarge (16 vCPU, 32 GiB)
  • c5a.8xlarge (32 vCPU, 64 GiB)
  • c5a.12xlarge (48 vCPU, 96 GiB)
  • c5a.16xlarge (64 vCPU, 128 GiB)
  • c5a.24xlarge (96 vCPU, 192 GiB)
  • c5ad.xlarge (4 vCPU, 8 GiB)
  • c5ad.2xlarge (8 vCPU, 16 GiB)
  • c5ad.4xlarge (16 vCPU, 32 GiB)
  • c5ad.8xlarge (32 vCPU, 64 GiB)
  • c5ad.12xlarge (48 vCPU, 96 GiB)
  • c5ad.16xlarge (64 vCPU, 128 GiB)
  • c5ad.24xlarge (96 vCPU, 192 GiB)
  • c5n.xlarge (4 vCPU, 10.5 GiB)
  • c5n.2xlarge (8 vCPU, 21 GiB)
  • c5n.4xlarge (16 vCPU, 42 GiB)
  • c5n.9xlarge (36 vCPU, 96 GiB)
  • c5n.18xlarge (72 vCPU, 192 GiB)
  • c5n.metal (72 vCPU, 192 GiB)
  • c6a.xlarge (4 vCPU, 8 GiB)
  • c6a.2xlarge (8 vCPU, 16 GiB)
  • c6a.4xlarge (16 vCPU, 32 GiB)
  • c6a.8xlarge (32 vCPU, 64 GiB)
  • c6a.12xlarge (48 vCPU, 96 GiB)
  • c6a.16xlarge (64 vCPU, 128 GiB)
  • c6a.24xlarge (96 vCPU, 192 GiB)
  • c6a.32xlarge (128 vCPU, 256 GiB)
  • c6a.48xlarge (192 vCPU, 384 GiB)
  • c6a.metal (192 vCPU, 384 GiB)
  • c6i.xlarge (4 vCPU, 8 GiB)
  • c6i.2xlarge (8 vCPU, 16 GiB)
  • c6i.4xlarge (16 vCPU, 32 GiB)
  • c6i.8xlarge (32 vCPU, 64 GiB)
  • c6i.12xlarge (48 vCPU, 96 GiB)
  • c6i.16xlarge (64 vCPU, 128 GiB)
  • c6i.24xlarge (96 vCPU, 192 GiB)
  • c6i.32xlarge (128 vCPU, 256 GiB)
  • c6i.metal (128 vCPU, 256 GiB)
  • c6id.xlarge (4 vCPU, 8 GiB)
  • c6id.2xlarge (8 vCPU, 16 GiB)
  • c6id.4xlarge (16 vCPU, 32 GiB)
  • c6id.8xlarge (32 vCPU, 64 GiB)
  • c6id.12xlarge (48 vCPU, 96 GiB)
  • c6id.16xlarge (64 vCPU, 128 GiB)
  • c6id.24xlarge (96 vCPU, 192 GiB)
  • c6id.32xlarge (128 vCPU, 256 GiB)
  • c6id.metal (128 vCPU, 256 GiB)
  • c6in.12xlarge (48 vCPU, 96 GiB)
  • c6in.16xlarge (64 vCPU, 128 GiB)
  • c6in.24xlarge (96 vCPU, 192 GiB)
  • c6in.2xlarge (8 vCPU, 16 GiB)
  • c6in.32xlarge (128 vCPU, 256 GiB)
  • c6in.4xlarge (16 vCPU, 32 GiB)
  • c6in.8xlarge (32 vCPU, 64 GiB)
  • c6in.xlarge (4 vCPU, 8 GiB)
  • c7a.xlarge (4 vCPU, 8 GiB)
  • c7a.2xlarge (8 vCPU, 16 GiB)
  • c7a.4xlarge (16 vCPU, 32 GiB)
  • c7a.8xlarge (32 vCPU, 64 GiB)
  • c7a.12xlarge (48 vCPU, 96 GiB)
  • c7a.16xlarge (64 vCPU, 128 GiB)
  • c7a.24xlarge (96 vCPU, 192 GiB)
  • c7a.32xlarge (128 vCPU, 256 GiB)
  • c7a.48xlarge (192 vCPU, 384 GiB)
  • c7a.metal-48xl (192 vCPU, 384 GiB)
  • c7i.xlarge (4 vCPU, 8 GiB)
  • c7i.2xlarge (8 vCPU, 16 GiB)
  • c7i.4xlarge (16 vCPU, 32 GiB)
  • c7i.8xlarge (32 vCPU, 64 GiB)
  • c7i.12xlarge (48 vCPU, 96 GiB)
  • c7i.16xlarge (64 vCPU, 128 GiB)
  • c7i.24xlarge (96 vCPU, 192 GiB)
  • c7i.48xlarge (192 vCPU, 384 GiB)
  • c7i-flex.xlarge (4 vCPU, 8 GiB)
  • c7i-flex.2xlarge (8 vCPU, 16 GiB)
  • c7i-flex.4xlarge (16 vCPU, 32 GiB)
  • c7i-flex.8xlarge (32 vCPU, 64 GiB)
  • c7i.metal-24xl (96 vCPU, 192 GiB)
  • c7i.metal-48xl (192 vCPU, 384 GiB)
  • hpc6a.48xlarge (96 vCPU, 384 GiB)
  • hpc6id.32xlarge (64 vCPU, 1024 GiB)
  • hpc7a.12xlarge (24 vCPU, 768 GiB)
  • hpc7a.24xlarge (48 vCPU, 768 GiB)
  • hpc7a.48xlarge (96 vCPU, 768 GiB)
  • hpc7a.96xlarge (192 vCPU, 768 GiB)
  • m5zn.12xlarge (48 vCPU, 192 GiB)
  • m5zn.2xlarge (8 vCPU, 32 GiB)
  • m5zn.3xlarge (16 vCPU, 48 GiB)
  • m5zn.6xlarge (32 vCPU, 96 GiB)
  • m5zn.xlarge (4 vCPU, 16 GiB)

Example 6.8. Storage optimized

  • c5ad.12xlarge (48 vCPU, 96 GiB)
  • c5ad.16xlarge (64 vCPU, 128 GiB)
  • c5ad.24xlarge (96 vCPU, 192 GiB)
  • c5ad.2xlarge (8 vCPU, 16 GiB)
  • c5ad.4xlarge (16 vCPU, 32 GiB)
  • c5ad.8xlarge (32 vCPU, 64 GiB)
  • c5ad.xlarge (4 vCPU, 8 GiB)
  • i3.xlarge (4 vCPU, 30.5 GiB)
  • i3.2xlarge (8 vCPU, 61 GiB)
  • i3.4xlarge (16 vCPU, 122 GiB)
  • i3.8xlarge (32 vCPU, 244 GiB)
  • i3.16xlarge (64 vCPU, 488 GiB)
  • i3.metal (72† vCPU, 512 GiB)
  • i3en.xlarge (4 vCPU, 32 GiB)
  • i3en.2xlarge (8 vCPU, 64 GiB)
  • i3en.3xlarge (12 vCPU, 96 GiB)
  • i3en.6xlarge (24 vCPU, 192 GiB)
  • i3en.12xlarge (48 vCPU, 384 GiB)
  • i3en.24xlarge (96 vCPU, 768 GiB)
  • i3en.metal (96 vCPU, 768 GiB)
  • i4i.xlarge (4 vCPU, 32 GiB)
  • i4i.2xlarge (8 vCPU, 64 GiB)
  • i4i.4xlarge (16 vCPU, 128 GiB)
  • i4i.8xlarge (32 vCPU, 256 GiB)
  • i4i.12xlarge (48 vCPU, 384 GiB)
  • i4i.16xlarge (64 vCPU, 512 GiB)
  • i4i.24xlarge (96 vCPU, 768 GiB)
  • i4i.32xlarge (128 vCPU, 1,024 GiB)
  • i4i.metal (128 vCPU, 1,024 GiB)
  • i7i.xlarge (4 vCPU, 32 GiB)
  • i7i.2xlarge (8 vCPU, 64 GiB)
  • i7i.4xlarge (16 vCPU, 128 GiB)
  • i7i.8xlarge (32 vCPU, 256 GiB)
  • i7i.12xlarge (48 vCPU, 384 GiB)
  • i7i.16xlarge (64 vCPU, 512 GiB)
  • i7i.24xlarge (96 vCPU, 768 GiB)
  • i7i.48xlarge (192 vCPU, 1,536 GiB)
  • i7i.metal-24xl (96 vCPU, 768 GiB)
  • i7i.metal-48xl (192 vCPU, 1,536 GiB)
  • i7ie.xlarge (4 vCPU, 32 GiB)
  • i7ie.2xlarge (8 vCPU, 64 GiB)
  • i7ie.3xlarge (12 vCPU, 96 GiB)
  • i7ie.6xlarge (24 vCPU, 192 GiB)
  • i7ie.12xlarge (48 vCPU, 384 GiB)
  • i7ie.18xlarge (72 vCPU, 576 GiB)
  • i7ie.24xlarge (96 vCPU, 768 GiB)
  • i7ie.48xlarge (192 vCPU, 1,536 GiB)
  • i7ie.metal-24xl (96 vCPU, 768 GiB)
  • i7ie.metal-48xl (192 vCPU, 1,536 GiB)
  • m5ad.xlarge (4 vCPU, 16 GiB)
  • m5ad.2xlarge (8 vCPU, 32 GiB)
  • m5ad.4xlarge (16 vCPU, 64 GiB)
  • m5ad.8xlarge (32 vCPU, 128 GiB)
  • m5ad.12xlarge (48 vCPU, 192 GiB)
  • m5ad.16xlarge (64 vCPU, 256 GiB)
  • m5ad.24xlarge (96 vCPU, 384 GiB)
  • m5d.xlarge (4 vCPU, 16 GiB)
  • m5d.2xlarge (8 vCPU, 32 GiB)
  • m5d.4xlarge (16 vCPU, 64 GiB)
  • m5d.8xlarge (32 vCPU, 28 GiB)
  • m5d.12xlarge (48 vCPU, 192 GiB)
  • m5d.16xlarge (64 vCPU, 256 GiB)
  • m5d.24xlarge (96 vCPU, 384 GiB)

† This instance type offers 72 logical processors on 36 physical cores.

Note

Virtual instance types initialize faster than ".metal" instance types.

Example 6.9. High memory

  • u-3tb1.56xlarge (224 vCPU, 3,072 GiB)
  • u-6tb1.56xlarge (224 vCPU, 6,144 GiB)
  • u-6tb1.112xlarge (448 vCPU, 6,144 GiB)
  • u-6tb1.metal (448 vCPU, 6,144 GiB)
  • u-9tb1.112xlarge (448 vCPU, 9,216 GiB)
  • u-9tb1.metal (448 vCPU, 9,216 GiB)
  • u-12tb1.112xlarge (448 vCPU, 12,288 GiB)
  • u-12tb1.metal (448 vCPU, 12,288 GiB)
  • u-18tb1.metal (448 vCPU, 18,432 GiB)
  • u-24tb1.metal (448 vCPU, 24,576 GiB)
  • u-24tb1.112xlarge (448 vCPU, 24,576 GiB)

Example 6.10. Network Optimized

  • c5n.xlarge (4 vCPU, 10.5 GiB)
  • c5n.2xlarge (8 vCPU, 21 GiB)
  • c5n.4xlarge (16 vCPU, 42 GiB)
  • c5n.9xlarge (36 vCPU, 96 GiB)
  • c5n.18xlarge (72 vCPU, 192 GiB)
  • m5dn.xlarge (4 vCPU, 16 GiB)
  • m5dn.2xlarge (8 vCPU, 32 GiB)
  • m5dn.4xlarge (16 vCPU, 64 GiB)
  • m5dn.8xlarge (32 vCPU, 128 GiB)
  • m5dn.12xlarge (48 vCPU, 192 GiB)
  • m5dn.16xlarge (64 vCPU, 256 GiB)
  • m5dn.24xlarge (96 vCPU, 384 GiB)
  • m5n.12xlarge (48 vCPU, 192 GiB)
  • m5n.16xlarge (64 vCPU, 256 GiB)
  • m5n.24xlarge (96 vCPU, 384 GiB)
  • m5n.xlarge (4 vCPU, 16 GiB)
  • m5n.2xlarge (8 vCPU, 32 GiB)
  • m5n.4xlarge (16 vCPU, 64 GiB)
  • m5n.8xlarge (32 vCPU, 128 GiB)

6.4.2. AWS Arm-based Graviton instance types
Copy link

In addition to x86-based architecture, ROSA with HCP offers the following Arm-based Graviton worker node instance types and sizes:

Note

Graviton instance types are only available for new clusters created after 24 July, 2024.

Example 6.11. General purpose

  • a1.xlarge (2 vCPU, 4 GiB)
  • a1.2xlarge (4 vCPU, 8 GiB)
  • a1.4xlarge (8 vCPU, 16 GiB)
  • a1.metal (16 vCPU, 32 GiB)
  • m6g.xlarge (2 vCPU, 8 GiB)
  • m6g.2xlarge (4 vCPU, 16 GiB)
  • m6g.4xlarge (8 vCPU, 32 GiB)
  • m6g.8xlarge (32 vCPU, 128 GiB)
  • m6g.12xlarge (48 vCPU, 192 GiB)
  • m6g.16xlarge (64 vCPU, 256 GiB)
  • m6g.metal (64 vCPU, 256 GiB)
  • m6gd.xlarge (2 vCPU, 8 GiB)
  • m6gd.2xlarge (4 vCPU, 16 GiB)
  • m6gd.4xlarge (8 vCPU, 32 GiB)
  • m6gd.8xlarge (32 vCPU, 128 GiB)
  • m6gd.12xlarge (48 vCPU, 192 GiB)
  • m6gd.16xlarge (64 vCPU, 256 GiB)
  • m6gd.metal (64 vCPU, 256 GiB)
  • m7g.xlarge (2 vCPU, 8 GiB)
  • m7g.2xlarge (4 vCPU, 16 GiB)
  • m7g.4xlarge (8 vCPU, 32 GiB)
  • m7g.8xlarge (32 vCPU, 128 GiB)
  • m7g.12xlarge (48 vCPU, 192 GiB)
  • m7g.16xlarge (64 vCPU, 256 GiB)
  • m7g.metal (64 vCPU, 256 GiB)
  • m7gd.2xlarge (4 vCPU, 16 GiB)
  • m7gd.4xlarge (8 vCPU, 32 GiB)
  • m7gd.8xlarge (32 vCPU, 128 GiB)
  • m7gd.12xlarge (48 vCPU, 192 GiB)
  • m7gd.16xlarge (64 vCPU, 256 GiB)
  • m7gd.xlarge (2 vCPU, 8 GiB)
  • m7gd.metal (64 vCPU, 256 GiB)
  • m8g.xlarge (4 vCPU, 16 GiB)
  • m8g.2xlarge (8 vCPU, 32 GiB)
  • m8g.4xlarge (16 vCPU, 64 GiB)
  • m8g.8xlarge (32 vCPU, 128 GiB)
  • m8g.12xlarge (48 vCPU, 192 GiB)
  • m8g.16xlarge (64 vCPU, 256 GiB)
  • m8g.24xlarge (96 vCPU, 384 GiB)
  • m8g.48xlarge (192 vCPU, 768 GiB)
  • m8g.metal-24xl (96 vCPU, 384 GiB)
  • m8g.metal-48xl (192 vCPU, 768 GiB)

Example 6.12. Burstable general purpose

  • t4g.xlarge (4 vCPU, 16 GiB)
  • t4g.2xlarge (8 vCPU, 32 GiB)

Example 6.13. Memory intensive

  • x2gd.xlarge (2 vCPU, 64 GiB)
  • x2gd.2xlarge (4 vCPU, 128 GiB)
  • x2gd.4xlarge (8 vCPU, 256 GiB)
  • x2gd.8xlarge (16 vCPU, 512 GiB)
  • x2gd.12xlarge (32 vCPU, 768 GiB)
  • x2gd.16xlarge (64 vCPU, 1,024 GiB)
  • x2gd.metal (64 vCPU, 1,024 GiB)
  • x8g.xlarge (4 vCPU, 64 GiB)
  • x8g.2xlarge (8 vCPU, 128 GiB)
  • x8g.4xlarge (16 vCPU, 256 GiB)
  • x8g.8xlarge (32 vCPU, 512 GiB)
  • x8g.12xlarge (48 vCPU, 768 GiB)
  • x8g.16xlarge (64 vCPU, 1,024 GiB)
  • x8g.24xlarge (96 vCPU, 1,536 GiB)
  • x8g.48xlarge (192 vCPU, 3,072 GiB)
  • x8g.metal-24xl (96 vCPU, 1,536 GiB)
  • x8g.metal-48xl (192 vCPU, 3,072 GiB)

Example 6.14. Memory optimized

  • r6g.xlarge (4 vCPU, 32 GiB)
  • r6g.2xlarge (8 vCPU, 64 GiB)
  • r6g.4xlarge (16 vCPU, 128 GiB)
  • r6g.8xlarge (32 vCPU, 256 GiB)
  • r6g.12xlarge (48 vCPU, 384 GiB)
  • r6g.16xlarge (64 vCPU, 512 GiB)
  • r6g.metal (64 vCPU, 512 GiB)
  • r6gd.xlarge (4 vCPU, 32 GiB)
  • r6gd.2xlarge (8 vCPU, 64 GiB)
  • r6gd.4xlarge (16 vCPU, 128 GiB)
  • r6gd.8xlarge (32 vCPU, 256 GiB)
  • r6gd.12xlarge (48 vCPU, 384 GiB)
  • r6gd.16xlarge (64 vCPU, 512 GiB)
  • r6gd.metal (64 vCPU, 512 GiB)
  • r7g.xlarge (4 vCPU, 32 GiB)
  • r7g.2xlarge (8 vCPU, 64 GiB)
  • r7g.4xlarge (16 vCPU, 128 GiB)
  • r7g.8xlarge (32 vCPU, 256 GiB)
  • r7g.12xlarge (48 vCPU, 384 GiB)
  • r7g.16xlarge (64 vCPU, 512 GiB)
  • r7g.metal (64 vCPU, 512 GiB)
  • r7gd.xlarge (4 vCPU, 32 GiB)
  • r7gd.2xlarge (8 vCPU, 64 GiB)
  • r7gd.4xlarge (16 vCPU, 128 GiB)
  • r7gd.8xlarge (32 vCPU, 256 GiB)
  • r7gd.12xlarge (48 vCPU, 384 GiB)
  • r7gd.16xlarge (64 vCPU, 512 GiB)
  • r7gd.metal (64 vCPU, 512 GiB)
  • r8g.xlarge (4 vCPU, 32 GiB)
  • r8g.2xlarge (8 vCPU, 64 GiB)
  • r8g.4xlarge (16 vCPU, 128 GiB)
  • r8g.8xlarge (32 vCPU, 256 GiB)
  • r8g.12xlarge (48 vCPU, 384 GiB)
  • r8g.16xlarge (64 vCPU, 512 GiB)
  • r8g.24xlarge (96 vCPU, 768 GiB)
  • r8g.48xlarge (192 vCPU, 1,536 GiB)
  • r8g.metal-24xl (96 vCPU, 768 GiB)
  • r8g.metal-48xl (192 vCPU, 1,536 GiB)

Example 6.15. Accelerated computing

  • g5g.xlarge (4 vCPU, 8 GiB)
  • g5g.2xlarge (8 vCPU, 16 GiB)
  • g5g.4xlarge (16 vCPU, 32 GiB)
  • g5g.8xlarge (32 vCPU, 64 GiB)
  • g5g.16xlarge (64 vCPU, 128 GiB)
  • g5g.metal (64 vCPU, 128 GiB)

Example 6.16. Compute optimized

  • c6g.xlarge (4 vCPU, 8 GiB)
  • c6g.2xlarge (8 vCPU, 16 GiB)
  • c6g.4xlarge (16 vCPU, 32 GiB)
  • c6g.8xlarge (32 vCPU, 64 GiB)
  • c6g.12xlarge (48 vCPU, 96 GiB)
  • c6g.16xlarge (64 vCPU, 128 GiB)
  • c6g.metal (64 vCPU, 128 GiB)
  • c6gd.xlarge (4 vCPU, 8 GiB)
  • c6gd.2xlarge (8 vCPU, 16 GiB)
  • c6gd.4xlarge (16 vCPU, 32 GiB)
  • c6gd.8xlarge (32 vCPU, 64 GiB)
  • c6gd.12xlarge (48 vCPU, 96 GiB)
  • c6gd.16xlarge (64 vCPU, 128 GiB)
  • c6gd.metal (64 vCPU, 128 GiB)
  • c6gn.xlarge (4 vCPU, 8 GiB)
  • c6gn.2xlarge (8 vCPU, 16 GiB)
  • c6gn.4xlarge (16 vCPU, 32 GiB)
  • c6gn.8xlarge (32 vCPU, 64 GiB)
  • c6gn.12xlarge (48 vCPU, 96 GiB)
  • c6gn.16xlarge (64 vCPU, 128 GiB)
  • c7g.xlarge (4 vCPU, 8 GiB)
  • c7g.2xlarge (4 vCPU, 8 GiB)
  • c7g.4xlarge (16 vCPU, 32 GiB)
  • c7g.8xlarge (32 vCPU, 64 GiB)
  • c7g.12xlarge (48 vCPU, 96 GiB)
  • c7g.16xlarge (64 vCPU, 128 GiB)
  • c7g.metal (64 vCPU, 128 GiB)
  • c7gd.xlarge (4 vCPU, 8 GiB)
  • c7gd.2xlarge (4 vCPU, 8 GiB)
  • c7gd.4xlarge (16 vCPU, 32 GiB)
  • c7gd.8xlarge (32 vCPU, 64 GiB)
  • c7gd.12xlarge (48 vCPU, 96 GiB)
  • c7gd.16xlarge (64 vCPU, 128 GiB)
  • c7gd.metal (64 vCPU, 128 GiB)
  • c7gn.xlarge (4 vCPU, 8 GiB)
  • c7gn.2xlarge (8 vCPU, 16 GiB)
  • c7gn.4xlarge (16 vCPU, 32 GiB)
  • c7gn.8xlarge (32 vCPU, 64 GiB)
  • c7gn.12xlarge (48 vCPU, 96 GiB)
  • c7gn.16xlarge (64 vCPU, 128 GiB)
  • c7gn.metal (64 vCPU, 128 GiB)
  • c8g.xlarge (4 vCPU, 8 GiB)
  • c8g.2xlarge (8 vCPU, 16 GiB)
  • c8g.4xlarge (16 vCPU, 32 GiB)
  • c8g.8xlarge (32 vCPU, 64 GiB)
  • c8g.12xlarge (48 vCPU, 96 GiB)
  • c8g.16xlarge (64 vCPU, 128 GiB)
  • c8g.24xlarge (96 vCPU, 192 GiB)
  • c8g.48xlarge (192 vCPU, 384 GiB)
  • c8g.metal-24xl (96 vCPU, 192 GiB)
  • c8g.metal-48xl (192 vCPU, 384 GiB)

Example 6.17. Storage optimized

  • i4g.xlarge (4 vCPU, 32 GiB)
  • i4g.2xlarge (8 vCPU, 64 GiB)
  • i4g.4xlarge (16 vCPU, 128 GiB)
  • i4g.8xlarge (32 vCPU, 256 GiB)
  • i4g.16xlarge (64 vCPU, 512 GiB)
  • is4gen.xlarge (4 vCPU, 16 GiB)
  • is4gen.2xlarge (8 vCPU, 32 GiB)
  • is4gen.4xlarge (16 vCPU, 64 GiB)
  • is4gen.8xlarge (32 vCPU, 128 GiB)
  • im4gn.xlarge (4 vCPU, 16 GiB)
  • im4gn.2xlarge (8 vCPU, 32 GiB)
  • im4gn.4xlarge (16 vCPU, 64 GiB)
  • im4gn.8xlarge (32 vCPU, 128 GiB)
  • im4gn.16xlarge (64 vCPU, 256 GiB)

Example 6.18. High performance computing (HPC)

  • hpc7g.4xlarge (16 vCPU, 128 GiB)
  • hpc7g.8xlarge (32 vCPU, 128 GiB)
  • hpc7g.16xlarge (64 vCPU, 128 GiB)

6.5. ROSA with HCP update life cycle
Copy link

6.5.1. Overview
Copy link

Red Hat provides a published product life cycle for Red Hat OpenShift Service on AWS in order for customers and partners to effectively plan, deploy, and support their applications running on the platform. Red Hat publishes this life cycle to provide as much transparency as possible and might make exceptions from these policies as conflicts arise.

Red Hat OpenShift Service on AWS is a managed deployment of Red Hat OpenShift and maintains an independent release schedule. More details about the managed offering can be found in the Red Hat OpenShift Service on AWS service definition. The availability of Security Advisories and Bug Fix Advisories for a specific version are dependent upon the Red Hat OpenShift Container Platform life cycle policy and subject to the Red Hat OpenShift Service on AWS maintenance schedule.

6.5.2. Definitions
Copy link

Expand
Table 6.2. Version reference
Version formatMajorMinorPatchMajor.minor.patch
 

x

y

z

x.y.z

Example

4

5

21

4.5.21

Major releases or X-releases

Referred to only as major releases or X-releases (X.y.z).

Examples

  • "Major release 5" → 5.y.z
  • "Major release 4" → 4.y.z
  • "Major release 3" → 3.y.z
Minor releases or Y-releases

Referred to only as minor releases or Y-releases (x.Y.z).

Examples

  • "Minor release 4" → 4.4.z
  • "Minor release 5" → 4.5.z
  • "Minor release 6" → 4.6.z
Patch releases or Z-releases

Referred to only as patch releases or Z-releases (x.y.Z).

Examples

  • "Patch release 14 of minor release 5" → 4.5.14
  • "Patch release 25 of minor release 5" → 4.5.25
  • "Patch release 26 of minor release 6" → 4.6.26

6.5.3. Major versions (X.y.z)
Copy link

Major versions of Red Hat OpenShift Service on AWS, for example version 4, are supported for one year following the release of a subsequent major version or the retirement of the product.

Example

  • If version 5 were made available on Red Hat OpenShift Service on AWS on January 1, version 4 would be allowed to continue running on managed clusters for 12 months, until December 31. After this time, clusters would need to be upgraded or migrated to version 5.

6.5.4. Minor versions (x.Y.z)
Copy link

Starting with the 4.8 OpenShift Container Platform minor version, Red Hat supports all minor versions for at least a 16 month period following general availability of the given minor version. Patch versions are not affected by the support period.

Customers are notified 60, 30, and 15 days before the end of the support period. Clusters must be upgraded to the latest patch version of the oldest supported minor version before the end of the support period, or Red Hat will automatically upgrade the control plane to the next supported minor version.

Example

  1. A customer’s cluster is currently running on 4.13.8. The 4.13 minor version became generally available on May 17, 2023.
  2. On July 19, August 16, and September 2, 2024, the customer is notified that their cluster will enter "Limited Support" status on September 17, 2024 if the cluster has not already been upgraded to a supported minor version.
  3. The cluster must be upgraded to 4.14 or later by September 17, 2024.
  4. If the upgrade has not been performed, the cluster’s control plane will be automatically upgraded to 4.14.26, and there will be no automatic upgrades to the cluster’s worker nodes.

6.5.5. Patch versions (x.y.Z)
Copy link

During the period in which a minor version is supported, Red Hat supports all OpenShift Container Platform patch versions unless otherwise specified.

For reasons of platform security and stability, a patch release may be deprecated, which would prevent installations of that release and trigger mandatory upgrades off that release.

Example

  1. 4.7.6 is found to contain a critical CVE.
  2. Any releases impacted by the CVE will be removed from the supported patch release list. In addition, any clusters running 4.7.6 will be scheduled for automatic upgrades within 48 hours.

6.5.6. Limited support status
Copy link

When a cluster transitions to a Limited Support status, Red Hat no longer proactively monitors the cluster, the SLA is no longer applicable, and credits requested against the SLA are denied. It does not mean that you no longer have product support. In some cases, the cluster can return to a fully-supported status if you remediate the violating factors. However, in other cases, you might have to delete and recreate the cluster.

A cluster might transition to a Limited Support status for many reasons, including the following scenarios:

If you remove or replace any native Red Hat OpenShift Service on AWS components or any other component that is installed and managed by Red Hat
If cluster administrator permissions were used, Red Hat is not responsible for any of your or your authorized users’ actions, including those that affect infrastructure services, service availability, or data loss. If Red Hat detects any such actions, the cluster might transition to a Limited Support status. Red Hat notifies you of the status change and you should either revert the action or create a support case to explore remediation steps that might require you to delete and recreate the cluster.

If you have questions about a specific action that might cause a cluster to transition to a Limited Support status or need further assistance, open a support ticket.

6.5.7. Supported versions exception policy
Copy link

Red Hat reserves the right to add or remove new or existing versions, or delay upcoming minor release versions, that have been identified to have one or more critical production impacting bugs or security issues without advance notice.

6.5.8. Installation policy
Copy link

While Red Hat recommends installation of the latest support release, Red Hat OpenShift Service on AWS supports installation of any supported release as covered by the preceding policy.

6.5.9. Deletion policy
Copy link

Red Hat reserves the right to delete Red Hat OpenShift Service on AWS clusters within 15 days if the service notifications requiring actions are not addressed. These actions include upgrading the cluster to a supported OpenShift version or resolving cluster health issues so that the service can auto-upgrade the cluster to a supported OpenShift version.

Red Hat OpenShift Service on AWS services will notify you when the cluster is unhealthy and when the OpenShift version is approaching EOL.

Important

Red Hat OpenShift Service on AWS clusters configured with delete protection enabled can still be deleted based on the deletion policy.

If a Red Hat OpenShift Service on AWS cluster is deleted, any applications or business hosted on the cluster will be impacted. Additionally, cloud resources may remain in the AWS account after cluster deletion, which will continue to incur costs.

6.5.10. Mandatory upgrades
Copy link

If a critical or important CVE, or other bug identified by Red Hat, significantly impacts the security or stability of the cluster, the customer must upgrade to the next supported patch release within two business days.

In extreme circumstances and based on Red Hat’s assessment of the CVE criticality to the environment, Red Hat will notify customers that they have two business days to schedule or manually update their cluster to the latest, secure patch release. In the case that an update is not performed after two business days, Red Hat will automatically update the cluster’s control plane to the latest, secure patch release to mitigate potential security breach(es) or instability. Red Hat might, at its own discretion, temporarily delay an automated update if requested by a customer through a support case.

6.5.11. Life cycle dates
Copy link

Expand
VersionGeneral availabilityEnd of life

4.19

Jun 16, 2025

Oct 21, 2026

4.18

Feb 26, 2025

Jun 21, 2026

4.17

Oct 14, 2024

Feb 14, 2026

4.16

Jul 2, 2024

Nov 2, 2025

4.15

Feb 27, 2024

Aug 27, 2025

6.6. SRE and service account access
Copy link

Red Hat site reliability engineering (SRE) access to Red Hat OpenShift Service on AWS (ROSA) clusters is outlined through identity and access management.

6.6.1. Identity and access management
Copy link

Most access by Red Hat SRE teams is done by using cluster Operators through automated configuration management.

Subprocessors

For a list of the available subprocessors, see the Red Hat Subprocessor List on the Red Hat Customer Portal.

6.6.2. SRE cluster access
Copy link

Red Hat SRE access to Red Hat OpenShift Service on AWS clusters is controlled through several layers of required authentication, all of which are managed by strict company policy. All authentication attempts to access a cluster and changes made within a cluster are recorded within audit logs, along with the specific account identity of the SRE responsible for those actions. These audit logs help ensure that all changes made by SREs to a customer’s cluster adhere to the strict policies and procedures that make up Red Hat’s managed services guidelines.

The information presented below is an overview of the process an SRE must perform to access a customer’s cluster.

  • Red Hat SRE requests a refreshed ID token from the Red Hat SSO (Cloud Services). This request is authenticated. The token is valid for fifteen minutes. After the token expires, you can refresh the token again and receive a new token. The ability to refresh to a new token is indefinite; however, the ability to refresh to a new token is revoked after 30 days of inactivity.
  • Red Hat SRE connects to the Red Hat VPN. The authentication to the VPN is completed by the Red Hat Corporate Identity and Access Management system (RH IAM). With RH IAM, SREs are multifactor and can be managed internally per organization by groups and existing onboarding and offboarding processes. After an SRE is authenticated and connected, the SRE can access the cloud services fleet management plane. Changes to the cloud services fleet management plane require many layers of approval and are maintained by strict company policy.
  • After authorization is complete, the SRE logs into the fleet management plane and receives a service account token that the fleet management plane created. The token is valid for 15 minutes. After the token is no longer valid, it is deleted.

  • With access granted to the fleet management plane, SRE uses various methods to access clusters, depending on network configuration.

    • Accessing a private or public cluster: Request is sent through a specific Network Load Balancer (NLB) by using an encrypted HTTP connection on port 6443.
    • Accessing a PrivateLink cluster: Request is sent to the Red Hat Transit Gateway, which then connects to a Red Hat VPC per region. The VPC that receives the request will be dependent on the target private cluster’s region. Within the VPC, there is a private subnet that contains the PrivateLink endpoint to the customer’s PrivateLink cluster.

6.6.3. Red Hat support access
Copy link

Members of the Red Hat Customer Experience and Engagement (CEE) team typically have read-only access to parts of the cluster. Specifically, CEE has limited access to the core and product namespaces and does not have access to the customer namespaces.

Expand
RoleCore namespaceLayered product namespaceCustomer namespaceAWS account*

OpenShift SRE - Normal operations [1]

Read: All

Write: Very

limited

Read: All

Write: None

Read: None

Write: None

Read: None

Write: None

OpenShift SRE - Elevated Access [2] (Gated by Approved Access)

Read: All

Write: All

Read: All

Write: All

Read: All

Write: All

Read: All

Write: All

CEE

Read: All

Write: None

Read: All

Write: None

Read: None

Write: None

Read: None

Write: None

Customer administrator

Read: None

Write: None

Read: None

Write: None

Read: All

Write: All

Read: All

Write: All

Customer user

Read: None

Write: None

Read: None

Write: None

Read: Limited [3]

Write: Limited [3]

Read: None

Write: None

Everybody else

Read: None

Write: None

Read: None

Write: None

Read: None

Write: None

Read: None

Write: None

  1. Limited to addressing common use cases such as failing deployments, upgrading a cluster, and replacing bad worker nodes.
  2. Elevated access gives SRE the access levels of a cluster-admin role and is gated by Approved Access. For more information, see "Default cluster roles" and "Approved Access".
  3. Limited to what is granted through RBAC by the Customer Administrator and namespaces created by the user.

6.6.4. Customer access
Copy link

Customer access is limited to namespaces created by the customer and permissions that are granted using RBAC by the Customer Administrator role. Access to the underlying infrastructure or product namespaces is generally not permitted without cluster-admin access. For more information about customer access and authentication, see the "Understanding Authentication" section of the documentation.

6.6.5. Access approval and review
Copy link

New Red Hat SRE user access requires management approval. Separated or transferred SRE accounts are removed as authorized users through an automated process. Additionally, the SRE performs periodic access review, including management sign-off of authorized user lists.

The access and identity authorization table includes responsibilities for managing authorized access to clusters, applications, and infrastructure resources. This includes tasks such as providing access control mechanisms, authentication, authorization, and managing access to resources.

Expand
ResourceService responsibilitiesCustomer responsibilities

Logging

Red Hat

  • Adhere to an industry standards-based tiered internal access process for platform audit logs.
  • Provide native OpenShift RBAC capabilities.
  • Configure OpenShift RBAC to control access to projects and by extension a project’s application logs.
  • For third-party or custom application logging solutions, the customer is responsible for access management.

Application networking

Red Hat

  • Provide native OpenShift RBAC and dedicated-admin capabilities.
  • Configure OpenShift dedicated-admin and RBAC to control access to route configuration as required.
  • Manage organization administrators for Red Hat to grant access to OpenShift Cluster Manager. The cluster manager is used to configure router options and provide service load balancer quota.

Cluster networking

Red Hat

  • Provide customer access controls through OpenShift Cluster Manager.
  • Provide native OpenShift RBAC and dedicated-admin capabilities.
  • Manage Red Hat organization membership of Red Hat accounts.
  • Manage organization administrators for Red Hat to grant access to OpenShift Cluster Manager.
  • Configure OpenShift dedicated-admin and RBAC to control access to route configuration as required.

Virtual networking management

Red Hat

  • Provide customer access controls through OpenShift Cluster Manager.
  • Manage optional user access to AWS components through OpenShift Cluster Manager.

Virtual storage management

Red Hat

  • Provide customer access controls through Red Hat OpenShift Cluster Manager.
  • Manage optional user access to AWS components through OpenShift Cluster Manager.
  • Create AWS IAM roles and attached policies necessary to enable ROSA service access.

Virtual compute management

Red Hat

  • Provide customer access controls through Red Hat OpenShift Cluster Manager.
  • Manage optional user access to AWS components through OpenShift Cluster Manager.
  • Create AWS IAM roles and attached policies necessary to enable ROSA service access.

AWS software (public AWS services)

AWS

Compute: Provide the Amazon EC2 service, used for ROSA control plane and worker nodes.

Storage: Provide Amazon EBS, used to allow ROSA to provision local node storage and persistent volume storage for the cluster.

Storage: Provide Amazon S3, used for the service’s built-in image registry.

Networking: Provide AWS Identity and Access Management (IAM), used by customers to control access to ROSA resources running on customer accounts.

  • Create AWS IAM roles and attached policies necessary to enable ROSA service access.
  • Use IAM tools to apply the appropriate permissions to AWS resources in the customer account.
  • To enable ROSA across your AWS organization, the customer is responsible for managing AWS Organizations administrators.
  • To enable ROSA across your AWS organization, the customer is responsible for distributing the ROSA entitlement grant using AWS License Manager.

Hardware and AWS global infrastructure

AWS

  • For information about physical access controls for AWS data centers, see Our Controls on the AWS Cloud Security page.
  • Customer is not responsible for AWS global infrastructure.

When you install a Red Hat OpenShift Service on AWS cluster, cluster-specific Operator AWS Identity and Access Management (IAM) roles are created. These IAM roles permit the ROSA cluster Operators to run core OpenShift functionality.

Cluster Operators use service accounts to assume IAM roles. When a service account assumes an IAM role, temporary AWS STS credentials are provided for the service account to use in the cluster Operator’s pod. If the assumed role has the necessary AWS privileges, the service account can run AWS SDK operations in the pod.

The following diagram illustrates the workflow for assuming AWS IAM roles in SRE owned projects:

Figure 6.1. Workflow for assuming AWS IAM roles in SRE owned projects

Workflow for assuming AWS IAM roles in Red Hat SRE owned projects
View larger image
Workflow for assuming AWS IAM roles in Red Hat SRE owned projects

The workflow has the following stages:

  1. Within each project that a cluster Operator runs, the Operator’s deployment spec has a volume mount for the projected service account token, and a secret containing AWS credential configuration for the pod. The token is audience-bound and time-bound. Every hour, ROSA generates a new token, and the AWS SDK reads the mounted secret containing the AWS credential configuration. This configuration has a path to the mounted token and the AWS IAM Role ARN. The secret’s credential configuration includes the following:

    • An $AWS_ARN_ROLE variable that has the ARN for the IAM role that has the permissions required to run AWS SDK operations.
    • An $AWS_WEB_IDENTITY_TOKEN_FILE variable that has the full path in the pod to the OpenID Connect (OIDC) token for the service account. The full path is /var/run/secrets/openshift/serviceaccount/token.
  2. When a cluster Operator needs to assume an AWS IAM role to access an AWS service (such as EC2), the AWS SDK client code running on the Operator invokes the AssumeRoleWithWebIdentity API call.

  3. The OIDC token is passed from the pod to the OIDC provider. The provider authenticates the service account identity if the following requirements are met:

    • The identity signature is valid and signed by the private key.

    • The sts.amazonaws.com audience is listed in the OIDC token and matches the audience configured in the OIDC provider.

      Note

      In ROSA with STS clusters, the OIDC provider is created during install and set as the service account issuer by default. The sts.amazonaws.com audience is set by default in the OIDC provider.

    • The OIDC token has not expired.
    • The issuer value in the token has the URL for the OIDC provider.
  4. If the project and service account are in the scope of the trust policy for the IAM role that is being assumed, then authorization succeeds.
  5. After successful authentication and authorization, temporary AWS STS credentials in the form of an AWS access token, secret key, and session token are passed to the pod for use by the service account. By using the credentials, the service account is temporarily granted the AWS permissions enabled in the IAM role.
  6. When the cluster Operator runs, the Operator that is using the AWS SDK in the pod consumes the secret that has the path to the projected service account and AWS IAM Role ARN to authenticate against the OIDC provider. The OIDC provider returns temporary STS credentials for authentication against the AWS API.

This document details the Red Hat, Amazon Web Services (AWS), and customer security responsibilities for the managed Red Hat OpenShift Service on AWS (ROSA).

Acronyms and terms

  • AWS - Amazon Web Services
  • CEE - Customer Experience and Engagement (Red Hat Support)
  • CI/CD - Continuous Integration / Continuous Delivery
  • CVE - Common Vulnerabilities and Exposures
  • PVs - Persistent Volumes
  • ROSA - Red Hat OpenShift Service on AWS
  • SRE - Red Hat Site Reliability Engineering
  • VPC - Virtual Private Cloud

6.7.1. Security and regulation compliance
Copy link

Security and regulation compliance includes tasks such as the implementation of security controls and compliance certification.

6.7.1.1. Data classification
Copy link

Red Hat defines and follows a data classification standard to determine the sensitivity of data and highlight inherent risk to the confidentiality and integrity of that data while it is collected, used, transmitted, stored, and processed. Customer-owned data is classified at the highest level of sensitivity and handling requirements.

6.7.1.2. Data management
Copy link

Red Hat OpenShift Service on AWS (ROSA) uses AWS Key Management Service (KMS) to help securely manage keys for encrypted data. These keys are used for control plane, infrastructure, and worker data volumes that are encrypted by default. Persistent volumes (PVs) for customer applications also use AWS KMS for key management.

When a customer deletes their ROSA cluster, all cluster data is permanently deleted, including control plane data volumes and customer application data volumes, such as persistent volumes (PV).

6.7.1.3. Vulnerability management
Copy link

Red Hat performs periodic vulnerability scanning of ROSA using industry standard tools. Identified vulnerabilities are tracked to their remediation according to timelines based on severity. Vulnerability scanning and remediation activities are documented for verification by third-party assessors in the course of compliance certification audits.

6.7.1.4. Network security
Copy link
6.7.1.4.1. Firewall and DDoS protection
Copy link

Each ROSA cluster is protected by a secure network configuration using firewall rules for AWS Security Groups. ROSA customers are also protected against DDoS attacks with AWS Shield Standard.

Customers can optionally configure their ROSA cluster endpoints, such as web console, API, and application router, to be made private so that the cluster control plane and applications are not accessible from the Internet. Red Hat SRE still requires Internet-accessible endpoints that are protected with IP allow-lists.

AWS customers can configure a private network connection to their ROSA cluster through technologies such as AWS VPC peering, AWS VPN, or AWS Direct Connect.

6.7.1.4.3. Cluster network access controls
Copy link

Fine-grained network access control rules can be configured by customers, on a per-project basis, using NetworkPolicy objects and the OVN-Kubernetes CNI.

6.7.1.5. Penetration testing
Copy link

Red Hat performs periodic penetration tests against ROSA. Tests are performed by an independent internal team by using industry standard tools and best practices.

Any issues that may be discovered are prioritized based on severity. Any issues found belonging to open source projects are shared with the community for resolution.

6.7.1.6. Compliance
Copy link

Red Hat OpenShift Service on AWS follows common industry best practices for security and controls. The certifications are outlined in the following table.

Expand
Table 6.3. Security and control certifications for Red Hat OpenShift Service on AWS
ComplianceRed Hat OpenShift Service on AWS

HIPAA Qualified[1]

Yes

ISO 27001

Yes

ISO 27017

Yes

ISO 27018

Yes

PCI DSS 4.0

Yes

SOC 1 Type 2

Yes

SOC 2 Type 2

Yes

SOC 3

Yes

FedRAMP High[2]

No

  1. For more information about Red Hat’s HIPAA Qualified ROSA offerings, see the HIPAA Overview.
  2. For more information about ROSA on GovCloud, see the FedRAMP Marketplace ROSA Agency and ROSA JAB listings.

Chapter 7. Required IAM resources
Copy link

Red Hat OpenShift Service on AWS uses the AWS Security Token Service (STS) to provide temporary, limited-permission credentials for your cluster. This means that before you deploy your cluster, you must create the following AWS Identity Access Management (IAM) resources:

  • Specific account-wide IAM roles and policies that provide the STS permissions required for ROSA support, installation, and compute functionality. This includes account-wide Operator policies.
  • Cluster-specific Operator IAM roles that permit the ROSA cluster Operators to carry out core OpenShift functionality.
  • An OpenID Connect (OIDC) provider that the cluster Operators use to authenticate.

  • If you deploy and manage your cluster using OpenShift Cluster Manager, you must create the following additional resources:

    • An OpenShift Cluster Manager IAM role to complete the installation on your cluster.
    • A user role without any permissions to verify your AWS account identity.

This document provides reference information about the IAM resources that you must deploy when you create a ROSA with HCP cluster. It also includes the aws CLI commands that are generated when you use manual mode with the rosa create command.

If you create ROSA clusters by using OpenShift Cluster Manager, you must have the following AWS IAM roles linked to your AWS account to create and manage the clusters.

These AWS IAM roles are as follows:

  • The ROSA user role (user-role) is an AWS role used by Red Hat to verify the customer’s AWS identity. This role has no additional permissions, and the role has a trust relationship with the Red Hat installer account.

  • An ocm-role resource grants the required permissions for installation of ROSA clusters in OpenShift Cluster Manager. You can apply basic or administrative permissions to the ocm-role resource. If you create an administrative ocm-role resource, OpenShift Cluster Manager can create the needed AWS Operator roles and OpenID Connect (OIDC) provider. This IAM role also creates a trust relationship with the Red Hat installer account as well.

    Note

    The ocm-role IAM resource refers to the combination of the IAM role and the necessary policies created with it.

You must create this user role as well as an administrative ocm-role resource, if you want to use the auto mode in OpenShift Cluster Manager to create your Operator role policies and OIDC provider.

Creating ROSA clusters in OpenShift Cluster Manager require an ocm-role IAM role. The basic ocm-role IAM role permissions let you to perform cluster maintenance within OpenShift Cluster Manager. To automatically create the operator roles and OpenID Connect (OIDC) provider, you must add the --admin option to the rosa create command. This command creates an ocm-role resource with additional permissions needed for administrative tasks.

Note

This elevated IAM role allows OpenShift Cluster Manager to automatically create the cluster-specific Operator roles and OIDC provider during cluster creation. For more information about this automatic role and policy creation, see the "Methods of account-wide role creation" link in Additional resources.

7.1.1.1. Understanding the user role
Copy link

In addition to an ocm-role IAM role, you must create a user role so that Red Hat OpenShift Service on AWS can verify your AWS identity. This role has no permissions, and it is only used to create a trust relationship between the installer account and your ocm-role resources.

The following tables show the associated basic and administrative permissions for the ocm-role resource.

Expand
Table 7.1. Associated permissions for the basic ocm-role resource
ResourceDescription

iam:GetOpenIDConnectProvider

This permission allows the basic role to retrieve information about the specified OpenID Connect (OIDC) provider.

iam:GetRole

This permission allows the basic role to retrieve any information for a specified role. Some of the data returned include the role’s path, GUID, ARN, and the role’s trust policy that grants permission to assume the role.

iam:ListRoles

This permission allows the basic role to list the roles within a path prefix.

iam:ListRoleTags

This permission allows the basic role to list the tags on a specified role.

ec2:DescribeRegions

This permission allows the basic role to return information about all of the enabled regions on your account.

ec2:DescribeRouteTables

This permission allows the basic role to return information about all of your route tables.

ec2:DescribeSubnets

This permission allows the basic role to return information about all of your subnets.

ec2:DescribeVpcs

This permission allows the basic role to return information about all of your virtual private clouds (VPCs).

sts:AssumeRole

This permission allows the basic role to retrieve temporary security credentials to access AWS resources that are beyond its normal permissions.

sts:AssumeRoleWithWebIdentity

This permission allows the basic role to retrieve temporary security credentials for users authenticated their account with a web identity provider.

Expand
Table 7.2. Additional permissions for the admin ocm-role resource
ResourceDescription

iam:AttachRolePolicy

This permission allows the admin role to attach a specified policy to the desired IAM role.

iam:CreateOpenIDConnectProvider

This permission creates a resource that describes an identity provider, which supports OpenID Connect (OIDC). When you create an OIDC provider with this permission, this provider establishes a trust relationship between the provider and AWS.

iam:CreateRole

This permission allows the admin role to create a role for your AWS account.

iam:ListPolicies

This permission allows the admin role to list any policies associated with your AWS account.

iam:ListPolicyTags

This permission allows the admin role to list any tags on a designated policy.

iam:PutRolePermissionsBoundary

This permission allows the admin role to change the permissions boundary for a user based on a specified policy.

iam:TagRole

This permission allows the admin role to add tags to an IAM role.

Creating an ocm-role IAM role

You create your ocm-role IAM roles by using the command-line interface (CLI).

Prerequisites

  • You have an AWS account.
  • You have Red Hat Organization Administrator privileges in the OpenShift Cluster Manager organization.
  • You have the permissions required to install AWS account-wide roles.
  • You have installed and configured the latest ROSA CLI, rosa, on your installation host.

Procedure

  • To create an ocm-role IAM role with basic privileges, run the following command: 

    $ rosa create ocm-role
    Copy to Clipboard Toggle word wrap

  • To create an ocm-role IAM role with admin privileges, run the following command: 

    $ rosa create ocm-role --admin
    Copy to Clipboard Toggle word wrap

    This command allows you to create the role by specifying specific attributes. The following example output shows the "auto mode" selected, which lets the ROSA CLI (rosa) create your Operator roles and policies. See "Methods of account-wide role creation" for more information.

Example output

I: Creating ocm role
? Role prefix: ManagedOpenShift
1 

? Enable admin capabilities for the OCM role (optional): No
2 

? Permissions boundary ARN (optional):
3 

? Role Path (optional):
4 

? Role creation mode: auto
5 

I: Creating role using 'arn:aws:iam::<ARN>:user/<UserName>'
? Create the 'ManagedOpenShift-OCM-Role-182' role? Yes
6 

I: Created role 'ManagedOpenShift-OCM-Role-182' with ARN  'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182'
I: Linking OCM role
? OCM Role ARN: arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182
7 

? Link the 'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182' role with organization '<AWS ARN>'? Yes
8 

I: Successfully linked role-arn 'arn:aws:iam::<ARN>:role/ManagedOpenShift-OCM-Role-182' with organization account '<AWS ARN>'
Copy to Clipboard Toggle word wrap

1
A prefix value for all of the created AWS resources. In this example, ManagedOpenShift prepends all of the AWS resources.
2
Choose if you want this role to have the additional admin permissions.
Note

You do not see this prompt if you used the --admin option.

3
The Amazon Resource Name (ARN) of the policy to set permission boundaries.
4
Specify an IAM path for the user name.
5
Choose the method to create your AWS roles. Using auto, the ROSA CLI generates and links the roles and policies. In the auto mode, you receive some different prompts to create the AWS roles.
6
The auto method asks if you want to create a specific ocm-role using your prefix.
7
Confirm that you want to associate your IAM role with your OpenShift Cluster Manager.
8
Links the created role with your AWS organization.

AWS IAM roles link to your AWS account to create and manage the clusters.

7.2. Account-wide IAM role and policy reference
Copy link

This section provides details about the account-wide IAM roles and policies that are required for ROSA deployments that use STS, including the Operator policies. It also includes the JSON files that define the policies.

The account-wide roles and policies are specific to an Red Hat OpenShift Service on AWS minor release version, for example Red Hat OpenShift Service on AWS 4, and are compatible with earlier versions. You can minimize the required STS resources by reusing the account-wide roles and policies for multiple clusters of the same minor version, regardless of their patch version.

7.2.1. Methods of account-wide role creation
Copy link

You can create account-wide roles by using the Red Hat OpenShift Service on AWS (ROSA) CLI, rosa, or the OpenShift Cluster Manager guided installation. You can create the roles manually or by using an automatic process that uses predefined names for these roles and policies.

Manual ocm-role resource creation

You can use the manual creation method if you have the necessary CLI access to create these roles on your system. You can run this option in your desired CLI tool or from OpenShift Cluster Manager. After you start the manual creation process, the CLI presents a series of commands for you to run that create the roles and link them to the needed policies.

Automatic ocm-role resource creation

If you created an ocm-role resource with administrative permissions, you can use the automatic creation method from OpenShift Cluster Manager. The ROSA CLI does not require that you have this admin ocm-role IAM resource to automatically create these roles and policies. Selecting this method creates the roles and policies that uses the default names.

If you use the ROSA guided installation on OpenShift Cluster Manager, you must have created an ocm-role resource with administrative permissions in the first step of the guided cluster installation. Without this role, you cannot use the automatic Operator role and policy creation option, but you can still create the cluster and its roles and policies with the manual process.

Expand
Table 7.3. ROSA Manage Subscription policy and policy file
ResourceDescription

ROSAManageSubscription

This policy streamlines permission setup by packaging necessary access rights, giving entities appropriate control over the ROSA subscription while preventing excessive permissions.

Expand
Table 7.4. ROSA installer role, policy, and policy files
ResourceDescription

HCP-ROSA-Installer-Role

An IAM role used by the ROSA installer.

ROSAInstallerPolicy

An IAM policy that provides the ROSA installer with the permissions required to complete cluster installation tasks.

HCP-ROSA-Installer-Role trust policy

Grants the Red  Hat installer temporary permission to act within your AWS account for the sole purpose of setting up a Red Hat OpenShift Service on AWS cluster.

Example 7.1. sts_hcp_installer_permission_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ReadPermissions",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeInstances",
                "ec2:DescribeInstanceTypes",
                "ec2:DescribeRegions",
                "ec2:DescribeReservedInstancesOfferings",
                "ec2:DescribeRouteTables",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeSecurityGroupRules",
                "ec2:DescribeSubnets",
                "ec2:DescribeVpcAttribute",
                "ec2:DescribeVpcs",
                "ec2:DescribeInstanceTypeOfferings",
                "ec2:DescribeCapacityReservations",
                "elasticloadbalancing:DescribeAccountLimits",
                "elasticloadbalancing:DescribeLoadBalancers",
                "iam:GetOpenIDConnectProvider",
                "iam:GetRole",
                "route53:GetHostedZone",
                "route53:ListHostedZones",
                "route53:ListHostedZonesByName",
                "route53:ListResourceRecordSets",
                "route53:GetAccountLimit",
                "servicequotas:GetServiceQuota"
            ],
            "Resource": "*"
        },
        {
            "Sid": "PassRoleToEC2",
            "Action": [
                "iam:PassRole"
            ],
            "Resource": [
                "arn:*:iam::*:role/*-ROSA-Worker-Role"
            ],
            "Effect": "Allow",
            "Condition": {
                "StringEquals": {
                    "iam:PassedToService": [
                        "ec2.amazonaws.com"
                    ]
                }
            }
        },
        {
            "Sid": "ManageInstanceProfiles",
            "Effect": "Allow",
            "Action": [
                "iam:AddRoleToInstanceProfile",
                "iam:RemoveRoleFromInstanceProfile",
                "iam:DeleteInstanceProfile",
                "iam:GetInstanceProfile"
            ],
            "Resource": [
                "arn:aws:iam::*:instance-profile/rosa-service-managed-*"
            ]
        },
        {
            "Sid": "CreateInstanceProfiles",
            "Effect": "Allow",
            "Action": [
                "iam:CreateInstanceProfile",
                "iam:TagInstanceProfile"
            ],
            "Resource": [
                "arn:aws:iam::*:instance-profile/rosa-service-managed-*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "GetSecretValue",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue"
            ],
            "Resource": [
                "*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "Route53ManageRecords",
            "Effect": "Allow",
            "Action": [
                "route53:ChangeResourceRecordSets"
            ],
            "Resource": "*",
            "Condition": {
                "ForAllValues:StringLike": {
                    "route53:ChangeResourceRecordSetsNormalizedRecordNames": [
                        "*.openshiftapps.com",
                        "*.devshift.org",
                        "*.hypershift.local",
                        "*.openshiftusgov.com",
                        "*.devshiftusgov.com"
                    ]
                }
            }
        },
        {
            "Sid": "Route53Manage",
            "Effect": "Allow",
            "Action": [
                "route53:ChangeTagsForResource",
                "route53:CreateHostedZone",
                "route53:DeleteHostedZone"
            ],
            "Resource": "*"
        },
        {
            "Sid": "CreateTags",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:volume/*"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "RunInstances"
                    ]
                }
            }
        },
        {
            "Sid": "RunInstancesNoCondition",
            "Effect": "Allow",
            "Action": "ec2:RunInstances",
            "Resource": [
                "arn:aws:ec2:*:*:subnet/*",
                "arn:aws:ec2:*:*:network-interface/*",
                "arn:aws:ec2:*:*:security-group/*",
                "arn:aws:ec2:*:*:snapshot/*"
            ]
        },
        {
            "Sid": "RunInstancesRestrictedRequestTag",
            "Effect": "Allow",
            "Action": "ec2:RunInstances",
            "Resource": [
                "arn:aws:ec2:*:*:instance/*",
                "arn:aws:ec2:*:*:volume/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "RunInstancesRedHatOwnedAMIs",
            "Effect": "Allow",
            "Action": [
                "ec2:RunInstances"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:image/*"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:Owner": [
                        "531415883065",
                        "251351625822",
                        "210686502322"
                    ]
                }
            }
        },
        {
            "Sid": "ManageInstancesRestrictedResourceTag",
            "Effect": "Allow",
            "Action": [
                "ec2:TerminateInstances",
                "ec2:GetConsoleOutput"
            ],
            "Resource": "arn:aws:ec2:*:*:instance/*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "CreateGrantRestrictedResourceTag",
            "Effect": "Allow",
            "Action": [
                "kms:CreateGrant"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat": "true"
                },
                "StringLike": {
                    "kms:ViaService": "ec2.*.amazonaws.com"
                },
                "Bool": {
                  "kms:GrantIsForAWSResource": true
                }
            }
        },
        {
            "Sid": "ManagedKMSRestrictedResourceTag",
            "Effect": "Allow",
            "Action": [
                "kms:DescribeKey",
                "kms:GenerateDataKeyWithoutPlaintext"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat": "true"
                }
            }
        },
        {
            "Sid": "CreateSecurityGroups",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "DeleteSecurityGroup",
            "Effect": "Allow",
            "Action": [
                "ec2:DeleteSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "SecurityGroupIngressEgress",
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupEgress"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "CreateSecurityGroupsVPCNoCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc/*"
            ]
        },
        {
            "Sid": "CreateTagsRestrictedActions",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "CreateSecurityGroup"
                    ]
                }
            }
        },
        {
            "Sid": "CreateTagsK8sSubnet",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:subnet/*"
            ],
            "Condition": {
                "ForAllValues:StringLike": {
                    "aws:TagKeys": [
                        "kubernetes.io/cluster/*"
                    ]
                }
            }
        },
        {
            "Sid": "DeleteTagsK8sSubnet",
            "Effect": "Allow",
            "Action": [
                "ec2:DeleteTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:subnet/*"
            ],
            "Condition": {
                "Null": {
                    "aws:TagKeys": "false"
                },
                "ForAllValues:StringLike": {
                    "aws:TagKeys": [
                        "kubernetes.io/cluster/*"
                    ]
                }
            }
        },
        {
            "Sid": "ListPoliciesAttachedToRoles",
            "Effect": "Allow",
            "Action": [
                "iam:ListAttachedRolePolicies",
                "iam:ListRolePolicies"
            ],
            "Resource": "arn:aws:iam::*:role/*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        }
    ]
}
Copy to Clipboard Toggle word wrap

Example 7.2. sts_hcp_installer_trust_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::710019948333:role/RH-Managed-OpenShift-Installer"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.5. ROSA worker node role, policy, and policy files
ResourceDescription

HCP-ROSA-Worker-Role

An IAM role used by the compute instances.

ROSAWorkerInstancePolicy

An IAM policy that provides the ROSA compute instances with the permissions required to manage their components.

HCP-ROSA-Worker-Role trust policy

Allows essential software on your worker nodes to securely connect and talk to the cluster’s control plane, which is managed remotely by Red Hat.

Example 7.3. sts_hcp_worker_instance_permission_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "EC2DescribeInstancesRegions",
            "Effect": "Allow",
            "Action": ["ec2:DescribeInstances", "ec2:DescribeRegions"],
            "Resource": "*"
        },
        {
            "Sid": "ECRGetAuthorizationToken",
            "Effect": "Allow",
            "Action": ["ecr:GetAuthorizationToken"],
            "Resource": "*"
        },
        {
            "Sid": "ECRReadOnlyAccessRedHatManaged",
            "Effect": "Allow",
            "Action": [
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:GetRepositoryPolicy",
                "ecr:DescribeRepositories",
                "ecr:ListImages",
                "ecr:DescribeImages",
                "ecr:BatchGetImage",
                "ecr:ListTagsForResource"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        }
    ]
}
Copy to Clipboard Toggle word wrap

Example 7.4. sts_hcp_worker_instance_trust_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "ec2.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.6. ROSA support role, policy, and policy files
ResourceDescription

HCP-ROSA-Support-Role

An IAM role used by the Red Hat Site Reliability Engineering (SRE) support team.

ROSASRESupportPolicy

An IAM policy that provides the Red Hat SRE support team with the permissions required to support ROSA clusters.

HCP-ROSA-Support-Role trust policy

Provides a secure mechanism for authorized Red Hat Site Reliability Engineers (SREs) to perform diagnostic and support functions on the cluster.

Example 7.5. sts_hcp_support_permission_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadPermissions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeRegions",
        "sts:DecodeAuthorizationMessage"
      ],
      "Resource": "*"
    },
    {
      "Sid": "Route53",
      "Effect": "Allow",
      "Action": [
        "route53:GetHostedZone",
        "route53:GetHostedZoneCount",
        "route53:ListHostedZones",
        "route53:ListHostedZonesByName",
        "route53:ListResourceRecordSets"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "DecribeIAMRoles",
      "Effect": "Allow",
      "Action": [
        "iam:GetRole",
        "iam:ListRoles"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "EC2DescribeInstance",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeIamInstanceProfileAssociations",
        "ec2:DescribeReservedInstances",
        "ec2:DescribeScheduledInstances"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "VPCNetwork",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeDhcpOptions",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeSubnets",
        "ec2:DescribeRouteTables"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "Cloudtrail",
      "Effect": "Allow",
      "Action": [
        "cloudtrail:DescribeTrails",
        "cloudtrail:LookupEvents"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "Cloudwatch",
      "Effect": "Allow",
      "Action": [
        "cloudwatch:GetMetricData",
        "cloudwatch:GetMetricStatistics",
        "cloudwatch:ListMetrics"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "DescribeVolumes",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeVolumes",
        "ec2:DescribeVolumesModifications",
        "ec2:DescribeVolumeStatus"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "DescribeLoadBalancers",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DescribeAccountLimits",
        "elasticloadbalancing:DescribeInstanceHealth",
        "elasticloadbalancing:DescribeListenerCertificates",
        "elasticloadbalancing:DescribeListeners",
        "elasticloadbalancing:DescribeLoadBalancerAttributes",
        "elasticloadbalancing:DescribeLoadBalancerPolicies",
        "elasticloadbalancing:DescribeLoadBalancerPolicyTypes",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeRules",
        "elasticloadbalancing:DescribeSSLPolicies",
        "elasticloadbalancing:DescribeTags",
        "elasticloadbalancing:DescribeTargetGroupAttributes",
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "DescribeVPC",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeVpcEndpointConnections",
        "ec2:DescribeVpcEndpoints"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "DescribeSecurityGroups",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroupReferences",
        "ec2:DescribeSecurityGroupRules",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeStaleSecurityGroups"
      ],
      "Resource": "*"
    },
    {
      "Sid": "DescribeAddressesAttribute",
      "Effect": "Allow",
      "Action": "ec2:DescribeAddressesAttribute",
      "Resource": "arn:aws:ec2:*:*:elastic-ip/*"
    },
    {
      "Sid": "DescribeInstance",
      "Effect": "Allow",
      "Action": [
        "iam:GetInstanceProfile"
      ],
      "Resource": "arn:aws:iam::*:instance-profile/*",
      "Condition": {
        "StringEquals": {
            "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "DescribeSpotFleetInstances",
      "Effect": "Allow",
      "Action": "ec2:DescribeSpotFleetInstances",
      "Resource": "arn:aws:ec2:*:*:spot-fleet-request/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "DescribeVolumeAttribute",
      "Effect": "Allow",
      "Action": "ec2:DescribeVolumeAttribute",
      "Resource": "arn:aws:ec2:*:*:volume/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "ManageInstanceLifecycle",
      "Effect": "Allow",
      "Action": [
        "ec2:RebootInstances",
        "ec2:StartInstances",
        "ec2:StopInstances",
        "ec2:TerminateInstances"
      ],
      "Resource": "arn:aws:ec2:*:*:instance/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    }
  ]
}
Copy to Clipboard Toggle word wrap

Example 7.6. sts_hcp_support_trust_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::710019948333:role/RH-Technical-Support-15234082"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.7. ROSA Kube Controller Operator policy and policy file
ResourceDescription

openshift-hcp-kube-controller-manager-credentials

An IAM policy that grants permissions to the kube controller to manage Amazon EC2, Elastic Load Balancing, and AWS KMS resources.

Example 7.7. openshift-hcp_kube-controller-manager-credentials-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadPermissions",
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeInstances",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs",
        "elasticloadbalancing:DescribeLoadBalancers",
        "elasticloadbalancing:DescribeLoadBalancerAttributes",
        "elasticloadbalancing:DescribeListeners",
        "elasticloadbalancing:DescribeTargetGroups",
        "elasticloadbalancing:DescribeTargetHealth",
        "elasticloadbalancing:DescribeLoadBalancerPolicies"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "KMSDescribeKey",
      "Effect": "Allow",
      "Action": [
        "kms:DescribeKey"
      ],
      "Resource": [
        "*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat": "true"
        }
      }
    },
    {
      "Sid": "LoadBalanacerManagement",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:AddTags",
        "elasticloadbalancing:ConfigureHealthCheck",
        "elasticloadbalancing:CreateLoadBalancerPolicy",
        "elasticloadbalancing:DeleteLoadBalancer",
        "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
        "elasticloadbalancing:ModifyLoadBalancerAttributes",
        "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
        "elasticloadbalancing:SetLoadBalancerPoliciesForBackendServer"
      ],
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "CreateTargetGroup",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:CreateTargetGroup"
      ],
      "Resource": [
        "*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "LoadBalanacerManagementResourceTag",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DeleteListener",
        "elasticloadbalancing:RegisterTargets",
        "elasticloadbalancing:ModifyTargetGroup",
        "elasticloadbalancing:DeleteTargetGroup",
        "elasticloadbalancing:ApplySecurityGroupsToLoadBalancer",
        "elasticloadbalancing:CreateLoadBalancerListeners",
        "elasticloadbalancing:DeleteLoadBalancerListeners",
        "elasticloadbalancing:AttachLoadBalancerToSubnets",
        "elasticloadbalancing:DetachLoadBalancerFromSubnets",
        "elasticloadbalancing:ModifyListener",
        "elasticloadbalancing:SetLoadBalancerPoliciesOfListener"
      ],
      "Resource": [
        "*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateListeners",
      "Effect": "Allow",
      "Action": [
          "elasticloadbalancing:CreateListener"
      ],
      "Resource": [
          "*"
      ],
      "Condition": {
          "StringEquals": {
              "aws:RequestTag/red-hat-managed": "true",
              "aws:ResourceTag/red-hat-managed": "true"
          }
      }
  },
    {
      "Sid": "CreateSecurityGroup",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateSecurityGroupVpc",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:vpc/*"
      ]
    },
    {
      "Sid": "CreateLoadBalancer",
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:CreateLoadBalancer"
      ],
      "Resource": [
        "arn:aws:elasticloadbalancing:*:*:loadbalancer/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "ModifySecurityGroup",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupIngress",
        "ec2:DeleteSecurityGroup"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateTagsSecurityGroups",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": "CreateSecurityGroup"
        }
      }
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.8. ROSA Control Plane Operator policy and policy file
ResourceDescription

openshift-hcp-control-plane-operator-credentials-policy

An IAM policy that grants required permissions to the Control Plane Operator to manage Amazon EC2 and Route 53 resources.

Example 7.8. openshift_hcp_control_plane_operator_credentials_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "ReadPermissions",
            "Action": [
                "ec2:DescribeVpcEndpoints",
                "ec2:DescribeVpcs",
                "ec2:DescribeSecurityGroups",
                "route53:ListHostedZones"
            ],
            "Effect": "Allow",
            "Resource": "*"
        },
        {
            "Sid": "CreateSecurityGroups",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "DeleteSecurityGroup",
            "Effect": "Allow",
            "Action": [
                "ec2:DeleteSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "SecurityGroupIngressEgress",
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:AuthorizeSecurityGroupEgress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupEgress"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "CreateSecurityGroupsVPCNoCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc/*"
            ]
        },
        {
            "Sid": "ListResourceRecordSets",
            "Action": [
                "route53:ListResourceRecordSets"
            ],
            "Effect": "Allow",
            "Resource": [
                "*"
            ]
        },
        {
            "Sid": "ChangeResourceRecordSetsRestrictedRecordNames",
            "Action": [
                "route53:ChangeResourceRecordSets"
            ],
            "Effect": "Allow",
            "Resource": [
                "*"
            ],
            "Condition": {
                "ForAllValues:StringLike": {
                    "route53:ChangeResourceRecordSetsNormalizedRecordNames": [
                        "*.hypershift.local"
                    ]
                }
            }
        },
        {
            "Sid": "VPCEndpointWithCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateVpcEndpoint"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc-endpoint/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:RequestTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "VPCEndpointResourceTagCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateVpcEndpoint"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:security-group*/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "VPCEndpointNoCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateVpcEndpoint"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc/*",
                "arn:aws:ec2:*:*:subnet/*",
                "arn:aws:ec2:*:*:route-table/*"
            ]
        },
        {
            "Sid": "ManageVPCEndpointWithCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:ModifyVpcEndpoint",
                "ec2:DeleteVpcEndpoints"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc-endpoint/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:ResourceTag/red-hat-managed": "true"
                }
            }
        },
        {
            "Sid": "ModifyVPCEndpoingNoCondition",
            "Effect": "Allow",
            "Action": [
                "ec2:ModifyVpcEndpoint"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:subnet/*"
            ]
        },
        {
            "Sid": "CreateTagsRestrictedActions",
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": [
                "arn:aws:ec2:*:*:vpc-endpoint/*",
                "arn:aws:ec2:*:*:security-group/*"
            ],
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": [
                        "CreateVpcEndpoint",
                        "CreateSecurityGroup"
                    ]
                }
            }
        }
    ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.9. ROSA Node Pool Management Operator policy and policy file
ResourceDescription

openshift-hcp-capa-controller-manager-credentials-policy

An IAM policy that grants required permissions to the NodePool controller to describe, run, and terminate Amazon EC2 instances managed as worker nodes. This policy also grants permissions to allow for disk encryption of the worker node root volume using AWS KMS keys, and to tag the elastic network interface that is attached to the worker node.

Example 7.9. openshift_hcp_capa_controller_manager_credentials_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadPermissions",
      "Action": [
        "ec2:DescribeDhcpOptions",
        "ec2:DescribeImages",
        "ec2:DescribeInstances",
        "ec2:DescribeInternetGateways",
        "ec2:DescribeNetworkInterfaces",
        "ec2:DescribeNetworkInterfaceAttribute",
        "ec2:DescribeRouteTables",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeVpcs"
      ],
      "Effect": "Allow",
      "Resource": [
        "*"
      ]
    },
    {
      "Sid": "CreateServiceLinkedRole",
      "Action": [
        "iam:CreateServiceLinkedRole"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:*:iam::*:role/aws-service-role/elasticloadbalancing.amazonaws.com/AWSServiceRoleForElasticLoadBalancing"
      ],
      "Condition": {
        "StringLike": {
          "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
        }
      }
    },
    {
      "Sid": "PassWorkerRole",
      "Action": [
        "iam:PassRole"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:*:iam::*:role/*-ROSA-Worker-Role"
      ],
      "Condition": {
        "StringEquals": {
          "iam:PassedToService": [
              "ec2.amazonaws.com"
          ]
        }
      }
    },
    {
      "Sid": "AuthorizeSecurityGroupIngressRestrictedResourceTag",
      "Effect": "Allow",
      "Action": [
        "ec2:AuthorizeSecurityGroupIngress"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:security-group/*",
        "arn:aws:ec2:*:*:security-group-rule/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "NetworkInterfaces",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "NetworkInterfacesNoCondition",
      "Effect": "Allow",
      "Action": [
        "ec2:ModifyNetworkInterfaceAttribute"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:security-group/*",
        "arn:aws:ec2:*:*:vpc/*"
      ]
    },
    {
      "Sid": "TerminateInstances",
      "Effect": "Allow",
      "Action": [
        "ec2:TerminateInstances"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateTags",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:volume/*",
        "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
            "ec2:CreateAction": [
                "RunInstances"
            ]
        }
      }
    },
    {
      "Sid": "CreateTagsCAPAControllerReconcileNetworkInterface",
      "Effect": "Allow",
      "Action": [
          "ec2:CreateTags"
      ],
      "Resource": [
          "arn:aws:ec2:*:*:network-interface/*"
      ],
      "Condition": {
        "StringEquals": {
              "aws:RequestTag/red-hat-managed": "true"
          }
       }
    },
    {
      "Sid": "CreateTagsCAPAControllerReconcileInstance",
      "Effect": "Allow",
      "Action": [
          "ec2:CreateTags"
      ],
      "Resource": [
          "arn:aws:ec2:*:*:instance/*"
      ],
      "Condition": {
          "StringEquals": {
              "aws:ResourceTag/red-hat-managed": "true"
          }
      }
    },
    {
      "Sid": "CreateTagsCAPAControllerReconcileVolume",
      "Effect": "Allow",
      "Action": [
          "ec2:CreateTags"
      ],
      "Resource": [
          "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
          "StringEquals": {
              "aws:RequestTag/red-hat-managed": "true"
          }
      }
    },
    {
      "Sid": "RunInstancesRequest",
      "Effect": "Allow",
      "Action": [
        "ec2:RunInstances"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "RunInstancesNoCondition",
      "Effect": "Allow",
      "Action": [
        "ec2:RunInstances"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:network-interface/*",
        "arn:aws:ec2:*:*:subnet/*",
        "arn:aws:ec2:*:*:security-group/*",
        "arn:aws:ec2:*:*:volume/*",
        "arn:aws:ec2:*:*:capacity-reservation/*"
      ]
    },
    {
      "Sid": "RunInstancesRedHatAMI",
      "Effect": "Allow",
      "Action": [
        "ec2:RunInstances"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:image/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:Owner": [
            "531415883065",
            "251351625822"
          ]
        }
      }
    },
    {
      "Sid": "ManagedKMSRestrictedResourceTag",
      "Effect": "Allow",
      "Action": [
        "kms:DescribeKey",
        "kms:GenerateDataKeyWithoutPlaintext"
      ],
      "Resource": "*",
      "Condition": {
        "StringLike": {
          "aws:ResourceTag/red-hat": "true"
        }
      }
    },
    {
      "Sid": "CreateGrantRestricted",
      "Effect": "Allow",
      "Action": [
        "kms:CreateGrant"
      ],
      "Resource": "*",
      "Condition": {
        "Bool": {
          "kms:GrantIsForAWSResource": true
        },
        "StringEquals": {
          "aws:ResourceTag/red-hat": "true"
        },
        "StringLike": {
          "kms:ViaService": "ec2.*.amazonaws.com"
        }
      }
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.10. ROSA Image Registry Operator policy and policy file
ResourceDescription

openshift-hcp-image-registry-operator-permission-policy

An IAM policy that grants required permissions to the Image Registry Operator to provision and manage resources for the ROSA in-cluster image registry and dependent services, including S3. This is required so that the operator can install and maintain the internal registry of a ROSA cluster.

Example 7.10. openshift_hcp_image_registry_operator_permission_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBuckets",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": "*"
    },
    {
      "Sid": "AllowSpecificBucketActions",
      "Effect": "Allow",
      "Action": [
        "s3:CreateBucket",
        "s3:DeleteBucket",
        "s3:GetBucketTagging",
        "s3:GetBucketPublicAccessBlock",
        "s3:GetEncryptionConfiguration",
        "s3:GetLifecycleConfiguration",
        "s3:GetBucketLocation",
        "s3:PutBucketPublicAccessBlock",
        "s3:PutBucketTagging",
        "s3:PutEncryptionConfiguration",
        "s3:PutLifecycleConfiguration"
      ],
      "Resource": [
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}-*",
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}?",
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}"
      ]
    },
    {
      "Sid": "AllowSpecificObjectActions",
      "Effect": "Allow",
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListMultipartUploadParts",
        "s3:PutObject"
      ],
      "Resource": [
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}-*/*",
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}?/*",
        "arn:aws:s3:::*-image-registry-${aws:RequestedRegion}/*"
      ]
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.11. ROSA Amazon EBSCI Driver Operator policy and policy file
ResourceDescription

openshift-hcp-cluster-csi-driver-ebs-operator-cloud-credentials-policy

An IAM policy that grants necessary permissions to the Amazon EBS CSI Driver Operator to install and maintain the Amazon EBS CSI driver on a ROSA cluster.

Example 7.11. openshift_hcp_cluster_csi_driver_ebs_operator_cloud_credentials_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeSnapshots",
        "ec2:DescribeTags",
        "ec2:DescribeVolumes",
        "ec2:DescribeVolumesModifications"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:AttachVolume",
        "ec2:DetachVolume"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:instance/*",
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DeleteVolume",
        "ec2:ModifyVolume"
       ],
      "Resource": [
        "arn:aws:ec2:*:*:volume/*"
       ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateVolume"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateVolumeFromSnapshot",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateVolume"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:snapshot/*"
      ]
    },
    {
      "Sid": "CreateSnapshotResourceTag",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSnapshot"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:volume/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Sid": "CreateSnapshotRequestTag",
      "Effect": "Allow",
      "Action": [
        "ec2:CreateSnapshot"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:snapshot/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DeleteSnapshot"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:snapshot/*"
      ],
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateTags"
      ],
      "Resource": [
        "arn:aws:ec2:*:*:volume/*",
        "arn:aws:ec2:*:*:snapshot/*"
      ],
      "Condition": {
        "StringEquals": {
          "ec2:CreateAction": [
            "CreateVolume",
            "CreateSnapshot"
          ]
        }
      }
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.12. ROSA Cloud Network Config Operator policy and policy file
ResourceDescription

openshift-hcp-cloud-network-config-cloud-credentials-permission-policy

An IAM policy that grants necessary permissions to the Amazon EBS CSI Driver Operator to install and maintain the Amazon EBS CSI driver on a ROSA cluster.

Example 7.12. openshift_hcp_cloud_network_config_cloud_credentials_permission_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
            "ec2:DescribeInstances",
            "ec2:DescribeInstanceStatus",
            "ec2:DescribeInstanceTypes",
            "ec2:DescribeSubnets",
            "ec2:DescribeNetworkInterfaces"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
            "ec2:UnassignPrivateIpAddresses",
            "ec2:AssignPrivateIpAddresses",
            "ec2:UnassignIpv6Addresses",
            "ec2:AssignIpv6Addresses"
      ],
      "Resource": "arn:aws:ec2:*:*:network-interface/*",
      "Condition": {
        "StringEquals": {
          "aws:ResourceTag/red-hat-managed": "true"
        }
      }
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.13. ROSA Ingress Operator policy and policy file
ResourceDescription

openshift-hcp-cluster-ingress-operator-cloud-credentials-policy

An IAM policy that provides the ROSA Ingress Operator with the permissions required to manage external access to a cluster.

Example 7.13. openshift_hcp_cluster_ingress_operator_cloud_credentials_policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "elasticloadbalancing:DescribeLoadBalancers",
        "route53:ListHostedZones",
        "tag:GetResources"
      ],
      "Resource": "*"
    },
    {
          "Effect": "Allow",
          "Action": [
              "route53:ChangeResourceRecordSets"
          ],
          "Resource": "*",
          "Condition": {
              "ForAllValues:StringLike": {
                  "route53:ChangeResourceRecordSetsNormalizedRecordNames": [
                    "*.openshiftapps.com",
                    "*.devshift.org",
                    "*.openshiftusgov.com",
                    "*.devshiftusgov.com"
                  ]
              }
          }
    }
  ]
}
Copy to Clipboard Toggle word wrap
Expand
Table 7.14. ROSA KMS Provider Operator policy and policy file
ResourceDescription

openshift-hcp-kms-provider-credential-policy.

An IAM policy grants required permissions to the built-in AWS Encryption Provider to manage AWS KMS keys that support etcd data encryption. This policy allows Amazon EC2 to use KMS keys that the AWS Encryption Provider provides to encrypt and decrypt etcd data.

Example 7.14. openshift_hcp_kms_provider_credential_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Sid": "VolumeEncryption",
        "Effect": "Allow",
        "Action": [
          "kms:Encrypt",
          "kms:Decrypt",
          "kms:DescribeKey"
        ],
        "Resource": "*",
        "Condition": {
          "StringEquals": {
            "aws:ResourceTag/red-hat": "true"
          }
        }
      }
    ]
}
Copy to Clipboard Toggle word wrap

This section lists the aws CLI commands that the rosa command generates in the terminal. You can run the command in either manual or automatic mode.

Using manual mode for account role creation

The manual role creation mode generates the aws commands for you to review and run. The following command starts that process, where <openshift_version> refers to your version of Red Hat OpenShift Service on AWS (ROSA), such as 4. 

$ rosa create account-roles --mode manual
Copy to Clipboard Toggle word wrap
Note

The provided command examples include the ManagedOpenShift prefix. The ManagedOpenShift prefix is the default value, if you do not specify a custom prefix by using the --prefix option.

Command output

aws iam create-role \
	--role-name ManagedOpenShift-Installer-Role \
	--assume-role-policy-document file://sts_installer_trust_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=rosa_role_type,Value=installer

aws iam put-role-policy \
	--role-name ManagedOpenShift-Installer-Role \
	--policy-name ManagedOpenShift-Installer-Role-Policy \
	--policy-document file://sts_installer_permission_policy.json

aws iam create-role \
	--role-name ManagedOpenShift-ControlPlane-Role \
	--assume-role-policy-document file://sts_instance_controlplane_trust_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=rosa_role_type,Value=instance_controlplane

aws iam put-role-policy \
	--role-name ManagedOpenShift-ControlPlane-Role \
	--policy-name ManagedOpenShift-ControlPlane-Role-Policy \
	--policy-document file://sts_instance_controlplane_permission_policy.json

aws iam create-role \
	--role-name ManagedOpenShift-Worker-Role \
	--assume-role-policy-document file://sts_instance_worker_trust_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=rosa_role_type,Value=instance_worker

aws iam put-role-policy \
	--role-name ManagedOpenShift-Worker-Role \
	--policy-name ManagedOpenShift-Worker-Role-Policy \
	--policy-document file://sts_instance_worker_permission_policy.json

aws iam create-role \
	--role-name ManagedOpenShift-Support-Role \
	--assume-role-policy-document file://sts_support_trust_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=rosa_role_type,Value=support

aws iam put-role-policy \
	--role-name ManagedOpenShift-Support-Role \
	--policy-name ManagedOpenShift-Support-Role-Policy \
	--policy-document file://sts_support_permission_policy.json

aws iam create-policy \
	--policy-name ManagedOpenShift-openshift-ingress-operator-cloud-credentials \
	--policy-document file://openshift_ingress_operator_cloud_credentials_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-ingress-operator Key=operator_name,Value=cloud-credentials

aws iam create-policy \
	--policy-name ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credent \
	--policy-document file://openshift_cluster_csi_drivers_ebs_cloud_credentials_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-cluster-csi-drivers Key=operator_name,Value=ebs-cloud-credentials

aws iam create-policy \
	--policy-name ManagedOpenShift-openshift-machine-api-aws-cloud-credentials \
	--policy-document file://openshift_machine_api_aws_cloud_credentials_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-machine-api Key=operator_name,Value=aws-cloud-credentials

aws iam create-policy \
	--policy-name ManagedOpenShift-openshift-cloud-credential-operator-cloud-crede \
	--policy-document file://openshift_cloud_credential_operator_cloud_credential_operator_iam_ro_creds_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-cloud-credential-operator Key=operator_name,Value=cloud-credential-operator-iam-ro-creds

aws iam create-policy \
	--policy-name ManagedOpenShift-openshift-image-registry-installer-cloud-creden \
	--policy-document file://openshift_image_registry_installer_cloud_credentials_policy.json \
	--tags Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value=ManagedOpenShift Key=operator_namespace,Value=openshift-image-registry Key=operator_name,Value=installer-cloud-credentials
Copy to Clipboard Toggle word wrap

Using auto mode for role creation

When you add the --mode auto argument, the Red Hat OpenShift Service on AWS (ROSA) CLI, rosa, creates your roles and policies. The following command starts that process: 

$ rosa create account-roles --mode auto
Copy to Clipboard Toggle word wrap
Note

The provided command examples include the ManagedOpenShift prefix. The ManagedOpenShift prefix is the default value, if you do not specify a custom prefix by using the --prefix option.

Command output

I: Creating roles using 'arn:aws:iam::<ARN>:user/<UserID>'
? Create the 'ManagedOpenShift-Installer-Role' role? Yes
I: Created role 'ManagedOpenShift-Installer-Role' with ARN 'arn:aws:iam::<ARN>:role/ManagedOpenShift-Installer-Role'
? Create the 'ManagedOpenShift-ControlPlane-Role' role? Yes
I: Created role 'ManagedOpenShift-ControlPlane-Role' with ARN 'arn:aws:iam::<ARN>:role/ManagedOpenShift-ControlPlane-Role'
? Create the 'ManagedOpenShift-Worker-Role' role? Yes
I: Created role 'ManagedOpenShift-Worker-Role' with ARN 'arn:aws:iam::<ARN>:role/ManagedOpenShift-Worker-Role'
? Create the 'ManagedOpenShift-Support-Role' role? Yes
I: Created role 'ManagedOpenShift-Support-Role' with ARN 'arn:aws:iam::<ARN>:role/ManagedOpenShift-Support-Role'
? Create the operator policies? Yes
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-machine-api-aws-cloud-credentials'
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-cloud-credential-operator-cloud-crede'
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-image-registry-installer-cloud-creden'
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-ingress-operator-cloud-credentials'
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credent'
I: Created policy with ARN 'arn:aws:iam::<ARN>:policy/ManagedOpenShift-openshift-cloud-network-config-controller-cloud'
I: To create a cluster with these roles, run the following command:
rosa create cluster --sts
Copy to Clipboard Toggle word wrap

7.3. Cluster-specific Operator IAM role reference
Copy link

Operator roles are used to obtain the temporary permissions required to carry out cluster operations, such as managing back-end storage, cloud ingress controller, and external access to a cluster.

When you create the Operator roles, the account-wide Operator policies for the matching cluster version are attached to the roles. AWS managed Operator policies are versioned in AWS IAM. The latest version of an AWS managed policy is always used, so you do not need to manage or schedule upgrades for AWS managed policies used by ROSA with HCP.

Note

If more than one matching policy is available in your account for an Operator role, an interactive list of options is provided when you create the role.

Expand
Table 7.15. Required Operator roles and AWS Managed policies for ROSA with HCP
Role nameAWS Managed policy nameRole description

openshift-cloud-network-config-controller-credentials

ROSACloudNetworkConfigOperatorPolicy

An IAM role required by the cloud network config controller to manage cloud network credentials for a cluster.

openshift-image-registry-installer-cloud-credentials

ROSAImageRegistryOperatorPolicy

An IAM role required by the ROSA Image Registry Operator to manage the OpenShift image registry storage in AWS S3 for a cluster.

kube-system-kube-controller-manager

ROSAKubeControllerPolicy

An IAM role required for OpenShift management on HCP clusters.

kube-system-capa-controller-manager

ROSANodePoolManagementPolicy

An IAM role required for node management on HCP clusters.

kube-system-control-plane-operator

ROSAControlPlaneOperatorPolicy

An IAM role required control plane management on HCP clusters.

kube-system-kms-provider

ROSAKMSProviderPolicy

An IAM role required for OpenShift management on HCP clusters.

openshift-ingress-operator-cloud-credentials

ROSAIngressOperatorPolicy

An IAM role required by the ROSA Ingress Operator to manage external access to a cluster.

openshift-cluster-csi-drivers-ebs-cloud-credentials

ROSAAmazonEBSCSIDriverOperatorPolicy

An IAM role required by ROSA to manage back-end storage through the Container Storage Interface (CSI).

7.3.1. Operator IAM role AWS CLI reference
Copy link

This section lists the aws CLI commands that are shown in the terminal when you run the following rosa command using manual mode: 

$ rosa create operator-roles --mode manual --cluster <cluster_name>
Copy to Clipboard Toggle word wrap
Note

When using manual mode, the aws commands are printed to the terminal for your review. After reviewing the aws commands, you must run them manually. Alternatively, you can specify --mode auto with the rosa create command to run the aws commands immediately.

Command output

aws iam create-role \
	--role-name <cluster_name>-<hash>-openshift-cluster-csi-drivers-ebs-cloud-credent \
	--assume-role-policy-document file://operator_cluster_csi_drivers_ebs_cloud_credentials_policy.json \
	--tags Key=rosa_cluster_id,Value=<id> Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value= Key=operator_namespace,Value=openshift-cluster-csi-drivers Key=operator_name,Value=ebs-cloud-credentials

aws iam attach-role-policy \
	--role-name <cluster_name>-<hash>-openshift-cluster-csi-drivers-ebs-cloud-credent \
	--policy-arn arn:aws:iam::<aws_account_id>:policy/ManagedOpenShift-openshift-cluster-csi-drivers-ebs-cloud-credent

aws iam create-role \
	--role-name <cluster_name>-<hash>-openshift-machine-api-aws-cloud-credentials \
	--assume-role-policy-document file://operator_machine_api_aws_cloud_credentials_policy.json \
	--tags Key=rosa_cluster_id,Value=<id> Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value= Key=operator_namespace,Value=openshift-machine-api Key=operator_name,Value=aws-cloud-credentials

aws iam attach-role-policy \
	--role-name <cluster_name>-<hash>-openshift-machine-api-aws-cloud-credentials \
	--policy-arn arn:aws:iam::<aws_account_id>:policy/ManagedOpenShift-openshift-machine-api-aws-cloud-credentials

aws iam create-role \
	--role-name <cluster_name>-<hash>-openshift-cloud-credential-operator-cloud-crede \
	--assume-role-policy-document file://operator_cloud_credential_operator_cloud_credential_operator_iam_ro_creds_policy.json \
	--tags Key=rosa_cluster_id,Value=<id> Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value= Key=operator_namespace,Value=openshift-cloud-credential-operator Key=operator_name,Value=cloud-credential-operator-iam-ro-creds

aws iam attach-role-policy \
	--role-name <cluster_name>-<hash>-openshift-cloud-credential-operator-cloud-crede \
	--policy-arn arn:aws:iam::<aws_account_id>:policy/ManagedOpenShift-openshift-cloud-credential-operator-cloud-crede

aws iam create-role \
	--role-name <cluster_name>-<hash>-openshift-image-registry-installer-cloud-creden \
	--assume-role-policy-document file://operator_image_registry_installer_cloud_credentials_policy.json \
	--tags Key=rosa_cluster_id,Value=<id> Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value= Key=operator_namespace,Value=openshift-image-registry Key=operator_name,Value=installer-cloud-credentials

aws iam attach-role-policy \
	--role-name <cluster_name>-<hash>-openshift-image-registry-installer-cloud-creden \
	--policy-arn arn:aws:iam::<aws_account_id>:policy/ManagedOpenShift-openshift-image-registry-installer-cloud-creden

aws iam create-role \
	--role-name <cluster_name>-<hash>-openshift-ingress-operator-cloud-credentials \
	--assume-role-policy-document file://operator_ingress_operator_cloud_credentials_policy.json \
	--tags Key=rosa_cluster_id,Value=<id> Key=rosa_openshift_version,Value=<openshift_version> Key=rosa_role_prefix,Value= Key=operator_namespace,Value=openshift-ingress-operator Key=operator_name,Value=cloud-credentials

aws iam attach-role-policy \
	--role-name <cluster_name>-<hash>-openshift-ingress-operator-cloud-credentials \
	--policy-arn arn:aws:iam::<aws_account_id>:policy/ManagedOpenShift-openshift-ingress-operator-cloud-credentials
Copy to Clipboard Toggle word wrap

Note

The command examples provided in the table include Operator roles that use the ManagedOpenShift prefix. If you defined a custom prefix when you created the account-wide roles and policies, including the Operator policies, you must reference it by using the --prefix <prefix_name> option when you create the Operator roles.

7.3.2. About custom Operator IAM role prefixes
Copy link

Each Red Hat OpenShift Service on AWS (ROSA) cluster requires cluster-specific Operator IAM roles.

By default, the Operator role names are prefixed with the cluster name and a random 4-digit hash. For example, the Ingress Cloud Credentials Operator IAM role for a cluster named mycluster has the default name mycluster-<hash>-openshift-ingress-operator-cloud-credentials, where <hash> is a random 4-digit string.

This default naming convention enables you to easily identify the Operator IAM roles for a cluster in your AWS account.

When you create the Operator roles for a cluster, you can optionally specify a custom prefix to use instead of <cluster_name>-<hash>. By using a custom prefix, you can prepend logical identifiers to your Operator role names to meet the requirements of your environment. For example, you might prefix the cluster name and the environment type, such as mycluster-dev. In that example, the Ingress Cloud Credentials Operator role name with the custom prefix is mycluster-dev-openshift-ingress-operator-cloud-credenti.

Note

The role names are truncated to 64 characters.

For ROSA installations that use STS, you must create a cluster-specific OIDC provider that is used by the cluster Operators to authenticate or create your own OIDC configuration for your own OIDC provider.

7.4.1. Creating an OIDC provider using the CLI
Copy link

You can create an OIDC provider that is hosted in your AWS account with the Red Hat OpenShift Service on AWS (ROSA) CLI, rosa.

Prerequisites

  • You have installed the latest version of the ROSA CLI.

Procedure

  • To create an OIDC provider, by using an unregistered or a registered OIDC configuration.

    • Unregistered OIDC configurations require you to create the OIDC provider through the cluster. Run the following to create the OIDC provider: 

      $ rosa create oidc-provider --mode manual --cluster <cluster_name>
      Copy to Clipboard Toggle word wrap
      Note

      When using manual mode, the aws command is printed to the terminal for your review. After reviewing the aws command, you must run it manually. Alternatively, you can specify --mode auto with the rosa create command to run the aws command immediately.

      Command output

      aws iam create-open-id-connect-provider \
	--url https://oidc.op1.openshiftapps.com/<oidc_config_id> \
      1 
      
	--client-id-list openshift sts.<aws_region>.amazonaws.com \
	--thumbprint-list <thumbprint>
      2
      Copy to Clipboard Toggle word wrap

      1
      The URL used to reach the OpenID Connect (OIDC) identity provider after the cluster is created.
      2
      The thumbprint is generated automatically when you run the rosa create oidc-provider command. For more information about using thumbprints with AWS Identity and Access Management (IAM) OIDC identity providers, see the AWS documentation.

    • Registered OIDC configurations use an OIDC configuration ID. Run the following command with your OIDC configuration ID: 

      $ rosa create oidc-provider --oidc-config-id <oidc_config_id> --mode auto -y
      Copy to Clipboard Toggle word wrap

      Command output

      I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName'
I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/241rh9ql5gpu99d7leokhvkp8icnalpf'
      Copy to Clipboard Toggle word wrap

7.4.2. Creating an OpenID Connect Configuration
Copy link

When using a cluster hosted by Red Hat, you can create a managed or unmanaged OpenID Connect (OIDC) configuration by using the Red Hat OpenShift Service on AWS (ROSA) CLI, rosa. A managed OIDC configuration is stored within Red Hat’s AWS account, while a generated unmanaged OIDC configuration is stored within your AWS account. The OIDC configuration is registered to be used with OpenShift Cluster Manager. When creating an unmanaged OIDC configuration, the CLI provides the private key for you.

Creating an OpenID Connect configuration

When creating a Red Hat OpenShift Service on AWS cluster, you can create the OpenID Connect (OIDC) configuration before creating your cluster. This configuration is registered to be used with OpenShift Cluster Manager.

Prerequisites

  • You have completed the AWS prerequisites for Red Hat OpenShift Service on AWS.
  • You have installed and configured the latest ROSA CLI, rosa, on your installation host.

Procedure

  1. To create your OIDC configuration alongside the AWS resources, run the following command: 

    $ rosa create oidc-config --mode=auto --yes
    Copy to Clipboard Toggle word wrap

    This command returns the following information.

    Example output

    ? Would you like to create a Managed (Red Hat hosted) OIDC Configuration Yes
I: Setting up managed OIDC configuration
I: To create Operator Roles for this OIDC Configuration, run the following command and remember to replace <user-defined> with a prefix of your choice:
	rosa create operator-roles --prefix <user-defined> --oidc-config-id 13cdr6b
If you are going to create a Hosted Control Plane cluster please include '--hosted-cp'
I: Creating OIDC provider using 'arn:aws:iam::4540112244:user/userName'
? Create the OIDC provider? Yes
I: Created OIDC provider with ARN 'arn:aws:iam::4540112244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/13cdr6b'
    Copy to Clipboard Toggle word wrap

    When creating your cluster, you must supply the OIDC config ID. The CLI output provides this value for --mode auto, otherwise you must determine these values based on aws CLI output for --mode manual.

  2. Optional: you can save the OIDC configuration ID as a variable to use later. Run the following command to save the variable: 

    $ export OIDC_ID=<oidc_config_id>
    1
    Copy to Clipboard Toggle word wrap
    1
    In the example output above, the OIDC configuration ID is 13cdr6b.

    • View the value of the variable by running the following command: 

      $ echo $OIDC_ID
      Copy to Clipboard Toggle word wrap

      Example output

      13cdr6b
      Copy to Clipboard Toggle word wrap

Verification

  • You can list the possible OIDC configurations available for your clusters that are associated with your user organization. Run the following command: 

    $ rosa list oidc-config
    Copy to Clipboard Toggle word wrap

    Example output

    ID                                MANAGED  ISSUER URL                                                             SECRET ARN
2330dbs0n8m3chkkr25gkkcd8pnj3lk2  true     https://dvbwgdztaeq9o.cloudfront.net/2330dbs0n8m3chkkr25gkkcd8pnj3lk2
233hvnrjoqu14jltk6lhbhf2tj11f8un  false    https://oidc-r7u1.s3.us-east-1.amazonaws.com                           aws:secretsmanager:us-east-1:242819244:secret:rosa-private-key-oidc-r7u1-tM3MDN
    Copy to Clipboard Toggle word wrap

Parameter options for creating your own OpenID Connect configuration

The following options may be added to the rosa create oidc-config command. All of these parameters are optional. Running the rosa create oidc-config command without parameters creates an unmanaged OIDC configuration.

Note

You are required to register the unmanaged OIDC configuration by posting a request to /oidc_configs through OpenShift Cluster Manager. You receive an ID in the response. Use this ID to create a cluster.

raw-files

Allows you to provide raw files for the private RSA key. This key is named rosa-private-key-oidc-<random_label_of_length_4>.key. You also receive a discovery document, named discovery-document-oidc-<random_label_of_length_4>.json, and a JSON Web Key Set, named jwks-oidc-<random_label_of_length_4>.json.

You use these files to set up the endpoint. This endpoint responds to /.well-known/openid-configuration with the discovery document and on keys.json with the JSON Web Key Set. The private key is stored in Amazon Web Services (AWS) Secrets Manager Service (SMS) as plaintext.

Example

$ rosa create oidc-config --raw-files
Copy to Clipboard Toggle word wrap

mode

Allows you to specify the mode to create your OIDC configuration. With the manual option, you receive AWS commands that set up the OIDC configuration in an S3 bucket. This option stores the private key in the Secrets Manager. With the manual option, the OIDC Endpoint URL is the URL for the S3 bucket. You must retrieve the Secrets Manager ARN to register the OIDC configuration with OpenShift Cluster Manager.

You receive the same OIDC configuration and AWS resources as the manual mode when using the auto option. A significant difference between the two options is that when using the auto option, ROSA calls AWS, so you do not need to take any further actions. The OIDC Endpoint URL is the URL for the S3 bucket. The CLI retrieves the Secrets Manager ARN, registers the OIDC configuration with OpenShift Cluster Manager, and reports the second rosa command that the user can run to continue with the creation of the STS cluster.

Example

$ rosa create oidc-config --mode=<auto|manual>
Copy to Clipboard Toggle word wrap

managed

Creates an OIDC configuration that is hosted under Red Hat’s AWS account. This command creates a private key that responds directly with an OIDC Config ID for you to use when creating the STS cluster.

Example

$ rosa create oidc-config --managed
Copy to Clipboard Toggle word wrap

Example output

W: For a managed OIDC Config only auto mode is supported. However, you may choose the provider creation mode
? OIDC Provider creation mode: auto
I: Setting up managed OIDC configuration
I: Please run the following command to create a cluster with this oidc config
rosa create cluster --sts --oidc-config-id 233jnu62i9aphpucsj9kueqlkr1vcgra
I: Creating OIDC provider using 'arn:aws:iam::242819244:user/userName'
? Create the OIDC provider? Yes
I: Created OIDC provider with ARN 'arn:aws:iam::242819244:oidc-provider/dvbwgdztaeq9o.cloudfront.net/233jnu62i9aphpucsj9kueqlkr1vcgra'
Copy to Clipboard Toggle word wrap

Service control policies (SCP) are a type of organization policy that manages permissions within your organization. SCPs ensure that accounts within your organization stay within your defined access control guidelines. These policies are maintained in AWS Organizations and control the services that are available within the attached AWS accounts. SCP management is the responsibility of the customer.

Note

When using AWS Security Token Service (STS), you must ensure that the service control policy does not block the following resources:

  • ec2:{}
  • iam:{}
  • tag:*

Verify that your service control policy (SCP) does not restrict any of these required permissions.

Expand
 ServiceActionsEffect

Required

Amazon EC2

All

Allow

Amazon EC2 Auto Scaling

All

Allow

Amazon S3

All

Allow

Identity And Access Management

All

Allow

Elastic Load Balancing

All

Allow

Elastic Load Balancing V2

All

Allow

Amazon CloudWatch

All

Allow

Amazon CloudWatch Events

All

Allow

Amazon CloudWatch Logs

All

Allow

AWS EC2 Instance Connect

SendSerialConsoleSSHPublicKey

Allow

AWS Support

All

Allow

AWS Key Management Service

All

Allow

AWS Security Token Service

All

Allow

AWS Tiro

CreateQuery

GetQueryAnswer

GetQueryExplanation

Allow

AWS Marketplace

Subscribe

Unsubscribe

View Subscriptions

Allow

AWS Resource Tagging

All

Allow

AWS Route53 DNS

All

Allow

AWS Service Quotas

ListServices

GetRequestedServiceQuotaChange

GetServiceQuota

RequestServiceQuotaIncrease

ListServiceQuotas

Allow

Optional

AWS Billing

ViewAccount

Viewbilling

ViewUsage

Allow

AWS Cost and Usage Report

All

Allow

AWS Cost Explorer Services

All

Allow

7.6. Customer-managed policies
Copy link

Red Hat OpenShift Service on AWS (ROSA) users are able to attach customer-managed policies to the IAM roles required to run and maintain ROSA clusters. This capability is not uncommon with AWS IAM roles. The ability to attach these policies to ROSA-specific IAM roles extends a ROSA cluster’s permission capabilities; for example, as a way to allow cluster components to access additional AWS resources that are otherwise not part of the ROSA-specific IAM policies.

To ensure that any critical customer applications that rely on customer-managed policies are not modified in any way during cluster or role upgrades, ROSA utilizes the ListAttachedRolesPolicies permission to retrieve the list of permission policies from roles and the ListRolePolicies permission to retrieve the list of policies from ROSA-specific roles. This information ensures that customer-managed policies are not impacted during cluster events, and allows Red Hat SREs to monitor both ROSA and customer-managed policies attached to ROSA-specific IAM roles, enhancing their ability to troubleshoot any cluster issues more effectively.

Warning

Attaching permission boundary policies to IAM roles that restrict ROSA-specific policies is not supported, as these policies could interrupt the functionality of the basic permissions necessary to successfully run and maintain your ROSA cluster. There are prepared permissions boundary policies for the ROSA (classic architecture) installer role. See the Additional resources section for more information.

Legal Notice
Copy link

Copyright © 2025 Red Hat

OpenShift documentation is licensed under the Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0).

Modified versions must remove all Red Hat trademarks.

Portions adapted from https://github.com/kubernetes-incubator/service-catalog/ with modifications by Red Hat.

Red Hat, Red Hat Enterprise Linux, the Red Hat logo, the Shadowman logo, JBoss, OpenShift, Fedora, the Infinity logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries.

Linux® is the registered trademark of Linus Torvalds in the United States and other countries.

Java® is a registered trademark of Oracle and/or its affiliates.

XFS® is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries.

MySQL® is a registered trademark of MySQL AB in the United States, the European Union and other countries.

Node.js® is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project.

The OpenStack® Word Mark and OpenStack logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation’s permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community.

All other trademarks are the property of their respective owners.

Back to top
Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Theme

© 2025 Red Hat