第 3 章 使用 Terraform 创建 ROSA (经典架构)集群


使用使用默认集群选项配置的 Terraform 集群模板快速创建 {rosa-classic-first} 集群。

以下描述的集群创建过程使用 Terraform 配置来准备带有以下资源的 {rosa-classic} AWS 安全令牌服务(STS)集群:

  • 带有受管 oidc-config 配置的 OIDC 供应商
  • 带有关联的 AWS Managed ROSA 策略的先决条件 IAM Operator 角色
  • 带有关联的 AWS Managed ROSA 策略的 IAM 帐户角色
  • 创建使用 STS 集群的 ROSA 所需的所有其他 AWS 资源

3.1.1. Terraform 概述

Terraform 是一个基础架构即代码工具,提供一次配置资源并根据需要复制这些资源的方法。Terraform 使用声明性语言完成创建任务。您可以声明基础架构资源的最终状态,Terraform 会根据您的规格创建这些资源。

前提条件

要在 Terraform 配置中使用 Red Hat Cloud Services 供应商,您必须满足以下条件:

  • 您已在 AWS (ROSA)命令行界面(CLI)工具上安装了 Red Hat OpenShift Service。
  • 您有离线的 Red Hat OpenShift Cluster Manager 令牌
  • 已安装 Terraform 版本 1.4.6 或更新版本。
  • 您已创建了 AWS 帐户范围的 IAM 角色。

    特定的帐户范围的 IAM 角色和策略提供 ROSA 支持、安装、control plane 和计算功能所需的 STS 权限。这包括集群范围的 Operator 策略。如需有关 AWS 帐户角色的更多信息,请参阅附加资源。

  • 您有一个 AWS 帐户 和相关凭证,供您创建资源。为 AWS 供应商配置了凭证。请参阅 AWS Terraform 供应商文档中的 身份验证和 配置部分。
  • 您至少在 AWS IAM 角色策略中具有以下权限,其运行 Terraform。在 AWS 控制台中检查这些权限。

    例 3.1. Terraform 的最低 AWS 权限

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "iam:GetPolicyVersion",
            "iam:DeletePolicyVersion",
            "iam:CreatePolicyVersion",
            "iam:UpdateAssumeRolePolicy",
            "secretsmanager:DescribeSecret",
            "iam:ListRoleTags",
            "secretsmanager:PutSecretValue",
            "secretsmanager:CreateSecret",
            "iam:TagRole",
            "secretsmanager:DeleteSecret",
            "iam:UpdateOpenIDConnectProviderThumbprint",
            "iam:DeletePolicy",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:ListInstanceProfilesForRole",
            "secretsmanager:GetSecretValue",
            "iam:DetachRolePolicy",
            "iam:ListAttachedRolePolicies",
            "iam:ListPolicyTags",
            "iam:ListRolePolicies",
            "iam:DeleteOpenIDConnectProvider",
            "iam:DeleteInstanceProfile",
            "iam:GetRole",
            "iam:GetPolicy",
            "iam:ListEntitiesForPolicy",
            "iam:DeleteRole",
            "iam:TagPolicy",
            "iam:CreateOpenIDConnectProvider",
            "iam:CreatePolicy",
            "secretsmanager:GetResourcePolicy",
            "iam:ListPolicyVersions",
            "iam:UpdateRole",
            "iam:GetOpenIDConnectProvider",
            "iam:TagOpenIDConnectProvider",
            "secretsmanager:TagResource",
            "sts:AssumeRoleWithWebIdentity",
            "iam:ListRoles"
          ],
          "Resource": [
            "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*",
            "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*",
            "arn:aws:iam::<ACCOUNT_ID>:role/*",
            "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*",
            "arn:aws:iam::<ACCOUNT_ID>:policy/*"
          ]
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": [
            "s3:*"
            ],
          "Resource": "*"
        }
      ]
    }
    Copy to Clipboard Toggle word wrap

使用 Terraform 时的注意事项

通常,使用 Terraform 管理云资源应按照预期完成任何更改,以便使用 Terraform 方法进行任何更改。在使用 Terraform 之外的工具(如 AWS 控制台或红帽控制台)时,请小心修改 Terraform 创建的云资源。使用 Terraform 以外的工具来管理已经由 Terraform 管理的云资源,从您声明的 Terraform 配置中引入配置偏移。

例如,如果您使用 Red Hat Hybrid Cloud Console 升级 Terraform 创建的集群,则需要在应用任何受影响的配置更改前协调 Terraform 状态。如需更多信息,请参阅 HashiCorp Developer 文档中的管理 Terraform 状态的资源

3.1.2. 默认集群规格概述

Expand
表 3.1. 使用 STS 集群规格的默认 ROSA
组件默认规格

帐户和角色

  • 默认 IAM 角色前缀: rosa-<6-digit-alphanumeric-string>

集群设置

  • 默认集群版本: 4.14
  • Cluster name: rosa-<6-digit-alphanumeric-string>
  • 使用 Red Hat OpenShift Cluster Manager 混合云控制台安装的默认 AWS 区域:us-east-2 (US East, Ohio)
  • 可用性:数据平面的多个区域
  • EC2 实例元数据服务(IMDS)已启用,并允许使用 IMDSv1 或 IMDSv2 (令牌可选)
  • 可用性: data plane 的单一区
  • 监控用户定义的项目:启用
  • 没有创建集群管理员角色

control plane 节点配置

  • control plane 节点实例类型:m5.2xlarge (8 vCPU, 32 GiB RAM)
  • control plane 节点数: 3

Compute 节点机器池

  • Compute 节点实例类型: m5.xlarge (4 vCPU 16, GiB RAM)
  • Compute 节点数: 3 个
  • 自动扩展:未启用
  • 没有额外的节点标签

网络配置

  • 集群隐私:公共或私有
  • 您可以选择在 Terraform 集群创建过程中创建新 VPC。
  • 您必须已经配置自己的 Virtual Private Cloud (VPC)
  • 没有配置集群范围的代理

无类别域间路由 (CIDR) 范围

  • Machine CIDR: 10.0.0.0/16
  • Service CIDR: 172.30.0.0/16
  • Pod CIDR: 10.128.0.0/14
  • 主机前缀:/23

    注意

    为内部 Kubernetes API 地址保留静态 IP 地址 172.20.0.1。机器、pod 和服务 CIDR 范围不得与此 IP 地址冲突。

集群角色和策略

  • 用于创建 Operator 角色和 OpenID Connect(OIDC)供应商的模式: auto

    注意

    对于在混合云控制台上使用 OpenShift Cluster Manager 的安装,自动 模式需要 admin-privileged OpenShift Cluster Manager 角色(ocm-role)。

  • 默认 Operator 角色前缀: rosa-<6-digit-alphanumeric-string>

存储

  • 节点卷:

    • 类型:AWS EBS GP3
    • 默认大小:300GiB (可在创建时调整)
  • 工作负载持久性卷:

    • 默认 StorageClass: gp3-csi
    • provisioner: ebs.csi.aws.com
    • 动态持久性卷置备

集群更新策略

  • 独立更新
  • 节点排空 1 小时宽限期

以下概述的集群创建过程演示了如何使用 Terraform 创建您的帐户范围的 IAM 角色和带有受管 OIDC 配置的 {rosa-classic} 集群。

3.1.3.1. 为 Terraform 准备您的环境

在使用 Terraform 创建 ROSA (经典)集群前,您需要导出 离线 Red Hat OpenShift Cluster Manager 令牌

流程

  1. 可选: 因为在安装过程中在当前目录中创建 Terraform 文件,所以您可以创建一个新的目录来存储这些文件并导航到其中:

    $ mkdir terraform-cluster && cd terraform-cluster
    Copy to Clipboard Toggle word wrap
  2. 使用 离线 Red Hat OpenShift Cluster Manager 令牌 向您的帐户授予权限。
  3. 运行以下命令复制离线令牌,并将令牌设置为环境变量:

    $ export RHCS_TOKEN=<your_offline_token>
    Copy to Clipboard Toggle word wrap
    注意

    此环境变量会在每个会话的末尾重置,如重启计算机或关闭终端。

验证

  • 导出令牌后,运行以下命令来验证值:

    $ echo $RHCS_TOKEN
    Copy to Clipboard Toggle word wrap

3.1.3.2. 在本地创建 Terraform 文件

设置 离线 Red Hat OpenShift Cluster Manager 令牌 后,您需要在本地创建 Terraform 文件以构建集群。您可以使用以下代码模板创建这些文件。

流程

  1. 运行以下命令来创建 main.tf 文件:

    $ cat<<-EOF>main.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = ">= 4.20.0"
        }
        rhcs = {
          version = ">= 1.6.2"
          source  = "terraform-redhat/rhcs"
        }
      }
    }
    
    # Export token using the RHCS_TOKEN environment variable
    provider "rhcs" {}
    
    provider "aws" {
      region = var.aws_region
      ignore_tags {
        key_prefixes = ["kubernetes.io/"]
      }
      default_tags {
        tags = var.default_aws_tags
      }
    }
    
    data "aws_availability_zones" "available" {}
    
    locals {
      # The default setting creates 3 availability zones. Set to "false" to create a single availability zones.
      region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1)
    }
    
    resource "random_string" "random_name" {
      length  = 6
      special = false
      upper   = false
    }
    
    locals {
      path                 = coalesce(var.path, "/")
      worker_node_replicas = try(var.worker_node_replicas, var.multi_az ? 3 : 2)
      # If cluster_name is not null, use that, otherwise generate a random cluster name
      cluster_name = coalesce(var.cluster_name, "rosa-\${random_string.random_name.result}")
    }
    
    # The network validator requires an additional 60 seconds to validate Terraform clusters.
    resource "time_sleep" "wait_60_seconds" {
      count = var.create_vpc ? 1 : 0
      depends_on = [module.vpc]
      create_duration = "60s"
    }
    
    module "rosa-classic" {
      source                 = "terraform-redhat/rosa-classic/rhcs"
      version                = "1.5.0"
      cluster_name           = local.cluster_name
      openshift_version      = var.openshift_version
      account_role_prefix    = local.cluster_name
      operator_role_prefix   = local.cluster_name
      replicas               = local.worker_node_replicas
      aws_availability_zones = local.region_azs
      create_oidc            = true
      private                = var.private_cluster
      aws_private_link       = var.private_cluster
      aws_subnet_ids         = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids
      multi_az               = var.multi_az
      create_account_roles   = true
      create_operator_roles  = true
    # Optional: Configure a cluster administrator user \ 
    1
    
    #
    # Option 1: Default cluster-admin user
    # Create an administrator user (cluster-admin) and automatically
    # generate a password by uncommenting the following parameter:
    #  create_admin_user = true
    # Generated administrator credentials are displayed in terminal output.
    #
    # Option 2: Specify administrator username and password
    # Create an administrator user and define your own password
    # by uncommenting and editing the values of the following parameters:
    #  admin_credentials_username = <username>
    #  admin_credentials_password = <password>
    
      depends_on = [time_sleep.wait_60_seconds]
    }
    EOF
    Copy to Clipboard Toggle word wrap
    1
    可选:通过取消注释适当的参数并编辑其值,在集群创建过程中创建管理员用户。
  2. 运行以下命令来创建 variables.tf 文件:

    注意

    在运行该命令 以构建 集群前复制并编辑此文件。

    $ cat<<-EOF>variables.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    variable "openshift_version" {
      type        = string
      default     = "4.14.20"
      description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled."
    }
    
    variable "create_vpc" {
      type        = bool
      description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'."
    }
    
    # ROSA Cluster info
    variable "cluster_name" {
      default     = null
      type        = string
      description = "The name of the ROSA cluster to create"
    }
    
    variable "additional_tags" {
      default = {
        Terraform   = "true"
        Environment = "dev"
      }
      description = "Additional AWS resource tags"
      type        = map(string)
    }
    
    variable "path" {
      description = "(Optional) The arn path for the account/operator roles as well as their policies."
      type        = string
      default     = null
    }
    
    variable "multi_az" {
      type        = bool
      description = "Multi AZ Cluster for High Availability"
      default     = true
    }
    
    variable "worker_node_replicas" {
      default     = 3
      description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes"
      type        = number
    }
    
    variable "aws_subnet_ids" {
      type        = list(any)
      description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster"
      default     = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"]
    }
    
    variable "private_cluster" {
      type        = bool
      description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'."
    }
    
    #VPC Info
    variable "vpc_name" {
      type        = string
      description = "VPC Name"
      default     = "tf-qs-vpc"
    }
    
    variable "vpc_cidr_block" {
      type        = string
      description = "value of the CIDR block to use for the VPC"
      default     = "10.0.0.0/16"
    }
    
    variable "private_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the private subnets"
      default     = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    }
    
    variable "public_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the public subnets"
      default     = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
    }
    
    variable "single_nat_gateway" {
      type        = bool
      description = "Single NAT or per NAT for subnet"
      default     = false
    }
    
    #AWS Info
    variable "aws_region" {
      type    = string
      default = "us-east-2"
    }
    
    variable "default_aws_tags" {
      type        = map(string)
      description = "Default tags for AWS"
      default     = {}
    }
    EOF
    Copy to Clipboard Toggle word wrap
  3. 运行以下命令来创建 vpc.tf 文件:

    $ cat<<-EOF>vpc.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = "5.1.2"
    
      count = var.create_vpc ? 1 : 0
      name  = var.vpc_name
      cidr  = var.vpc_cidr_block
    
      azs             = local.region_azs
      private_subnets = var.private_subnet_cidrs
      public_subnets  = var.public_subnet_cidrs
    
      enable_nat_gateway   = true
      single_nat_gateway   = var.single_nat_gateway
      enable_dns_hostnames = true
      enable_dns_support   = true
    
      tags = var.additional_tags
    }
    EOF
    Copy to Clipboard Toggle word wrap

    您已准备好启动 Terraform。

3.1.3.3. 使用 Terraform 创建 ROSA 集群

创建 Terraform 文件后,您必须启动 Terraform 以提供所有所需的依赖项。然后应用 Terraform 计划。

重要

不要修改 Terraform 状态文件。如需更多信息,请参阅使用 Terraform 时的注意事项

流程

  1. 将 Terraform 设置为根据您的 Terraform 文件创建资源,运行以下命令:

    $ terraform init
    Copy to Clipboard Toggle word wrap
  2. 可选 :运行以下命令来验证您复制的 Terraform 是否正确:

    $ terraform validate
    Copy to Clipboard Toggle word wrap

    输出示例

    Success! The configuration is valid.
    Copy to Clipboard Toggle word wrap

  3. 运行以下命令,使用 Terraform 创建集群:

    $ terraform apply
    Copy to Clipboard Toggle word wrap

    Terraform 界面需要两个问题来创建集群,如下所示:

    输出示例

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'.
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'.
    
      Enter a value:
    Copy to Clipboard Toggle word wrap

  4. 当 Terraform 界面列出要创建或修改的资源并提示确认时,输入 yes 才能继续或取消:

    输出示例

    Plan: 74 to add, 0 to change, 0 to destroy.
    
    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.
    
      Enter a value: yes
    Copy to Clipboard Toggle word wrap

    如果输入 yes,您的 Terraform 计划将启动,创建 AWS 帐户角色、Operator 角色和 ROSA Classic 集群。

验证

  1. 运行以下命令验证集群是否已创建:

    $ rosa list clusters
    Copy to Clipboard Toggle word wrap

    显示集群的 ID、名称和状态的输出示例

    ID                                NAME          STATE  TOPOLOGY
    27c3snjsupa9obua74ba8se5kcj11269  rosa-tf-demo  ready  Classic (STS)
    Copy to Clipboard Toggle word wrap

  2. 运行以下命令验证您的帐户角色是否已创建:

    $ rosa list account-roles
    Copy to Clipboard Toggle word wrap

    输出示例

    I: Fetching account roles
    ROLE NAME                                   ROLE TYPE      ROLE ARN                                                           OPENSHIFT VERSION  AWS Managed
    ROSA-demo-ControlPlane-Role                 Control plane  arn:aws:iam::<ID>:role/ROSA-demo-ControlPlane-Role                 4.14               No
    ROSA-demo-Installer-Role                    Installer      arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role                    4.14               No
    ROSA-demo-Support-Role                      Support        arn:aws:iam::<ID>:role/ROSA-demo-Support-Role                      4.14               No
    ROSA-demo-Worker-Role                       Worker         arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role                       4.14               No
    Copy to Clipboard Toggle word wrap

  3. 运行以下命令验证您的 Operator 角色是否已创建:

    $ rosa list operator-roles
    Copy to Clipboard Toggle word wrap

    显示 Terraform 创建的 Operator 角色的输出示例

    I: Fetching operator roles
    ROLE PREFIX    AMOUNT IN BUNDLE
    rosa-demo      6
    Copy to Clipboard Toggle word wrap

3.1.3.4. 使用 Terraform 删除 ROSA 集群

使用 terraform destroy 命令删除通过 terraform apply 命令创建的所有资源。

注意

在销毁资源前,不要修改 Terraform .tf 文件。这些变量与要删除的资源匹配。

流程

  1. 在运行 terraform apply 命令创建集群的目录中,运行以下命令删除集群:

    $ terraform destroy
    Copy to Clipboard Toggle word wrap

    Terraform 接口提示您输入两个变量。它们应与创建集群时提供的答案匹配:

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.'
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.'
    
      Enter a value:
    Copy to Clipboard Toggle word wrap
  2. 输入 yes 以启动角色和集群删除:

    输出示例

    Plan: 0 to add, 0 to change, 74 to destroy.
    
    Do you really want to destroy all resources?
      Terraform will destroy all your managed infrastructure, as shown above.
      There is no undo. Only 'yes' will be accepted to confirm.
    
      Enter a value: yes
    Copy to Clipboard Toggle word wrap

验证

  1. 运行以下命令验证集群是否已销毁:

    $ rosa list clusters
    Copy to Clipboard Toggle word wrap

    没有显示集群的输出示例

    I: No clusters available
    Copy to Clipboard Toggle word wrap

  2. 运行以下命令,验证帐户角色是否已销毁:

    $ rosa list account-roles
    Copy to Clipboard Toggle word wrap

    没有显示 Terraform 创建帐户角色的输出示例

    I: Fetching account roles
    I: No account roles available
    Copy to Clipboard Toggle word wrap

  3. 运行以下命令,验证 Operator 角色是否已销毁:

    $ rosa list operator-roles
    Copy to Clipboard Toggle word wrap

    输出显示没有 Terraform 创建的 Operator 角色的示例

    I: Fetching operator roles
    I: No operator roles available
    Copy to Clipboard Toggle word wrap

返回顶部
Red Hat logoGithubredditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务,以及可以信赖的内容,帮助红帽用户创新并实现他们的目标。 了解我们当前的更新.

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情,请参阅红帽博客.

關於紅帽

我们提供强化的解决方案,使企业能够更轻松地跨平台和环境(从核心数据中心到网络边缘)工作。

Theme

© 2025 Red Hat