搜索

第 3 章 使用 Terraform 创建 ROSA (经典架构)集群

download PDF

3.1. 使用 Terraform 创建默认 ROSA (经典架构)集群

使用使用默认集群选项配置的 Terraform 集群模板快速创建 Red Hat OpenShift Service on AWS (ROSA) (经典架构)集群。

以下描述的集群创建过程使用 Terraform 配置来准备带有以下资源的 ROSA (经典架构)AWS 安全令牌服务(STS)集群:

  • 带有受管 oidc-config 配置的 OIDC 供应商
  • 带有关联的 AWS Managed ROSA 策略的先决条件 IAM Operator 角色
  • 带有关联的 AWS Managed ROSA 策略的 IAM 帐户角色
  • 创建使用 STS 集群的 ROSA 所需的所有其他 AWS 资源

3.1.1. Terraform 概述

Terraform 是一个基础架构即代码工具,提供一次配置资源并根据需要复制这些资源的方法。Terraform 使用声明性语言完成创建任务。您可以声明基础架构资源的最终状态,Terraform 会根据您的规格创建这些资源。

前提条件

要在 Terraform 配置中使用 Red Hat Cloud Services 供应商,您必须满足以下条件:

  • 您已在 AWS (ROSA)命令行界面(CLI)工具上安装了 Red Hat OpenShift Service。
  • 您有离线的 Red Hat OpenShift Cluster Manager 令牌
  • 已安装 Terraform 版本 1.4.6 或更新版本。
  • 您已创建了 AWS 帐户范围的 IAM 角色。

    特定的帐户范围的 IAM 角色和策略提供 ROSA 支持、安装、control plane 和计算功能所需的 STS 权限。这包括集群范围的 Operator 策略。如需有关 AWS 帐户角色的更多信息,请参阅附加资源。

  • 您有一个 AWS 帐户 和相关凭证,供您创建资源。为 AWS 供应商配置了凭证。请参阅 AWS Terraform 供应商文档中的 身份验证和 配置部分。
  • 您至少在 AWS IAM 角色策略中具有以下权限,其运行 Terraform。在 AWS 控制台中检查这些权限。

    例 3.1. Terraform 的最低 AWS 权限

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
            "iam:GetPolicyVersion",
            "iam:DeletePolicyVersion",
            "iam:CreatePolicyVersion",
            "iam:UpdateAssumeRolePolicy",
            "secretsmanager:DescribeSecret",
            "iam:ListRoleTags",
            "secretsmanager:PutSecretValue",
            "secretsmanager:CreateSecret",
            "iam:TagRole",
            "secretsmanager:DeleteSecret",
            "iam:UpdateOpenIDConnectProviderThumbprint",
            "iam:DeletePolicy",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:ListInstanceProfilesForRole",
            "secretsmanager:GetSecretValue",
            "iam:DetachRolePolicy",
            "iam:ListAttachedRolePolicies",
            "iam:ListPolicyTags",
            "iam:ListRolePolicies",
            "iam:DeleteOpenIDConnectProvider",
            "iam:DeleteInstanceProfile",
            "iam:GetRole",
            "iam:GetPolicy",
            "iam:ListEntitiesForPolicy",
            "iam:DeleteRole",
            "iam:TagPolicy",
            "iam:CreateOpenIDConnectProvider",
            "iam:CreatePolicy",
            "secretsmanager:GetResourcePolicy",
            "iam:ListPolicyVersions",
            "iam:UpdateRole",
            "iam:GetOpenIDConnectProvider",
            "iam:TagOpenIDConnectProvider",
            "secretsmanager:TagResource",
            "sts:AssumeRoleWithWebIdentity",
            "iam:ListRoles"
          ],
          "Resource": [
            "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*",
            "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*",
            "arn:aws:iam::<ACCOUNT_ID>:role/*",
            "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*",
            "arn:aws:iam::<ACCOUNT_ID>:policy/*"
          ]
        },
        {
          "Sid": "VisualEditor1",
          "Effect": "Allow",
          "Action": [
            "s3:*"
            ],
          "Resource": "*"
        }
      ]
    }

使用 Terraform 时的注意事项

通常,使用 Terraform 管理云资源应按照预期完成任何更改,以便使用 Terraform 方法进行任何更改。在使用 Terraform 之外的工具(如 AWS 控制台或红帽控制台)时,请小心修改 Terraform 创建的云资源。使用 Terraform 以外的工具来管理已经由 Terraform 管理的云资源,从您声明的 Terraform 配置中引入配置偏移。

例如,如果您使用 Red Hat Hybrid Cloud Console 升级 Terraform 创建的集群,则需要在应用任何受影响的配置更改前协调 Terraform 状态。如需更多信息,请参阅 HashiCorp Developer 文档中的管理 Terraform 状态的资源

3.1.2. 默认集群规格概述

表 3.1. 使用 STS 集群规格的默认 ROSA
组件默认规格

帐户和角色

  • 默认 IAM 角色前缀: rosa-<6-digit-alphanumeric-string>
  • 没有创建集群管理员角色

集群设置

  • 默认集群版本: 4.14
  • Cluster name: rosa-<6-digit-alphanumeric-string>
  • 使用 Red Hat OpenShift Cluster Manager 混合云控制台安装的默认 AWS 区域:us-east-2 (US East, Ohio)
  • 可用性:数据平面的多个区域
  • 启用默认的 EC2 IMDS 端点(v1 和 v2)
  • 监控用户定义的项目:启用

Encryption

  • 云存储会加密
  • 没有启用额外的 etcd 加密
  • 默认 AWS 密钥管理服务(KMS)密钥用作持久数据的加密密钥

control plane 节点配置

  • control plane 节点实例类型:m5.2xlarge (8 vCPU, 32 GiB RAM)
  • control plane 节点数: 3

基础架构节点配置

  • 基础架构节点实例类型:r5.xlarge (4 vCPU, 32 GiB RAM)
  • 基础架构节点数: 2

Compute 节点机器池

  • Compute 节点实例类型: m5.xlarge (4 vCPU 16, GiB RAM)
  • Compute 节点数: 3 个
  • 自动扩展:未启用
  • 没有额外的节点标签

网络配置

  • 集群隐私:公共或私有
  • 您可以选择在 Terraform 集群创建过程中创建新 VPC。
  • 没有配置集群范围的代理

无类别域间路由 (CIDR) 范围

  • Machine CIDR: 10.0.0.0/16
  • Service CIDR: 172.30.0.0/16
  • Pod CIDR: 10.128.0.0/14
  • 主机前缀:/23

集群角色和策略

  • 用于创建 Operator 角色和 OpenID Connect(OIDC)供应商的模式: auto

    注意

    对于在混合云控制台上使用 OpenShift Cluster Manager 的安装,自动 模式需要管理员特权的 OpenShift Cluster Manager 角色。

  • 默认 Operator 角色前缀: rosa-<6-digit-alphanumeric-string>

集群更新策略

  • 独立更新
  • 节点排空 1 小时宽限期

3.1.3. 使用 Terraform 创建默认 ROSA (经典架构)集群

以下概述的集群创建过程演示了如何使用 Terraform 创建帐户范围的 IAM 角色和带有受管 OIDC 配置的 ROSA (经典架构)集群。

3.1.3.1. 为 Terraform 准备您的环境

在使用 Terraform 在 AWS 集群上创建 Red Hat OpenShift Service 前,您需要导出 离线 Red Hat OpenShift Cluster Manager 令牌

流程

  1. 可选: 因为在安装过程中在当前目录中创建 Terraform 文件,所以您可以创建一个新的目录来存储这些文件并导航到其中:

    $ mkdir terraform-cluster && cd terraform-cluster
  2. 使用 离线 Red Hat OpenShift Cluster Manager 令牌 向您的帐户授予权限。
  3. 运行以下命令复制离线令牌,并将令牌设置为环境变量:

    $ export RHCS_TOKEN=<your_offline_token>
    注意

    此环境变量会在每个会话的末尾重置,如重启计算机或关闭终端。

验证

  • 导出令牌后,运行以下命令来验证值:

    $ echo $RHCS_TOKEN

3.1.3.2. 在本地创建 Terraform 文件

设置 离线 Red Hat OpenShift Cluster Manager 令牌 后,您需要在本地创建 Terraform 文件以构建集群。您可以使用以下代码模板创建这些文件。

流程

  1. 运行以下命令来创建 main.tf 文件:

    $ cat<<-EOF>main.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    
    terraform {
      required_providers {
        aws = {
          source  = "hashicorp/aws"
          version = ">= 4.20.0"
        }
        rhcs = {
          version = ">= 1.6.2"
          source  = "terraform-redhat/rhcs"
        }
      }
    }
    
    # Export token using the RHCS_TOKEN environment variable
    provider "rhcs" {}
    
    provider "aws" {
      region = var.aws_region
      ignore_tags {
        key_prefixes = ["kubernetes.io/"]
      }
      default_tags {
        tags = var.default_aws_tags
      }
    }
    
    data "aws_availability_zones" "available" {}
    
    locals {
      # The default setting creates 3 availability zones. Set to "false" to create a single availability zones.
      region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1)
    }
    
    resource "random_string" "random_name" {
      length  = 6
      special = false
      upper   = false
    }
    
    locals {
      path                 = coalesce(var.path, "/")
      worker_node_replicas = try(var.worker_node_replicas, var.multi_az ? 3 : 2)
      # If cluster_name is not null, use that, otherwise generate a random cluster name
      cluster_name = coalesce(var.cluster_name, "rosa-\${random_string.random_name.result}")
    }
    
    # The network validator requires an additional 60 seconds to validate Terraform clusters.
    resource "time_sleep" "wait_60_seconds" {
      count = var.create_vpc ? 1 : 0
      depends_on = [module.vpc]
      create_duration = "60s"
    }
    
    module "rosa-classic" {
      source                 = "terraform-redhat/rosa-classic/rhcs"
      version                = "1.5.0"
      cluster_name           = local.cluster_name
      openshift_version      = var.openshift_version
      account_role_prefix    = local.cluster_name
      operator_role_prefix   = local.cluster_name
      replicas               = local.worker_node_replicas
      aws_availability_zones = local.region_azs
      create_oidc            = true
      private                = var.private_cluster
      aws_private_link       = var.private_cluster
      aws_subnet_ids         = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids
      multi_az               = var.multi_az
      create_account_roles   = true
      create_operator_roles  = true
    
      depends_on = [time_sleep.wait_60_seconds]
    }
    EOF
  2. 运行以下命令来创建 variables.tf 文件:

    注意

    在运行该命令 以构建 集群前复制并编辑此文件。

    $ cat<<-EOF>variables.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    variable "openshift_version" {
      type        = string
      default     = "4.14.20"
      description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled."
    }
    
    variable "create_vpc" {
      type        = bool
      description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'."
    }
    
    # ROSA Cluster info
    variable "cluster_name" {
      default     = null
      type        = string
      description = "The name of the ROSA cluster to create"
    }
    
    variable "additional_tags" {
      default = {
        Terraform   = "true"
        Environment = "dev"
      }
      description = "Additional AWS resource tags"
      type        = map(string)
    }
    
    variable "path" {
      description = "(Optional) The arn path for the account/operator roles as well as their policies."
      type        = string
      default     = null
    }
    
    variable "multi_az" {
      type        = bool
      description = "Multi AZ Cluster for High Availability"
      default     = true
    }
    
    variable "worker_node_replicas" {
      default     = 3
      description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes"
      type        = number
    }
    
    variable "aws_subnet_ids" {
      type        = list(any)
      description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster"
      default     = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"]
    }
    
    variable "private_cluster" {
      type        = bool
      description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'."
    }
    
    #VPC Info
    variable "vpc_name" {
      type        = string
      description = "VPC Name"
      default     = "tf-qs-vpc"
    }
    
    variable "vpc_cidr_block" {
      type        = string
      description = "value of the CIDR block to use for the VPC"
      default     = "10.0.0.0/16"
    }
    
    variable "private_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the private subnets"
      default     = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
    }
    
    variable "public_subnet_cidrs" {
      type        = list(any)
      description = "The CIDR blocks to use for the public subnets"
      default     = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]
    }
    
    variable "single_nat_gateway" {
      type        = bool
      description = "Single NAT or per NAT for subnet"
      default     = false
    }
    
    #AWS Info
    variable "aws_region" {
      type    = string
      default = "us-east-2"
    }
    
    variable "default_aws_tags" {
      type        = map(string)
      description = "Default tags for AWS"
      default     = {}
    }
    EOF
  3. 运行以下命令来创建 vpc.tf 文件:

    $ cat<<-EOF>vpc.tf
    #
    # Copyright (c) 2023 Red Hat, Inc.
    #
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #   http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    #
    module "vpc" {
      source  = "terraform-aws-modules/vpc/aws"
      version = "5.1.2"
    
      count = var.create_vpc ? 1 : 0
      name  = var.vpc_name
      cidr  = var.vpc_cidr_block
    
      azs             = local.region_azs
      private_subnets = var.private_subnet_cidrs
      public_subnets  = var.public_subnet_cidrs
    
      enable_nat_gateway   = true
      single_nat_gateway   = var.single_nat_gateway
      enable_dns_hostnames = true
      enable_dns_support   = true
    
      tags = var.additional_tags
    }
    EOF

    您已准备好启动 Terraform。

3.1.3.3. 使用 Terraform 创建 ROSA 集群

创建 Terraform 文件后,您必须启动 Terraform 以提供所有所需的依赖项。然后应用 Terraform 计划。

重要

不要修改 Terraform 状态文件。如需更多信息,请参阅使用 Terraform 时的注意事项

流程

  1. 将 Terraform 设置为根据您的 Terraform 文件创建资源,运行以下命令:

    $ terraform init
  2. 可选 :运行以下命令来验证您复制的 Terraform 是否正确:

    $ terraform validate

    输出示例

    Success! The configuration is valid.

  3. 运行以下命令,使用 Terraform 创建集群:

    $ terraform apply

    Terraform 界面需要两个问题来创建集群,并熟悉以下内容:

    输出示例

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'.
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'.
    
      Enter a value:

  4. 当 Terraform 界面列出要创建或修改的资源并提示确认时,输入 yes 才能继续或取消:

    输出示例

    Plan: 74 to add, 0 to change, 0 to destroy.
    
    Do you want to perform these actions?
      Terraform will perform the actions described above.
      Only 'yes' will be accepted to approve.
    
      Enter a value: yes

    如果输入 yes,您的 Terraform 计划将启动,创建 AWS 帐户角色、Operator 角色和 ROSA Classic 集群。

验证

  1. 运行以下命令验证集群是否已创建:

    $ rosa list clusters

    显示集群的 ID、名称和状态的输出示例:

    ID                                NAME          STATE  TOPOLOGY
    27c3snjsupa9obua74ba8se5kcj11269  rosa-tf-demo  ready  Classic (STS)

  2. 运行以下命令验证您的帐户角色是否已创建:

    $ rosa list account-roles

    输出示例

    I: Fetching account roles
    ROLE NAME                                   ROLE TYPE      ROLE ARN                                                           OPENSHIFT VERSION  AWS Managed
    ROSA-demo-ControlPlane-Role                 Control plane  arn:aws:iam::<ID>:role/ROSA-demo-ControlPlane-Role                 4.14               No
    ROSA-demo-Installer-Role                    Installer      arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role                    4.14               No
    ROSA-demo-Support-Role                      Support        arn:aws:iam::<ID>:role/ROSA-demo-Support-Role                      4.14               No
    ROSA-demo-Worker-Role                       Worker         arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role                       4.14               No

  3. 运行以下命令验证您的 Operator 角色是否已创建:

    $ rosa list operator-roles

    显示 Terraform 创建的 Operator 角色的输出示例:

    I: Fetching operator roles
    ROLE PREFIX    AMOUNT IN BUNDLE
    rosa-demo      6

3.1.3.4. 使用 Terraform 删除 ROSA 集群

使用 terraform destroy 命令删除通过 terraform apply 命令创建的所有资源。

注意

在销毁资源前,不要修改 Terraform .tf 文件。这些变量与要删除的资源匹配。

流程

  1. 在运行 terraform apply 命令创建集群的目录中,运行以下命令删除集群:

    $ terraform destroy

    Terraform 接口提示您输入两个变量。它们应与创建集群时提供的答案匹配:

    var.create_vpc
      If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.'
    
      Enter a value:
    
    var.private_cluster
      If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.'
    
      Enter a value:
  2. 输入 yes 以启动角色和集群删除:

    输出示例

    Plan: 0 to add, 0 to change, 74 to destroy.
    
    Do you really want to destroy all resources?
      Terraform will destroy all your managed infrastructure, as shown above.
      There is no undo. Only 'yes' will be accepted to confirm.
    
      Enter a value: yes

验证

  1. 运行以下命令验证集群是否已销毁:

    $ rosa list clusters

    没有显示集群的输出示例

    I: No clusters available

  2. 运行以下命令,验证帐户角色是否已销毁:

    $ rosa list account-roles

    显示没有 Terraform 创建的帐户角色的输出示例:

    I: Fetching account roles
    I: No account roles available

  3. 运行以下命令,验证 Operator 角色是否已销毁:

    $ rosa list operator-roles

    显示没有由 Terraform 创建的 Operator 角色的输出示例:

    I: Fetching operator roles
    I: No operator roles available

3.1.4. 其他资源

Red Hat logoGithubRedditYoutubeTwitter

学习

尝试、购买和销售

社区

关于红帽文档

通过我们的产品和服务,以及可以信赖的内容,帮助红帽用户创新并实现他们的目标。

让开源更具包容性

红帽致力于替换我们的代码、文档和 Web 属性中存在问题的语言。欲了解更多详情,请参阅红帽博客.

關於紅帽

我们提供强化的解决方案,使企业能够更轻松地跨平台和环境(从核心数据中心到网络边缘)工作。

© 2024 Red Hat, Inc.