第 3 章 使用 Terraform 创建 ROSA (经典架构)集群
3.1. 使用 Terraform 创建默认 ROSA (经典架构)集群
使用使用默认集群选项配置的 Terraform 集群模板快速创建 Red Hat OpenShift Service on AWS (ROSA) (经典架构)集群。
以下描述的集群创建过程使用 Terraform 配置来准备带有以下资源的 ROSA (经典架构)AWS 安全令牌服务(STS)集群:
-
带有受管
oidc-config
配置的 OIDC 供应商 - 带有关联的 AWS Managed ROSA 策略的先决条件 IAM Operator 角色
- 带有关联的 AWS Managed ROSA 策略的 IAM 帐户角色
- 创建使用 STS 集群的 ROSA 所需的所有其他 AWS 资源
3.1.1. Terraform 概述
Terraform 是一个基础架构即代码工具,提供一次配置资源并根据需要复制这些资源的方法。Terraform 使用声明性语言完成创建任务。您可以声明基础架构资源的最终状态,Terraform 会根据您的规格创建这些资源。
前提条件
要在 Terraform 配置中使用 Red Hat Cloud Services 供应商,您必须满足以下条件:
- 您已在 AWS (ROSA)命令行界面(CLI)工具上安装了 Red Hat OpenShift Service。
- 您有离线的 Red Hat OpenShift Cluster Manager 令牌。
- 已安装 Terraform 版本 1.4.6 或更新版本。
您已创建了 AWS 帐户范围的 IAM 角色。
特定的帐户范围的 IAM 角色和策略提供 ROSA 支持、安装、control plane 和计算功能所需的 STS 权限。这包括集群范围的 Operator 策略。如需有关 AWS 帐户角色的更多信息,请参阅附加资源。
- 您有一个 AWS 帐户 和相关凭证,供您创建资源。为 AWS 供应商配置了凭证。请参阅 AWS Terraform 供应商文档中的 身份验证和 配置部分。
您至少在 AWS IAM 角色策略中具有以下权限,其运行 Terraform。在 AWS 控制台中检查这些权限。
例 3.1. Terraform 的最低 AWS 权限
{ "Version": "2012-10-17", "Statement": [ { "Sid": "VisualEditor0", "Effect": "Allow", "Action": [ "iam:GetPolicyVersion", "iam:DeletePolicyVersion", "iam:CreatePolicyVersion", "iam:UpdateAssumeRolePolicy", "secretsmanager:DescribeSecret", "iam:ListRoleTags", "secretsmanager:PutSecretValue", "secretsmanager:CreateSecret", "iam:TagRole", "secretsmanager:DeleteSecret", "iam:UpdateOpenIDConnectProviderThumbprint", "iam:DeletePolicy", "iam:CreateRole", "iam:AttachRolePolicy", "iam:ListInstanceProfilesForRole", "secretsmanager:GetSecretValue", "iam:DetachRolePolicy", "iam:ListAttachedRolePolicies", "iam:ListPolicyTags", "iam:ListRolePolicies", "iam:DeleteOpenIDConnectProvider", "iam:DeleteInstanceProfile", "iam:GetRole", "iam:GetPolicy", "iam:ListEntitiesForPolicy", "iam:DeleteRole", "iam:TagPolicy", "iam:CreateOpenIDConnectProvider", "iam:CreatePolicy", "secretsmanager:GetResourcePolicy", "iam:ListPolicyVersions", "iam:UpdateRole", "iam:GetOpenIDConnectProvider", "iam:TagOpenIDConnectProvider", "secretsmanager:TagResource", "sts:AssumeRoleWithWebIdentity", "iam:ListRoles" ], "Resource": [ "arn:aws:secretsmanager:*:<ACCOUNT_ID>:secret:*", "arn:aws:iam::<ACCOUNT_ID>:instance-profile/*", "arn:aws:iam::<ACCOUNT_ID>:role/*", "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/*", "arn:aws:iam::<ACCOUNT_ID>:policy/*" ] }, { "Sid": "VisualEditor1", "Effect": "Allow", "Action": [ "s3:*" ], "Resource": "*" } ] }
使用 Terraform 时的注意事项
通常,使用 Terraform 管理云资源应按照预期完成任何更改,以便使用 Terraform 方法进行任何更改。在使用 Terraform 之外的工具(如 AWS 控制台或红帽控制台)时,请小心修改 Terraform 创建的云资源。使用 Terraform 以外的工具来管理已经由 Terraform 管理的云资源,从您声明的 Terraform 配置中引入配置偏移。
例如,如果您使用 Red Hat Hybrid Cloud Console 升级 Terraform 创建的集群,则需要在应用任何受影响的配置更改前协调 Terraform 状态。如需更多信息,请参阅 HashiCorp Developer 文档中的管理 Terraform 状态的资源。
3.1.2. 默认集群规格概述
组件 | 默认规格 |
---|---|
帐户和角色 |
|
集群设置 |
|
Encryption |
|
control plane 节点配置 |
|
基础架构节点配置 |
|
Compute 节点机器池 |
|
网络配置 |
|
无类别域间路由 (CIDR) 范围 |
|
集群角色和策略 |
|
集群更新策略 |
|
3.1.3. 使用 Terraform 创建默认 ROSA (经典架构)集群
以下概述的集群创建过程演示了如何使用 Terraform 创建帐户范围的 IAM 角色和带有受管 OIDC 配置的 ROSA (经典架构)集群。
3.1.3.1. 为 Terraform 准备您的环境
在使用 Terraform 在 AWS 集群上创建 Red Hat OpenShift Service 前,您需要导出 离线 Red Hat OpenShift Cluster Manager 令牌。
流程
可选: 因为在安装过程中在当前目录中创建 Terraform 文件,所以您可以创建一个新的目录来存储这些文件并导航到其中:
$ mkdir terraform-cluster && cd terraform-cluster
- 使用 离线 Red Hat OpenShift Cluster Manager 令牌 向您的帐户授予权限。
运行以下命令复制离线令牌,并将令牌设置为环境变量:
$ export RHCS_TOKEN=<your_offline_token>
注意此环境变量会在每个会话的末尾重置,如重启计算机或关闭终端。
验证
导出令牌后,运行以下命令来验证值:
$ echo $RHCS_TOKEN
3.1.3.2. 在本地创建 Terraform 文件
设置 离线 Red Hat OpenShift Cluster Manager 令牌 后,您需要在本地创建 Terraform 文件以构建集群。您可以使用以下代码模板创建这些文件。
流程
运行以下命令来创建
main.tf
文件:$ cat<<-EOF>main.tf # # Copyright (c) 2023 Red Hat, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # terraform { required_providers { aws = { source = "hashicorp/aws" version = ">= 4.20.0" } rhcs = { version = ">= 1.6.2" source = "terraform-redhat/rhcs" } } } # Export token using the RHCS_TOKEN environment variable provider "rhcs" {} provider "aws" { region = var.aws_region ignore_tags { key_prefixes = ["kubernetes.io/"] } default_tags { tags = var.default_aws_tags } } data "aws_availability_zones" "available" {} locals { # The default setting creates 3 availability zones. Set to "false" to create a single availability zones. region_azs = var.multi_az ? slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 3) : slice([for zone in data.aws_availability_zones.available.names : format("%s", zone)], 0, 1) } resource "random_string" "random_name" { length = 6 special = false upper = false } locals { path = coalesce(var.path, "/") worker_node_replicas = try(var.worker_node_replicas, var.multi_az ? 3 : 2) # If cluster_name is not null, use that, otherwise generate a random cluster name cluster_name = coalesce(var.cluster_name, "rosa-\${random_string.random_name.result}") } # The network validator requires an additional 60 seconds to validate Terraform clusters. resource "time_sleep" "wait_60_seconds" { count = var.create_vpc ? 1 : 0 depends_on = [module.vpc] create_duration = "60s" } module "rosa-classic" { source = "terraform-redhat/rosa-classic/rhcs" version = "1.5.0" cluster_name = local.cluster_name openshift_version = var.openshift_version account_role_prefix = local.cluster_name operator_role_prefix = local.cluster_name replicas = local.worker_node_replicas aws_availability_zones = local.region_azs create_oidc = true private = var.private_cluster aws_private_link = var.private_cluster aws_subnet_ids = var.create_vpc ? var.private_cluster ? module.vpc[0].private_subnets : concat(module.vpc[0].public_subnets, module.vpc[0].private_subnets) : var.aws_subnet_ids multi_az = var.multi_az create_account_roles = true create_operator_roles = true # Optional: Configure a cluster administrator user <.> # # Option 1: Default cluster-admin user # Create an administrator user (cluster-admin) and automatically # generate a password by uncommenting the following parameter: # create_admin_user = true # Generated administrator credentials are displayed in terminal output. # # Option 2: Specify administrator username and password # Create an administrator user and define your own password # by uncommenting and editing the values of the following parameters: # admin_credentials_username = <username> # admin_credentials_password = <password> depends_on = [time_sleep.wait_60_seconds] } EOF
<.> 可选:通过取消注释适当的参数并编辑其值,在集群创建过程中创建管理员用户。
运行以下命令来创建
variables.tf
文件:注意在运行该命令 以构建 集群前复制并编辑此文件。
$ cat<<-EOF>variables.tf # # Copyright (c) 2023 Red Hat, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # variable "openshift_version" { type = string default = "4.14.20" description = "Desired version of OpenShift for the cluster, for example '4.14.20'. If version is greater than the currently running version, an upgrade will be scheduled." } variable "create_vpc" { type = bool description = "If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'." } # ROSA Cluster info variable "cluster_name" { default = null type = string description = "The name of the ROSA cluster to create" } variable "additional_tags" { default = { Terraform = "true" Environment = "dev" } description = "Additional AWS resource tags" type = map(string) } variable "path" { description = "(Optional) The arn path for the account/operator roles as well as their policies." type = string default = null } variable "multi_az" { type = bool description = "Multi AZ Cluster for High Availability" default = true } variable "worker_node_replicas" { default = 3 description = "Number of worker nodes to provision. Single zone clusters need at least 2 nodes, multizone clusters need at least 3 nodes" type = number } variable "aws_subnet_ids" { type = list(any) description = "A list of either the public or public + private subnet IDs to use for the cluster blocks to use for the cluster" default = ["subnet-01234567890abcdef", "subnet-01234567890abcdef", "subnet-01234567890abcdef"] } variable "private_cluster" { type = bool description = "If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'." } #VPC Info variable "vpc_name" { type = string description = "VPC Name" default = "tf-qs-vpc" } variable "vpc_cidr_block" { type = string description = "value of the CIDR block to use for the VPC" default = "10.0.0.0/16" } variable "private_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the private subnets" default = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"] } variable "public_subnet_cidrs" { type = list(any) description = "The CIDR blocks to use for the public subnets" default = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"] } variable "single_nat_gateway" { type = bool description = "Single NAT or per NAT for subnet" default = false } #AWS Info variable "aws_region" { type = string default = "us-east-2" } variable "default_aws_tags" { type = map(string) description = "Default tags for AWS" default = {} } EOF
运行以下命令来创建
vpc.tf
文件:$ cat<<-EOF>vpc.tf # # Copyright (c) 2023 Red Hat, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.2" count = var.create_vpc ? 1 : 0 name = var.vpc_name cidr = var.vpc_cidr_block azs = local.region_azs private_subnets = var.private_subnet_cidrs public_subnets = var.public_subnet_cidrs enable_nat_gateway = true single_nat_gateway = var.single_nat_gateway enable_dns_hostnames = true enable_dns_support = true tags = var.additional_tags } EOF
您已准备好启动 Terraform。
3.1.3.3. 使用 Terraform 创建 ROSA 集群
创建 Terraform 文件后,您必须启动 Terraform 以提供所有所需的依赖项。然后应用 Terraform 计划。
不要修改 Terraform 状态文件。如需更多信息,请参阅使用 Terraform 时的注意事项
流程
将 Terraform 设置为根据您的 Terraform 文件创建资源,运行以下命令:
$ terraform init
可选 :运行以下命令来验证您复制的 Terraform 是否正确:
$ terraform validate
输出示例
Success! The configuration is valid.
运行以下命令,使用 Terraform 创建集群:
$ terraform apply
Terraform 界面需要两个问题来创建集群,并熟悉以下内容:
输出示例
var.create_vpc If you would like to create a new VPC, set this value to 'true'. If you do not want to create a new VPC, set this value to 'false'. Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true'. If you want a publicly available cluster, set this value to 'false'. Enter a value:
当 Terraform 界面列出要创建或修改的资源并提示确认时,输入
yes
才能继续或取消:输出示例
Plan: 74 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes
如果输入
yes
,您的 Terraform 计划将启动,创建 AWS 帐户角色、Operator 角色和 ROSA Classic 集群。
验证
运行以下命令验证集群是否已创建:
$ rosa list clusters
显示集群的 ID、名称和状态的输出示例
ID NAME STATE TOPOLOGY 27c3snjsupa9obua74ba8se5kcj11269 rosa-tf-demo ready Classic (STS)
运行以下命令验证您的帐户角色是否已创建:
$ rosa list account-roles
输出示例
I: Fetching account roles ROLE NAME ROLE TYPE ROLE ARN OPENSHIFT VERSION AWS Managed ROSA-demo-ControlPlane-Role Control plane arn:aws:iam::<ID>:role/ROSA-demo-ControlPlane-Role 4.14 No ROSA-demo-Installer-Role Installer arn:aws:iam::<ID>:role/ROSA-demo-Installer-Role 4.14 No ROSA-demo-Support-Role Support arn:aws:iam::<ID>:role/ROSA-demo-Support-Role 4.14 No ROSA-demo-Worker-Role Worker arn:aws:iam::<ID>:role/ROSA-demo-Worker-Role 4.14 No
运行以下命令验证您的 Operator 角色是否已创建:
$ rosa list operator-roles
显示 Terraform 创建的 Operator 角色的输出示例
I: Fetching operator roles ROLE PREFIX AMOUNT IN BUNDLE rosa-demo 6
3.1.3.4. 使用 Terraform 删除 ROSA 集群
使用 terraform destroy
命令删除通过 terraform apply
命令创建的所有资源。
在销毁资源前,不要修改 Terraform .tf
文件。这些变量与要删除的资源匹配。
流程
在运行
terraform apply
命令创建集群的目录中,运行以下命令删除集群:$ terraform destroy
Terraform 接口提示您输入两个变量。它们应与创建集群时提供的答案匹配:
var.create_vpc If you would like to create a new VPC, set this value to 'true.' If you do not want to create a new VPC, set this value to 'false.' Enter a value: var.private_cluster If you want to create a private cluster, set this value to 'true.' If you want a publicly available cluster, set this value to 'false.' Enter a value:
输入
yes
以启动角色和集群删除:输出示例
Plan: 0 to add, 0 to change, 74 to destroy. Do you really want to destroy all resources? Terraform will destroy all your managed infrastructure, as shown above. There is no undo. Only 'yes' will be accepted to confirm. Enter a value: yes
验证
运行以下命令验证集群是否已销毁:
$ rosa list clusters
没有显示集群的输出示例
I: No clusters available
运行以下命令,验证帐户角色是否已销毁:
$ rosa list account-roles
没有显示 Terraform 创建帐户角色的输出示例
I: Fetching account roles I: No account roles available
运行以下命令,验证 Operator 角色是否已销毁:
$ rosa list operator-roles
输出显示没有 Terraform 创建的 Operator 角色的示例
I: Fetching operator roles I: No operator roles available