Chapter 1. AWS Load Balancer Operator
The AWS Load Balancer Operator is an Operator supported by Red Hat that users can optionally install on SRE-managed Red Hat OpenShift Service on AWS clusters.
Load Balancers created by the AWS Load Balancer Operator cannot be used for OpenShift Routes, and should only be used for individual services or ingress resources that do not need the full layer 7 capabilities of an OpenShift Route.
The AWS Load Balancer Operator is used to install, manage and configure the AWS Load Balancer Controller in a Red Hat OpenShift Service on AWS cluster.
The AWS Load Balancer Controller provisions AWS Application Load Balancers (ALB) when you create Kubernetes Ingress resources and AWS Network Load Balancers (NLB) when you create a Kubernetes Service resource with a type of LoadBalancer.
Compared with the default AWS in-tree load balancer provider, this controller is developed with advanced annotations for both ALBs and NLBs. Some advanced use cases are:
- Using native Kubernetes Ingress objects with ALBs
- Integrate ALBs with the AWS Web Application Firewall (WAF) service
- Specify custom NLB source IP ranges
- Specify custom NLB internal IP addresses
1.1. Preparing to install the AWS Load Balancer Operator Copy linkLink copied to clipboard!
Before you install the AWS Load Balancer Operator, ensure that your cluster fulfills requirements and that your AWS VPC resources are appropriately tagged. You also have the option to configure some helpful environment variables.
1.1.1. Cluster requirements Copy linkLink copied to clipboard!
Your cluster must be deployed across three availability zones, using a pre-existing VPC with three public subnets.
These requirements mean that the AWS Load Balancer Operator may not be suitable for some PrivateLink clusters. AWS NLBs may be a better choice for such clusters.
1.1.2. Set up temporary environment variables Copy linkLink copied to clipboard!
You have the option to set up temporary environment variables to hold resource identifiers and configuration details. Using temporary environment variables streamlines the process of running the installation commands for the AWS Load Balancer Operator.
If you do not want to use environment variables to store certain values, you can manually enter those values in the relevant installation commands.
Prerequisites
-
You have installed the AWS CLI (
aws
). -
You have installed the OC CLI (
oc
).
Procedure
Log in to your cluster as a cluster administrator using the OpenShift CLI (
oc
).oc login --token=<token> --server=<cluster_url>
$ oc login --token=<token> --server=<cluster_url>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the following commands to set up environment variables.
Copy to Clipboard Copied! Toggle word wrap Toggle overflow These commands create environment variables that you can use in this terminal session to pass their values to the command line interface.
Verify that the variable values are set correctly by running the following command:
echo "Cluster name: ${CLUSTER_NAME}
$ echo "Cluster name: ${CLUSTER_NAME} Region: ${REGION} OIDC Endpoint: ${OIDC_ENDPOINT} AWS Account ID: ${AWS_ACCOUNT_ID}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Cluster name: <cluster_id> Region: <region> OIDC Endpoint: oidc.op1.openshiftapps.com/<oidc_id> AWS Account ID: <aws_id>
Cluster name: <cluster_id> Region: <region> OIDC Endpoint: oidc.op1.openshiftapps.com/<oidc_id> AWS Account ID: <aws_id>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Next steps
- Use the same terminal session to continue with AWS Load Balancer Operator installation, to ensure that your environment variables are not lost.
1.1.3. Tag the AWS VPC and subnets Copy linkLink copied to clipboard!
You must tag your AWS VPC resources before you install the AWS Load Balancer Operator.
Prerequisites
-
You have installed the AWS CLI (
aws
). -
You have installed the OC CLI (
oc
).
Procedure
Optional: Set up environment variables for AWS VPC resources.
export VPC_ID=<vpc-id> export PUBLIC_SUBNET_IDS="<public-subnet-a-id> <public-subnet-b-id> <public-subnet-c-id>" export PRIVATE_SUBNET_IDS="<private-subnet-a-id> <private-subnet-b-id> <private-subnet-c-id>"
$ export VPC_ID=<vpc-id> $ export PUBLIC_SUBNET_IDS="<public-subnet-a-id> <public-subnet-b-id> <public-subnet-c-id>" $ export PRIVATE_SUBNET_IDS="<private-subnet-a-id> <private-subnet-b-id> <private-subnet-c-id>"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Tag your VPC to associate it with your cluster:
aws ec2 create-tags --resources ${VPC_ID} --tags Key=kubernetes.io/cluster/${CLUSTER_NAME},Value=owned --region ${REGION}
$ aws ec2 create-tags --resources ${VPC_ID} --tags Key=kubernetes.io/cluster/${CLUSTER_NAME},Value=owned --region ${REGION}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Tag your public subnets to allow changes by elastic load balancing roles, and tag your private subnets to allow changes by internal elastic load balancing roles:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Run the script:
bash ${SCRATCH}/tag-subnets.sh
bash ${SCRATCH}/tag-subnets.sh
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Additional resources
- To set up a Red Hat OpenShift Service on AWS cluster with multiple availability zones, see A multi-AZ Red Hat OpenShift Service on AWS cluster
1.2. Installing the AWS Load Balancer Operator Copy linkLink copied to clipboard!
You can install the AWS Load Balancer Operator using the OpenShift CLI (oc
). Use the same terminal session you used in Setting up your environment to install the AWS Load Balancer Operator to make use of the environment variables.
Procedure
Create a new project within your cluster for the AWS Load Balancer Operator:
oc new-project aws-load-balancer-operator
$ oc new-project aws-load-balancer-operator
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an AWS IAM policy for the AWS Load Balancer Operator.
Download the appropriate IAM policy:
curl -o ${SCRATCH}/operator-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/refs/heads/main/hack/operator-permission-policy.json
$ curl -o ${SCRATCH}/operator-permission-policy.json https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/refs/heads/main/hack/operator-permission-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the permission policy for the Operator:
aws iam create-policy \ --policy-name aws-load-balancer-operator-policy \ --policy-document file://${SCRATCH}/operator-permission-policy.json \ --region ${REGION}
$ aws iam create-policy \ --policy-name aws-load-balancer-operator-policy \ --policy-document file://${SCRATCH}/operator-permission-policy.json \ --region ${REGION}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Take note of the Operator policy ARN in the output. This is referred to as the
$OPERATOR_POLICY_ARN
for the remainder of this process.
Create an AWS IAM role for the AWS Load Balancer Operator:
Create the trust policy for the Operator role:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the Operator role using the trust policy:
aws iam create-role --role-name "${CLUSTER_NAME}-alb-operator" \ --assume-role-policy-document "file://${SCRATCH}/operator-trust-policy.json"
$ aws iam create-role --role-name "${CLUSTER_NAME}-alb-operator" \ --assume-role-policy-document "file://${SCRATCH}/operator-trust-policy.json"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Take note of the Operator role ARN in the output. This is referred to as the
$OPERATOR_ROLE_ARN
for the remainder of this process.Associate the Operator role and policy:
aws iam attach-role-policy --role-name "${CLUSTER_NAME}-alb-operator" \ --policy-arn $OPERATOR_POLICY_ARN
$ aws iam attach-role-policy --role-name "${CLUSTER_NAME}-alb-operator" \ --policy-arn $OPERATOR_POLICY_ARN
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Install the AWS Load Balancer Operator by creating an
OperatorGroup
and aSubscription
:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create an AWS IAM policy for the AWS Load Balancer Controller.
Download the appropriate IAM policy:
curl -o ${SCRATCH}/controller-permission-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.12.0/docs/install/iam_policy.json
$ curl -o ${SCRATCH}/controller-permission-policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.12.0/docs/install/iam_policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the permission policy for the Controller:
aws iam create-policy \ --region ${REGION} \ --policy-name aws-load-balancer-controller-policy \ --policy-document file://${SCRATCH}/controller-permission-policy.json
$ aws iam create-policy \ --region ${REGION} \ --policy-name aws-load-balancer-controller-policy \ --policy-document file://${SCRATCH}/controller-permission-policy.json
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Take note of the Controller policy ARN in the output. This is referred to as the
$CONTROLLER_POLICY_ARN
for the remainder of this process.
Create an AWS IAM role for the AWS Load Balancer Controller:
Create the trust policy for the Controller role:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the Controller role using the trust policy:
CONTROLLER_ROLE_ARN=$(aws iam create-role --role-name "${CLUSTER_NAME}-albo-controller" \ --assume-role-policy-document "file://${SCRATCH}/controller-trust-policy.json" \ --query Role.Arn --output text) echo ${CONTROLLER_ROLE_ARN}
CONTROLLER_ROLE_ARN=$(aws iam create-role --role-name "${CLUSTER_NAME}-albo-controller" \ --assume-role-policy-document "file://${SCRATCH}/controller-trust-policy.json" \ --query Role.Arn --output text) echo ${CONTROLLER_ROLE_ARN}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Take note of the Controller role ARN in the output. This is referred to as the
$CONTROLLER_ROLE_ARN
for the remainder of this process.Associate the Controller role and policy:
aws iam attach-role-policy \ --role-name "${CLUSTER_NAME}-albo-controller" \ --policy-arn ${CONTROLLER_POLICY_ARN}
$ aws iam attach-role-policy \ --role-name "${CLUSTER_NAME}-albo-controller" \ --policy-arn ${CONTROLLER_POLICY_ARN}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Deploy an instance of the AWS Load Balancer Controller:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow NoteIf you get an error here wait a minute and try again, it means the Operator has not completed installing yet.
Confirm that the Operator and Controller pods are both running:
oc -n aws-load-balancer-operator get pods
$ oc -n aws-load-balancer-operator get pods
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If you do not see output similar to the following, wait a few moments and retry.
Example output
NAME READY STATUS RESTARTS AGE aws-load-balancer-controller-cluster-6ddf658785-pdp5d 1/1 Running 0 99s aws-load-balancer-operator-controller-manager-577d9ffcb9-w6zqn 2/2 Running 0 2m4s
NAME READY STATUS RESTARTS AGE aws-load-balancer-controller-cluster-6ddf658785-pdp5d 1/1 Running 0 99s aws-load-balancer-operator-controller-manager-577d9ffcb9-w6zqn 2/2 Running 0 2m4s
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
Additional resources
1.3. Validating Operator installation Copy linkLink copied to clipboard!
Deploy a basic sample application and create ingress and load balancing services to confirm that the AWS Load Balancer Operator and Controller deployed correctly.
Procedure
Create a new project:
oc new-project hello-world
$ oc new-project hello-world
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new
hello-world
application based on thehello-openshift
image:oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
$ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Configure a
NodePort
service for an AWS Application Load Balancer (ALB) to connect to:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Deploy an AWS ALB for the application:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Test access to the AWS ALB endpoint for the application:
NoteALB provisioning takes a few minutes. If you receive an error that says
curl: (6) Could not resolve host
, please wait and try again.ALB_INGRESS=$(oc -n hello-world get ingress hello-openshift-alb \ -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') curl "http://${ALB_INGRESS}"
$ ALB_INGRESS=$(oc -n hello-world get ingress hello-openshift-alb \ -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') $ curl "http://${ALB_INGRESS}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Hello OpenShift!
Hello OpenShift!
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Deploy an AWS Network Load Balancer (NLB) for the application:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Test access to the NLB endpoint for the application:
NoteNLB provisioning takes a few minutes. If you receive an error that says
curl: (6) Could not resolve host
, please wait and try again.NLB=$(oc -n hello-world get service hello-openshift-nlb \ -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') curl "http://${NLB}"
$ NLB=$(oc -n hello-world get service hello-openshift-nlb \ -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') $ curl "http://${NLB}"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Hello OpenShift!
Hello OpenShift!
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can now delete the sample application and all resources in the
hello-world
namespace.oc delete project hello-world
$ oc delete project hello-world
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
1.4. Removing the AWS Load Balancer Operator Copy linkLink copied to clipboard!
If you no longer need to use the AWS Load Balancer Operator, you can remove the Operator and delete any related roles and policies.
Procedure
Delete the Operator Subscription:
oc delete subscription aws-load-balancer-operator -n aws-load-balancer-operator
$ oc delete subscription aws-load-balancer-operator -n aws-load-balancer-operator
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Detach and delete the relevant AWS IAM roles:
aws iam detach-role-policy \ --role-name "<cluster-id>-alb-operator" \ --policy-arn <operator-policy-arn> aws iam delete-role \ --role-name "<cluster-id>-alb-operator"
$ aws iam detach-role-policy \ --role-name "<cluster-id>-alb-operator" \ --policy-arn <operator-policy-arn> $ aws iam delete-role \ --role-name "<cluster-id>-alb-operator"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Delete the AWS IAM policy:
aws iam delete-policy --policy-arn <operator-policy-arn>
$ aws iam delete-policy --policy-arn <operator-policy-arn>
Copy to Clipboard Copied! Toggle word wrap Toggle overflow