Chapter 20. AWS Load Balancer Operator

Procedure

1. You can deploy the AWS Load Balancer Operator on demand from OperatorHub, by creating a

   Subscription

   object by running the following command:

   $ oc -n aws-load-balancer-operator get sub aws-load-balancer-operator --template='{{.status.installplan.name}}{{"\n"}}'

   Example output

   install-zlfbt

2. Check if the status of an install plan is

   Complete

   by running the following command:

   $ oc -n aws-load-balancer-operator get ip <install_plan_name> --template='{{.status.phase}}{{"\n"}}'

   Example output

   Complete

3. View the status of the

   aws-load-balancer-operator-controller-manager

   deployment by running the following command:

   $ oc get -n aws-load-balancer-operator deployment/aws-load-balancer-operator-controller-manager

   Example output

   NAME                                           READY     UP-TO-DATE   AVAILABLE   AGE
   aws-load-balancer-operator-controller-manager  1/1       1            1           23h

Procedure

1. Navigate to Operators OperatorHub in the OpenShift Container Platform web console.
2. Select the AWS Load Balancer Operator. You can use the Filter by keyword text box or use the filter list to search for the AWS Load Balancer Operator from the list of Operators.
3. Select the

   aws-load-balancer-operator

   namespace.

4. On the Install Operator page, select the following options:

   1. Update the channel as stable-v1.
   2. Installation mode as All namespaces on the cluster (default).
   3. Installed Namespace as

      aws-load-balancer-operator

      . If the

      aws-load-balancer-operator

      namespace does not exist, it gets created during the Operator installation.
   4. Select Update approval as Automatic or Manual. By default, the Update approval is set to Automatic. If you select automatic updates, the Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without any intervention. If you select manual updates, the OLM creates an update request. As a cluster administrator, you must then manually approve that update request to update the Operator updated to the new version.
5. Click Install.

Procedure

1. Create a

   Namespace

   object:

   1. Create a YAML file that defines the

      Namespace

      object:

      Example namespace.yaml file

      apiVersion: v1
      kind: Namespace
      metadata:
        name: aws-load-balancer-operator

   2. Create the

      Namespace

      object by running the following command:

      $ oc apply -f namespace.yaml

2. Create a

   CredentialsRequest

   object:

   1. Create a YAML file that defines the

      CredentialsRequest

      object:

      Example credentialsrequest.yaml file

      apiVersion: cloudcredential.openshift.io/v1
      kind: CredentialsRequest
      metadata:
        name: aws-load-balancer-operator
        namespace: openshift-cloud-credential-operator
      spec:
        providerSpec:
          apiVersion: cloudcredential.openshift.io/v1
          kind: AWSProviderSpec
          statementEntries:
            - action:
                - ec2:DescribeSubnets
              effect: Allow
              resource: "*"
            - action:
                - ec2:CreateTags
                - ec2:DeleteTags
              effect: Allow
              resource: arn:aws:ec2:*:*:subnet/*
            - action:
                - ec2:DescribeVpcs
              effect: Allow
              resource: "*"
        secretRef:
          name: aws-load-balancer-operator
          namespace: aws-load-balancer-operator
        serviceAccountNames:
          - aws-load-balancer-operator-controller-manager

   2. Create the

      CredentialsRequest

      object by running the following command:

      $ oc apply -f credentialsrequest.yaml

3. Create an

   OperatorGroup

   object:

   1. Create a YAML file that defines the

      OperatorGroup

      object:

      Example operatorgroup.yaml file

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: aws-lb-operatorgroup
        namespace: aws-load-balancer-operator
      spec:
        upgradeStrategy: Default

   2. Create the

      OperatorGroup

      object by running the following command:

      $ oc apply -f operatorgroup.yaml

4. Create a

   Subscription

   object:

   1. Create a YAML file that defines the

      Subscription

      object:

      Example subscription.yaml file

      apiVersion: operators.coreos.com/v1alpha1
      kind: Subscription
      metadata:
        name: aws-load-balancer-operator
        namespace: aws-load-balancer-operator
      spec:
        channel: stable-v1
        installPlanApproval: Automatic
        name: aws-load-balancer-operator
        source: redhat-operators
        sourceNamespace: openshift-marketplace

   2. Create the

      Subscription

      object by running the following command:

      $ oc apply -f subscription.yaml

Verification

1. Get the name of the install plan from the subscription:

   $ oc -n aws-load-balancer-operator \
       get subscription aws-load-balancer-operator \
       --template='{{.status.installplan.name}}{{"\n"}}'

2. Check the status of the install plan:

   $ oc -n aws-load-balancer-operator \
       get ip <install_plan_name> \
       --template='{{.status.phase}}{{"\n"}}'

   The output must be

   Complete

   .

Procedure

1. Create the

   aws-load-balancer-operator

   namespace by running the following command:

   $ oc create namespace aws-load-balancer-operator

2. Download the

   CredentialsRequest

   custom resource (CR) of the AWS Load Balancer Operator, and create a directory to store it by running the following command:

   $ curl --create-dirs -o <path-to-credrequests-dir>/cr.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/operator-credentials-request.yaml

3. Use the

   ccoctl

   tool to process

   CredentialsRequest

   objects of the AWS Load Balancer Operator, by running the following command:

   $ ccoctl aws create-iam-roles \
       --name <name> --region=<aws_region> \
       --credentials-requests-dir=<path-to-credrequests-dir> \
       --identity-provider-arn <oidc-arn>

4. Apply the secrets generated in the manifests directory of your cluster by running the following command:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

5. Verify that the credentials secret of the AWS Load Balancer Operator is created by running the following command:

   $ oc -n aws-load-balancer-operator get secret aws-load-balancer-operator --template='{{index .data "credentials"}}' | base64 -d

   Example output

   [default]
   sts_regional_endpoints = regional
   role_arn = arn:aws:iam::999999999999:role/aws-load-balancer-operator-aws-load-balancer-operator
   web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

Procedure

1. The AWS Load Balancer Operator creates the

   CredentialsRequest

   object in the

   openshift-cloud-credential-operator

   namespace for each

   AWSLoadBalancerController

   custom resource (CR). You can extract and save the created

   CredentialsRequest

   object in a directory by running the following command:

   $ oc get credentialsrequest -n openshift-cloud-credential-operator  \
       aws-load-balancer-controller-<cr-name> -o yaml > <path-to-credrequests-dir>/cr.yaml 

   1

   1

   The aws-load-balancer-controller-<cr-name> parameter specifies the credential request name created by the AWS Load Balancer Operator. The cr-name specifies the name of the AWS Load Balancer Controller instance.

2. Use the

   ccoctl

   tool to process all

   CredentialsRequest

   objects in the

   credrequests

   directory by running the following command:

   $ ccoctl aws create-iam-roles \
       --name <name> --region=<aws_region> \
       --credentials-requests-dir=<path-to-credrequests-dir> \
       --identity-provider-arn <oidc-arn>

3. Apply the secrets generated in manifests directory to your cluster, by running the following command:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

4. Verify that the

   aws-load-balancer-controller

   pod is created:

   $ oc -n aws-load-balancer-operator get pods
   NAME                                                            READY   STATUS    RESTARTS   AGE
   aws-load-balancer-controller-cluster-9b766d6-gg82c              1/1     Running   0          137m
   aws-load-balancer-operator-controller-manager-b55ff68cc-85jzg   2/2     Running   0          3h26m

Procedure

1. Download the CredentialsRequest custom resource (CR) of the AWS Load Balancer Controller, and create a directory to store it by running the following command:

   $ curl --create-dirs -o <path-to-credrequests-dir>/cr.yaml https://raw.githubusercontent.com/openshift/aws-load-balancer-operator/main/hack/controller/controller-credentials-request.yaml

2. Use the

   ccoctl

   tool to process the

   CredentialsRequest

   object of the controller:

   $ ccoctl aws create-iam-roles \
           --name <name> --region=<aws_region> \
           --credentials-requests-dir=<path-to-credrequests-dir> \
           --identity-provider-arn <oidc-arn>

3. Apply the secrets to your cluster:

   $ ls manifests/*-credentials.yaml | xargs -I{} oc apply -f {}

4. Verify the credentials secret has been created for use by the controller:

   $ oc -n aws-load-balancer-operator get secret aws-load-balancer-controller-manual-cluster --template='{{index .data "credentials"}}' | base64 -d

   Example output

   [default]
       sts_regional_endpoints = regional
       role_arn = arn:aws:iam::999999999999:role/aws-load-balancer-operator-aws-load-balancer-controller
       web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

5. Create the

   AWSLoadBalancerController

   resource YAML file, for example,

   sample-aws-lb-manual-creds.yaml

   , as follows:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController 

   1

   
   metadata:
     name: cluster 

   2

   
   spec:
     credentials:
       name: <secret-name> 

   3

   1

   Defines the AWSLoadBalancerController resource.

   2

   Defines the AWS Load Balancer Controller instance name. This instance name gets added as a suffix to all related resources.

   3

   Specifies the secret name containing AWS credentials that the controller uses.

Procedure

1. Create a YAML file that defines the

   AWSLoadBalancerController

   object:

   Example sample-aws-lb.yaml file

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController 

   1

   
   metadata:
     name: cluster 

   2

   
   spec:
     subnetTagging: Auto 

   3

   
     additionalResourceTags: 

   4

   
     - key: example.org/security-scope
       value: staging
     ingressClass: alb 

   5

   
     config:
       replicas: 2 

   6

   
     enabledAddons: 

   7

   
       - AWSWAFv2 

   8

   1

   Defines the AWSLoadBalancerController object.

   2

   Defines the AWS Load Balancer Controller name. This instance name gets added as a suffix to all related resources.

   3

   Configures the subnet tagging method for the AWS Load Balancer Controller. The following values are valid:

   • Auto

     : The AWS Load Balancer Operator determines the subnets that belong to the cluster and tags them appropriately. The Operator cannot determine the role correctly if the internal subnet tags are not present on internal subnet.

   • Manual

     : You manually tag the subnets that belong to the cluster with the appropriate role tags. Use this option if you installed your cluster on user-provided infrastructure.

   4

   Defines the tags used by the AWS Load Balancer Controller when it provisions AWS resources.

   5

   Defines the ingress class name. The default value is alb.

   6

   Specifies the number of replicas of the AWS Load Balancer Controller.

   7

   Specifies annotations as an add-on for the AWS Load Balancer Controller.

   8

   Enables the alb.ingress.kubernetes.io/wafv2-acl-arn annotation.

2. Create the

   AWSLoadBalancerController

   object by running the following command:

   $ oc create -f sample-aws-lb.yaml

3. Create a YAML file that defines the

   Deployment

   resource:

   Example sample-aws-lb.yaml file

   apiVersion: apps/v1
   kind: Deployment 

   1

   
   metadata:
     name: <echoserver> 

   2

   
     namespace: echoserver
   spec:
     selector:
       matchLabels:
         app: echoserver
     replicas: 3 

   3

   
     template:
       metadata:
         labels:
           app: echoserver
       spec:
         containers:
           - image: openshift/origin-node
             command:
              - "/bin/socat"
             args:
               - TCP4-LISTEN:8080,reuseaddr,fork
               - EXEC:'/bin/bash -c \"printf \\\"HTTP/1.0 200 OK\r\n\r\n\\\"; sed -e \\\"/^\r/q\\\"\"'
             imagePullPolicy: Always
             name: echoserver
             ports:
               - containerPort: 8080

   1

   Defines the deployment resource.

   2

   Specifies the deployment name.

   3

   Specifies the number of replicas of the deployment.

4. Create a YAML file that defines the

   Service

   resource:

   Example service-albo.yaml file:

   apiVersion: v1
   kind: Service 

   1

   
   metadata:
     name: <echoserver> 

   2

   
     namespace: echoserver
   spec:
     ports:
       - port: 80
         targetPort: 8080
         protocol: TCP
     type: NodePort
     selector:
       app: echoserver

   1

   Defines the service resource.

   2

   Specifies the service name.

5. Create a YAML file that defines the

   Ingress

   resource:

   Example ingress-albo.yaml file:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: <name> 

   1

   
     namespace: echoserver
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: alb
     rules:
       - http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: <echoserver> 

   2

   
                   port:
                     number: 80

   1

   Specify a name for the Ingress resource.

   2

   Specifies the service name.

Procedure

1. Create an

   IngressClassParams

   resource YAML file, for example,

   sample-single-lb-params.yaml

   , as follows:

   apiVersion: elbv2.k8s.aws/v1beta1 

   1

   
   kind: IngressClassParams
   metadata:
     name: single-lb-params 

   2

   
   spec:
     group:
       name: single-lb 

   3

   1

   Defines the API group and version of the IngressClassParams resource.

   2

   Specifies the IngressClassParams resource name.

   3

   Specifies the IngressGroup resource name. All of the Ingress resources of this class belong to this IngressGroup.

2. Create the

   IngressClassParams

   resource by running the following command:

   $ oc create -f sample-single-lb-params.yaml

3. Create the

   IngressClass

   resource YAML file, for example,

   sample-single-lb-class.yaml

   , as follows:

   apiVersion: networking.k8s.io/v1 

   1

   
   kind: IngressClass
   metadata:
     name: single-lb 

   2

   
   spec:
     controller: ingress.k8s.aws/alb 

   3

   
     parameters:
       apiGroup: elbv2.k8s.aws 

   4

   
       kind: IngressClassParams 

   5

   
       name: single-lb-params 

   6

   1

   Defines the API group and version of the IngressClass resource.

   2

   Specifies the ingress class name.

   3

   Defines the controller name. The ingress.k8s.aws/alb value denotes that all ingress resources of this class should be managed by the AWS Load Balancer Controller.

   4

   Defines the API group of the IngressClassParams resource.

   5

   Defines the resource type of the IngressClassParams resource.

   6

   Defines the IngressClassParams resource name.

4. Create the

   IngressClass

   resource by running the following command:

   $ oc create -f sample-single-lb-class.yaml

5. Create the

   AWSLoadBalancerController

   resource YAML file, for example,

   sample-single-lb.yaml

   , as follows:

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     subnetTagging: Auto
     ingressClass: single-lb 

   1

   1

   Defines the name of the IngressClass resource.

6. Create the

   AWSLoadBalancerController

   resource by running the following command:

   $ oc create -f sample-single-lb.yaml

7. Create the

   Ingress

   resource YAML file, for example,

   sample-multiple-ingress.yaml

   , as follows:

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-1 

   1

   
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing 

   2

   
       alb.ingress.kubernetes.io/group.order: "1" 

   3

   
       alb.ingress.kubernetes.io/target-type: instance 

   4

   
   spec:
     ingressClassName: single-lb 

   5

   
     rules:
     - host: example.com 

   6

   
       http:
           paths:
           - path: /blog 

   7

   
             pathType: Prefix
             backend:
               service:
                 name: example-1 

   8

   
                 port:
                   number: 80 

   9

   
   ---
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-2
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/group.order: "2"
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: single-lb
     rules:
     - host: example.com
       http:
           paths:
           - path: /store
             pathType: Prefix
             backend:
               service:
                 name: example-2
                 port:
                   number: 80
   ---
   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: example-3
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing
       alb.ingress.kubernetes.io/group.order: "3"
       alb.ingress.kubernetes.io/target-type: instance
   spec:
     ingressClassName: single-lb
     rules:
     - host: example.com
       http:
           paths:
           - path: /
             pathType: Prefix
             backend:
               service:
                 name: example-3
                 port:
                   number: 80

   1

   Specifies the ingress name.

   2

   Indicates the load balancer to provision in the public subnet to access the internet.

   3

   Specifies the order in which the rules from the multiple ingress resources are matched when the request is received at the load balancer.

   4

   Indicates that the load balancer will target OpenShift Container Platform nodes to reach the service.

   5

   Specifies the ingress class that belongs to this ingress.

   6

   Defines a domain name used for request routing.

   7

   Defines the path that must route to the service.

   8

   Defines the service name that serves the endpoint configured in the Ingress resource.

   9

   Defines the port on the service that serves the endpoint.

8. Create the

   Ingress

   resource by running the following command:

   $ oc create -f sample-multiple-ingress.yaml

Procedure

1. Create a YAML file that defines the

   AWSLoadBalancerController

   resource:

   Example add-tls-termination-albc.yaml file

   apiVersion: networking.olm.openshift.io/v1
   kind: AWSLoadBalancerController
   metadata:
     name: cluster
   spec:
     subnetTagging: Auto
     ingressClass: tls-termination 

   1

   1

   Defines the ingress class name. If the ingress class is not present in your cluster the AWS Load Balancer Controller creates one. The AWS Load Balancer Controller reconciles the additional ingress class values if spec.controller is set to ingress.k8s.aws/alb.

2. Create a YAML file that defines the

   Ingress

   resource:

   Example add-tls-termination-ingress.yaml file

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: <example> 

   1

   
     annotations:
       alb.ingress.kubernetes.io/scheme: internet-facing 

   2

   
       alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-2:xxxxx 

   3

   
   spec:
     ingressClassName: tls-termination 

   4

   
     rules:
     - host: <example.com> 

   5

   
       http:
           paths:
             - path: /
               pathType: Exact
               backend:
                 service:
                   name: <example-service> 

   6

   
                   port:
                     number: 80

   1

   Specifies the ingress name.

   2

   The controller provisions the load balancer for ingress in a public subnet to access the load balancer over the internet.

   3

   The Amazon Resource Name (ARN) of the certificate that you attach to the load balancer.

   4

   Defines the ingress class name.

   5

   Defines the domain for traffic routing.

   6

   Defines the service for traffic routing.

1. Create the config map to contain the certificate authority (CA) bundle in the

   aws-load-balancer-operator

   namespace by running the following command:

   $ oc -n aws-load-balancer-operator create configmap trusted-ca

2. To inject the trusted CA bundle into the config map, add the

   config.openshift.io/inject-trusted-cabundle=true

   label to the config map by running the following command:

   $ oc -n aws-load-balancer-operator label cm trusted-ca config.openshift.io/inject-trusted-cabundle=true

3. Update the AWS Load Balancer Operator subscription to access the config map in the AWS Load Balancer Operator deployment by running the following command:

   $ oc -n aws-load-balancer-operator patch subscription aws-load-balancer-operator --type='merge' -p '{"spec":{"config":{"env":[{"name":"TRUSTED_CA_CONFIGMAP_NAME","value":"trusted-ca"}],"volumes":[{"name":"trusted-ca","configMap":{"name":"trusted-ca"}}],"volumeMounts":[{"name":"trusted-ca","mountPath":"/etc/pki/tls/certs/albo-tls-ca-bundle.crt","subPath":"ca-bundle.crt"}]}}}'

4. After the AWS Load Balancer Operator is deployed, verify that the CA bundle is added to the

   aws-load-balancer-operator-controller-manager

   deployment by running the following command:

   $ oc -n aws-load-balancer-operator exec deploy/aws-load-balancer-operator-controller-manager -c manager -- bash -c "ls -l /etc/pki/tls/certs/albo-tls-ca-bundle.crt; printenv TRUSTED_CA_CONFIGMAP_NAME"

   Example output

   -rw-r--r--. 1 root 1000690000 5875 Jan 11 12:25 /etc/pki/tls/certs/albo-tls-ca-bundle.crt
   trusted-ca

5. Optional: Restart deployment of the AWS Load Balancer Operator every time the config map changes by running the following command:

   $ oc -n aws-load-balancer-operator rollout restart deployment/aws-load-balancer-operator-controller-manager