Chapter 11. Tutorial: Using AWS Controllers for Kubernetes on Red Hat OpenShift Service on AWS

Procedure

1. Configure the following environment variables, changing the cluster name to suit your cluster:

   $ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}"  | sed 's/-[a-z0-9]\{5\}$//')
   $ export REGION=$(rosa describe cluster -c ${ROSA_CLUSTER_NAME} --output json | jq -r .region.id)
   $ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o json | jq -r .spec.serviceAccountIssuer | sed  's|^https://||')
   $ export AWS_ACCOUNT_ID=`aws sts get-caller-identity --query Account --output text`
   $ export ACK_SERVICE=s3
   $ export ACK_SERVICE_ACCOUNT=ack-${ACK_SERVICE}-controller
   $ export POLICY_ARN=arn:aws:iam::aws:policy/AmazonS3FullAccess
   $ export AWS_PAGER=""
   $ export SCRATCH="/tmp/${ROSA_CLUSTER_NAME}/ack"
   $ mkdir -p ${SCRATCH}

2. Ensure all fields output correctly before moving to the next section:

   $ echo "Cluster: ${ROSA_CLUSTER_NAME}, Region: ${REGION}, OIDC Endpoint: ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"

Procedure

1. Create an AWS Identity Access Management (IAM) trust policy for the ACK Operator:

   $ cat <<EOF > "${SCRATCH}/trust-policy.json"
   {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Condition": {
      "StringEquals" : {
        "${OIDC_ENDPOINT}:sub": "system:serviceaccount:ack-system:${ACK_SERVICE_ACCOUNT}"
      }
    },
    "Principal": {
      "Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
    },
    "Action": "sts:AssumeRoleWithWebIdentity"
    }
    ]
   }
   EOF

2. Create an AWS IAM role for the ACK Operator to assume with the AmazonS3FullAccess policy attached:

   Note

   You can find the recommended policy in each project’s GitHub repository, for example https://github.com/aws-controllers-k8s/s3-controller/blob/main/config/iam/recommended-policy-arn.

   $ ROLE_ARN=$(aws iam create-role --role-name "ack-${ACK_SERVICE}-controller" \
      --assume-role-policy-document "file://${SCRATCH}/trust-policy.json" \
      --query Role.Arn --output text)
   $ echo $ROLE_ARN
   
   $ aws iam attach-role-policy --role-name "ack-${ACK_SERVICE}-controller" \
        --policy-arn ${POLICY_ARN}

Procedure

1. Create a project to install the ACK S3 Operator into:

   $ oc new-project ack-system

2. Create a file with the ACK S3 Operator configuration:

   Note

   ACK_WATCH_NAMESPACE is purposefully left blank so the controller can properly watch all namespaces in the cluster.

   $ cat << EOF  "${SCRATCH}/config.txt"
   ACK_ENABLE_DEVELOPMENT_LOGGING=true
   ACK_LOG_LEVEL=debug
   ACK_WATCH_NAMESPACE=
   AWS_REGION=${REGION}
   AWS_ENDPOINT_URL=
   ACK_RESOURCE_TAGS=${CLUSTER_NAME}
   ENABLE_LEADER_ELECTION=true
   LEADER_ELECTION_NAMESPACE=
   RECONCILE_DEFAULT_MAX_CONCURRENT_SYNCS=1
   FEATURE_FLAGS=
   FEATURE_GATES=
   EOF

3. Use the file from the previous step to create a ConfigMap:

   $ oc -n ack-system create configmap \
     --from-env-file=${SCRATCH}/config.txt ack-${ACK_SERVICE}-user-config

4. Install the ACK S3 Operator from the software catalog:

   $ cat << EOF | oc apply -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: ack-${ACK_SERVICE}-controller
     namespace: ack-system
   spec:
     upgradeStrategy: Default
   ---
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: ack-${ACK_SERVICE}-controller
     namespace: ack-system
   spec:
     channel: alpha
     installPlanApproval: Automatic
     name: ack-${ACK_SERVICE}-controller
     source: community-operators
     sourceNamespace: openshift-marketplace
   EOF

5. Annotate the ACK S3 Operator service account with the AWS IAM role to assume and restart the deployment:

   $ oc -n ack-system annotate serviceaccount ${ACK_SERVICE_ACCOUNT} \
     eks.amazonaws.com/role-arn=${ROLE_ARN} && \
     oc -n ack-system rollout restart deployment ack-${ACK_SERVICE}-controller

6. Verify that the ACK S3 Operator is running:

   $ oc -n ack-system get pods

   Example

   NAME                                 READY   STATUS    RESTARTS   AGE
   ack-s3-controller-585f6775db-s4lfz   1/1     Running   0          51s

Procedure

1. Deploy an S3 bucket resource:

   $ cat << EOF | oc apply -f -
   apiVersion: s3.services.k8s.aws/v1alpha1
   kind: Bucket
   metadata:
      name: ${CLUSTER-NAME}-bucket
      namespace: ack-system
   spec:
      name: ${CLUSTER-NAME}-bucket
   EOF

2. Verify the S3 bucket was created in AWS:

   $ aws s3 ls | grep ${CLUSTER_NAME}-bucket

   Example output

   2023-10-04 14:51:45 mrmc-test-maz-bucket

Procedure

1. Delete the S3 bucket resource:

   $ oc -n ack-system delete bucket.s3.services.k8s.aws/${CLUSTER-NAME}-bucket

2. Delete the ACK S3 Operator and the AWS IAM roles:

   $ oc -n ack-system delete subscription ack-${ACK_SERVICE}-controller
   $ aws iam detach-role-policy \
     --role-name "ack-${ACK_SERVICE}-controller" \
     --policy-arn ${POLICY_ARN}
   $ aws iam delete-role \
     --role-name "ack-${ACK_SERVICE}-controller"

3. Delete the ack-system project:

   $ oc delete project ack-system