Chapter 5. Developing Operators

Procedure

1. Create a project.

   1. Create your project directory:

      $ mkdir memcached-operator

   2. Change into the project directory:

      $ cd memcached-operator

   3. Run the operator-sdk init command to initialize the project:

      $ operator-sdk init \
          --domain=example.com \
          --repo=github.com/example-inc/memcached-operator

      The command uses the Go plugin by default.

2. Create an API.

   Create a simple Memcached API:

   $ operator-sdk create api \
       --resource=true \
       --controller=true \
       --group cache \
       --version v1 \
       --kind Memcached

3. Build and push the Operator image.

   Use the default Makefile targets to build and push your Operator. Set IMG with a pull spec for your image that uses a registry you can push to:

   $ make docker-build docker-push IMG=<registry>/<user>/<image_name>:<tag>

4. Run the Operator.

   1. Install the CRD:

      $ make install

   2. Deploy the project to the cluster. Set IMG to the image that you pushed:

      $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

5. Create a sample custom resource (CR).

   1. Create a sample CR:

      $ oc apply -f config/samples/cache_v1_memcached.yaml \
          -n memcached-operator-system

   2. Watch for the CR to reconcile the Operator:

      $ oc logs deployment.apps/memcached-operator-controller-manager \
          -c manager \
          -n memcached-operator-system

6. Delete a CR

   Delete a CR by running the following command:

   $ oc delete -f config/samples/cache_v1_memcached -n memcached-operator-system

7. Clean up.

   Run the following command to clean up the resources that have been created as part of this procedure:

   $ make undeploy

Procedure

1. Create a directory for the project:

   $ mkdir -p $HOME/projects/memcached-operator

2. Change to the directory:

   $ cd $HOME/projects/memcached-operator

3. Activate support for Go modules:

   $ export GO111MODULE=on

4. Run the operator-sdk init command to initialize the project:

   $ operator-sdk init \
       --domain=example.com \
       --repo=github.com/example-inc/memcached-operator

   Note

   The operator-sdk init command uses the Go plugin by default.

   The operator-sdk init command generates a go.mod file to be used with Go modules. The --repo flag is required when creating a project outside of $GOPATH/src/, because generated files require a valid module path.

Procedure

1. Run the following command to create an API with group cache, version, v1, and kind Memcached:

   $ operator-sdk create api \
       --group=cache \
       --version=v1 \
       --kind=Memcached

2. When prompted, enter y for creating both the resource and controller:

   Create Resource [y/n]
   y
   Create Controller [y/n]
   y

   Example output

   Writing scaffold for you to edit...
   api/v1/memcached_types.go
   controllers/memcached_controller.go
   ...

Procedure

1. Edit the controllers/memcached_controller.go file to include the following:

   1. Import the proxy package from the operator-lib library:

      import (
        ...
         "github.com/operator-framework/operator-lib/proxy"
      )

   2. Add the proxy.ReadProxyVarsFromEnv helper function to the reconcile loop and append the results to the Operand environments:

      for i, container := range dep.Spec.Template.Spec.Containers {
      		dep.Spec.Template.Spec.Containers[i].Env = append(container.Env, proxy.ReadProxyVarsFromEnv()...)
      }
      ...

2. Set the environment variable on the Operator deployment by adding the following to the config/manager/manager.yaml file:

   containers:
    - args:
      - --leader-elect
      - --leader-election-id=ansible-proxy-demo
      image: controller:latest
      name: manager
      env:
        - name: "HTTP_PROXY"
          value: "http_proxy_test"

Procedure

1. Change to the namespace where your Operator is installed. For example, if you deployed the Operator using the make deploy command:

   $ oc project memcached-operator-system

2. Edit the sample Memcached CR manifest at config/samples/cache_v1_memcached.yaml to contain the following specification:

   apiVersion: cache.example.com/v1
   kind: Memcached
   metadata:
     name: memcached-sample
   ...
   spec:
   ...
     size: 3

3. Create the CR:

   $ oc apply -f config/samples/cache_v1_memcached.yaml

4. Ensure that the Memcached Operator creates the deployment for the sample CR with the correct size:

   $ oc get deployments

   Example output

   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   memcached-operator-controller-manager   1/1     1            1           8m
   memcached-sample                        3/3     3            3           1m

5. Check the pods and CR status to confirm the status is updated with the Memcached pod names.

   1. Check the pods:

      $ oc get pods

      Example output

      NAME                                  READY     STATUS    RESTARTS   AGE
      memcached-sample-6fd7c98d8-7dqdr      1/1       Running   0          1m
      memcached-sample-6fd7c98d8-g5k7v      1/1       Running   0          1m
      memcached-sample-6fd7c98d8-m7vn7      1/1       Running   0          1m

   2. Check the CR status:

      $ oc get memcached/memcached-sample -o yaml

      Example output

      apiVersion: cache.example.com/v1
      kind: Memcached
      metadata:
      ...
        name: memcached-sample
      ...
      spec:
        size: 3
      status:
        nodes:
        - memcached-sample-6fd7c98d8-7dqdr
        - memcached-sample-6fd7c98d8-g5k7v
        - memcached-sample-6fd7c98d8-m7vn7

6. Update the deployment size.

   1. Update config/samples/cache_v1_memcached.yaml file to change the spec.size field in the Memcached CR from 3 to 5:

      $ oc patch memcached memcached-sample \
          -p '{"spec":{"size": 5}}' \
          --type=merge

   2. Confirm that the Operator changes the deployment size:

      $ oc get deployments

      Example output

      NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator-controller-manager   1/1     1            1           10m
      memcached-sample                        5/5     5            5           3m

7. Delete the CR by running the following command:

   $ oc delete -f config/samples/cache_v1_memcached.yaml

8. Clean up the resources that have been created as part of this tutorial.

   • If you used the make deploy command to test the Operator, run the following command:

     $ make undeploy

   • If you used the operator-sdk run bundle command to test the Operator, run the following command:

     $ operator-sdk cleanup <project_name>

Procedure

1. Make the following changes to the config/default/manager_auth_proxy_patch.yaml file:

   ...
   spec:
     template:
       spec:
         containers:
         - name: kube-rbac-proxy
           image: registry.redhat.io/openshift4/ose-kube-rbac-proxy:v4.11 1
           args:
           - "--secure-listen-address=0.0.0.0:8443"
           - "--upstream=http://127.0.0.1:8080/"
           - "--logtostderr=true"
           - "--v=0" 2
   ...
   resources:
     limits:
       cpu: 500m
       memory: 128Mi
     requests:
       cpu: 5m
       memory: 64Mi 3

   1

   Update the tag version from v4.10 to v4.11.

   2

   Reduce the debugging log level from --v=10 to --v=0.

   3

   Add resource requests and limits.

2. Make the following changes to your Makefile:

   1. Enable support for image digests by adding the following environment variables to your Makefile:

      Old Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      ...

      New Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      
      # BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
      BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
      
      # USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
      # You can enable this value if you would like to use SHA Based Digests
      # To enable set flag to true
      USE_IMAGE_DIGESTS ?= false
      ifeq ($(USE_IMAGE_DIGESTS), true)
      	BUNDLE_GEN_FLAGS += --use-image-digests
      endif

   2. Edit your Makefile to replace the bundle target with the BUNDLE_GEN_FLAGS environment variable:

      Old Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)

      New Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle $(BUNDLE_GEN_FLAGS)

   3. Edit your Makefile to update opm to version 1.23.0:

      .PHONY: opm
      OPM = ./bin/opm
      opm: ## Download opm locally if necessary.
      ifeq (,$(wildcard $(OPM)))
      ifeq (,$(shell which opm 2>/dev/null))
      	@{ \
      	set -e ;\
      	mkdir -p $(dir $(OPM)) ;\
      	OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \
      	curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.23.0/$${OS}-$${ARCH}-opm ;\ 1
      	chmod +x $(OPM) ;\
      	}
      else
      OPM = $(shell which opm)
      endif
      endif

      1

      Replace v1.19.1 with v1.23.0.

   4. Edit your Makefile to replace the go get targets with go install targets:

      Old Makefile

      CONTROLLER_GEN = $(shell pwd)/bin/controller-gen
      .PHONY: controller-gen
      controller-gen: ## Download controller-gen locally if necessary.
      	$(call go-get-tool,$(CONTROLLER_GEN),sigs.k8s.io/controller-tools/cmd/controller-gen@v0.8.0)
      
      KUSTOMIZE = $(shell pwd)/bin/kustomize
      .PHONY: kustomize
      kustomize: ## Download kustomize locally if necessary.
      	$(call go-get-tool,$(KUSTOMIZE),sigs.k8s.io/kustomize/kustomize/v3@v3.8.7)
      
      ENVTEST = $(shell pwd)/bin/setup-envtest
      .PHONY: envtest
      envtest: ## Download envtest-setup locally if necessary.
      	$(call go-get-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest@latest)
      
      # go-get-tool will 'go get' any package $2 and install it to $1.
      PROJECT_DIR := $(shell dirname $(abspath $(lastword $(MAKEFILE_LIST))))
      define go-get-tool
      @[ -f $(1) ] || { \
      set -e ;\
      TMP_DIR=$$(mktemp -d) ;\
      cd $$TMP_DIR ;\
      go mod init tmp ;\
      echo "Downloading $(2)" ;\
      GOBIN=$(PROJECT_DIR)/bin go get $(2) ;\
      rm -rf $$TMP_DIR ;\
      }
      endef

      New Makefile

      ##@ Build Dependencies
      
      ## Location to install dependencies to
      LOCALBIN ?= $(shell pwd)/bin
      $(LOCALBIN):
      	mkdir -p $(LOCALBIN)
      
      ## Tool Binaries
      KUSTOMIZE ?= $(LOCALBIN)/kustomize
      CONTROLLER_GEN ?= $(LOCALBIN)/controller-gen
      ENVTEST ?= $(LOCALBIN)/setup-envtest
      
      ## Tool Versions
      KUSTOMIZE_VERSION ?= v3.8.7
      CONTROLLER_TOOLS_VERSION ?= v0.8.0
      
      KUSTOMIZE_INSTALL_SCRIPT ?= "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"
      .PHONY: kustomize
      kustomize: $(KUSTOMIZE) ## Download kustomize locally if necessary.
      $(KUSTOMIZE): $(LOCALBIN)
      	curl -s $(KUSTOMIZE_INSTALL_SCRIPT) | bash -s -- $(subst v,,$(KUSTOMIZE_VERSION)) $(LOCALBIN)
      
      .PHONY: controller-gen
      controller-gen: $(CONTROLLER_GEN) ## Download controller-gen locally if necessary.
      $(CONTROLLER_GEN): $(LOCALBIN)
      	GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-tools/cmd/controller-gen@$(CONTROLLER_TOOLS_VERSION)
      
      .PHONY: envtest
      envtest: $(ENVTEST) ## Download envtest-setup locally if necessary.
      $(ENVTEST): $(LOCALBIN)
      	GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest

   5. Update ENVTEST_K8S_VERSION and controller-gen fields in your Makefile to support Kubernetes 1.24:

      ...
      ENVTEST_K8S_VERSION = 1.24 1
      ...
      sigs.k8s.io/controller-tools/cmd/controller-gen@v0.9.0 2

      1

      Update version 1.22 to 1.24.

      2

      Update version 0.7.0 to 0.9.0.

   6. Apply the changes to your Makefile and rebuild your Operator by entering the following command:

      $ make

3. Make the following changes to the go.mod file to update Go and its dependencies:

   go 1.18 1
   
   require (
     github.com/onsi/ginkgo v1.16.5 2
     github.com/onsi/gomega v1.18.1 3
     k8s.io/api v0.24.0 4
     k8s.io/apimachinery v0.24.0 5
     k8s.io/client-go v0.24.0 6
     sigs.k8s.io/controller-runtime v0.12.1 7
   )

   1

   Update version 1.16 to 1.18.

   2

   Update version v1.16.4 to v1.16.5.

   3

   Update version v1.15.0 to v1.18.1.

   4 5 6

   Update version v0.22.1 to v0.24.0.

   7

   Update version v0.10.0 to v0.12.1.

4. Download and clean up the dependencies by entering the following command:

   $ go mod tidy

5. If you use the api/webhook_suitetest.go and controllers/suite_test.go suite test files, make the following changes:

   Old suite test file

   cfg, err := testEnv.Start()

   New suite test file

   var err error
   // cfg is defined in this file globally.
   cfg, err = testEnv.Start()

6. If you use the Kubernetes declarative plugin, update your Dockerfile with the following changes:

   1. Add the following changes below the line that begins COPY controllers/ controllers/:

      # https://github.com/kubernetes-sigs/kubebuilder-declarative-pattern/blob/master/docs/addon/walkthrough/README.md#adding-a-manifest
      # Stage channels and make readable
      COPY channels/ /channels/
      RUN chmod -R a+rx /channels/

   2. Add the following changes below the line that begins COPY --from=builder /workspace/manager .:

      # copy channels
      COPY --from=builder /channels /channels

Procedure

1. Create a project.

   1. Create your project directory:

      $ mkdir memcached-operator

   2. Change into the project directory:

      $ cd memcached-operator

   3. Run the operator-sdk init command with the ansible plugin to initialize the project:

      $ operator-sdk init \
          --plugins=ansible \
          --domain=example.com

2. Create an API.

   Create a simple Memcached API:

   $ operator-sdk create api \
       --group cache \
       --version v1 \
       --kind Memcached \
       --generate-role 1

   1

   Generates an Ansible role for the API.

3. Build and push the Operator image.

   Use the default Makefile targets to build and push your Operator. Set IMG with a pull spec for your image that uses a registry you can push to:

   $ make docker-build docker-push IMG=<registry>/<user>/<image_name>:<tag>

4. Run the Operator.

   1. Install the CRD:

      $ make install

   2. Deploy the project to the cluster. Set IMG to the image that you pushed:

      $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

5. Create a sample custom resource (CR).

   1. Create a sample CR:

      $ oc apply -f config/samples/cache_v1_memcached.yaml \
          -n memcached-operator-system

   2. Watch for the CR to reconcile the Operator:

      $ oc logs deployment.apps/memcached-operator-controller-manager \
          -c manager \
          -n memcached-operator-system

      Example output

      ...
      I0205 17:48:45.881666       7 leaderelection.go:253] successfully acquired lease memcached-operator-system/memcached-operator
      {"level":"info","ts":1612547325.8819902,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting EventSource","source":"kind source: cache.example.com/v1, Kind=Memcached"}
      {"level":"info","ts":1612547325.98242,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting Controller"}
      {"level":"info","ts":1612547325.9824686,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting workers","worker count":4}
      {"level":"info","ts":1612547348.8311093,"logger":"runner","msg":"Ansible-runner exited successfully","job":"4037200794235010051","name":"memcached-sample","namespace":"memcached-operator-system"}

6. Delete a CR

   Delete a CR by running the following command:

   $ oc delete -f config/samples/cache_v1_memcached -n memcached-operator-system

7. Clean up.

   Run the following command to clean up the resources that have been created as part of this procedure:

   $ make undeploy

Procedure

1. Create a directory for the project:

   $ mkdir -p $HOME/projects/memcached-operator

2. Change to the directory:

   $ cd $HOME/projects/memcached-operator

3. Run the operator-sdk init command with the ansible plugin to initialize the project:

   $ operator-sdk init \
       --plugins=ansible \
       --domain=example.com

Procedure

1. Update the roles/memcached/tasks/main.yml file with the following structure:

   ---
   - name: start memcached
     k8s:
       definition:
         kind: Deployment
         apiVersion: apps/v1
         metadata:
           name: '{{ ansible_operator_meta.name }}-memcached'
           namespace: '{{ ansible_operator_meta.namespace }}'
         spec:
           replicas: "{{size}}"
           selector:
             matchLabels:
               app: memcached
           template:
             metadata:
               labels:
                 app: memcached
             spec:
               containers:
               - name: memcached
                 command:
                 - memcached
                 - -m=64
                 - -o
                 - modern
                 - -v
                 image: "docker.io/memcached:1.4.36-alpine"
                 ports:
                   - containerPort: 11211

   This memcached role ensures a memcached deployment exist and sets the deployment size.

2. Set default values for variables used in your Ansible role by editing the roles/memcached/defaults/main.yml file:

   ---
   # defaults file for Memcached
   size: 1

3. Update the Memcached sample resource in the config/samples/cache_v1_memcached.yaml file with the following structure:

   apiVersion: cache.example.com/v1
   kind: Memcached
   metadata:
     labels:
       app.kubernetes.io/name: memcached
       app.kubernetes.io/instance: memcached-sample
       app.kubernetes.io/part-of: memcached-operator
       app.kubernetes.io/managed-by: kustomize
       app.kubernetes.io/created-by: memcached-operator
     name: memcached-sample
   spec:
     size: 3

   The key-value pairs in the custom resource (CR) spec are passed to Ansible as extra variables.

Procedure

1. Add the environment variables to the deployment by updating the roles/memcached/tasks/main.yml file with the following:

   ...
   env:
      - name: HTTP_PROXY
        value: '{{ lookup("env", "HTTP_PROXY") | default("", True) }}'
      - name: http_proxy
        value: '{{ lookup("env", "HTTP_PROXY") | default("", True) }}'
   ...

2. Set the environment variable on the Operator deployment by adding the following to the config/manager/manager.yaml file:

   containers:
    - args:
      - --leader-elect
      - --leader-election-id=ansible-proxy-demo
      image: controller:latest
      name: manager
      env:
        - name: "HTTP_PROXY"
          value: "http_proxy_test"

Procedure

1. Change to the namespace where your Operator is installed. For example, if you deployed the Operator using the make deploy command:

   $ oc project memcached-operator-system

2. Edit the sample Memcached CR manifest at config/samples/cache_v1_memcached.yaml to contain the following specification:

   apiVersion: cache.example.com/v1
   kind: Memcached
   metadata:
     name: memcached-sample
   ...
   spec:
   ...
     size: 3

3. Create the CR:

   $ oc apply -f config/samples/cache_v1_memcached.yaml

4. Ensure that the Memcached Operator creates the deployment for the sample CR with the correct size:

   $ oc get deployments

   Example output

   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   memcached-operator-controller-manager   1/1     1            1           8m
   memcached-sample                        3/3     3            3           1m

5. Check the pods and CR status to confirm the status is updated with the Memcached pod names.

   1. Check the pods:

      $ oc get pods

      Example output

      NAME                                  READY     STATUS    RESTARTS   AGE
      memcached-sample-6fd7c98d8-7dqdr      1/1       Running   0          1m
      memcached-sample-6fd7c98d8-g5k7v      1/1       Running   0          1m
      memcached-sample-6fd7c98d8-m7vn7      1/1       Running   0          1m

   2. Check the CR status:

      $ oc get memcached/memcached-sample -o yaml

      Example output

      apiVersion: cache.example.com/v1
      kind: Memcached
      metadata:
      ...
        name: memcached-sample
      ...
      spec:
        size: 3
      status:
        nodes:
        - memcached-sample-6fd7c98d8-7dqdr
        - memcached-sample-6fd7c98d8-g5k7v
        - memcached-sample-6fd7c98d8-m7vn7

6. Update the deployment size.

   1. Update config/samples/cache_v1_memcached.yaml file to change the spec.size field in the Memcached CR from 3 to 5:

      $ oc patch memcached memcached-sample \
          -p '{"spec":{"size": 5}}' \
          --type=merge

   2. Confirm that the Operator changes the deployment size:

      $ oc get deployments

      Example output

      NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator-controller-manager   1/1     1            1           10m
      memcached-sample                        5/5     5            5           3m

7. Delete the CR by running the following command:

   $ oc delete -f config/samples/cache_v1_memcached.yaml

8. Clean up the resources that have been created as part of this tutorial.

   • If you used the make deploy command to test the Operator, run the following command:

     $ make undeploy

   • If you used the operator-sdk run bundle command to test the Operator, run the following command:

     $ operator-sdk cleanup <project_name>

Procedure

1. Make the following changes to the config/default/manager_auth_proxy_patch.yaml file:

   ...
   spec:
     template:
       spec:
         containers:
         - name: kube-rbac-proxy
           image: registry.redhat.io/openshift4/ose-kube-rbac-proxy:v4.11 1
           args:
           - "--secure-listen-address=0.0.0.0:8443"
           - "--upstream=http://127.0.0.1:8080/"
           - "--logtostderr=true"
           - "--v=0" 2
   ...
   resources:
     limits:
       cpu: 500m
       memory: 128Mi
     requests:
       cpu: 5m
       memory: 64Mi 3

   1

   Update the tag version from v4.10 to v4.11.

   2

   Reduce the debugging log level from --v=10 to --v=0.

   3

   Add resource requests and limits.

2. Make the following changes to your Makefile:

   1. Enable support for image digests by adding the following environment variables to your Makefile:

      Old Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      ...

      New Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      
      # BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
      BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
      
      # USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
      # You can enable this value if you would like to use SHA Based Digests
      # To enable set flag to true
      USE_IMAGE_DIGESTS ?= false
      ifeq ($(USE_IMAGE_DIGESTS), true)
      	BUNDLE_GEN_FLAGS += --use-image-digests
      endif

   2. Edit your Makefile to replace the bundle target with the BUNDLE_GEN_FLAGS environment variable:

      Old Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)

      New Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle $(BUNDLE_GEN_FLAGS)

   3. Edit your Makefile to update opm to version 1.23.0:

      .PHONY: opm
      OPM = ./bin/opm
      opm: ## Download opm locally if necessary.
      ifeq (,$(wildcard $(OPM)))
      ifeq (,$(shell which opm 2>/dev/null))
      	@{ \
      	set -e ;\
      	mkdir -p $(dir $(OPM)) ;\
      	OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \
      	curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.23.0/$${OS}-$${ARCH}-opm ;\ 1
      	chmod +x $(OPM) ;\
      	}
      else
      OPM = $(shell which opm)
      endif
      endif

      1

      Replace v1.19.1 with v1.23.0.

   4. Apply the changes to your Makefile and rebuild your Operator by entering the following command:

      $ make

3. Update the image tag in your Operator’s Dockerfile as shown in the following example:

   Example Dockerfile

   FROM registry.redhat.io/openshift4/ose-ansible-operator:v4.11 1

   1

   Update the version tag to v4.11.

4. Update your requirements.yml file as shown in the following example:

   collections:
     - name: community.kubernetes
       version: "2.0.1" 1
     - name: operator_sdk.util
       version: "0.4.0" 2
     - name: kubernetes.core
       version: "2.3.1" 3
     - name: cloud.common 4
       version: "2.1.1"

   1

   Update version 1.2.1 to 2.0.1.

   2

   Update version 0.3.1 to 0.4.0.

   3

   Update version 2.2.0 to 2.3.1.

   4

   Add support for the Operator Ansible SDK by adding the cloud.common collection.

   Important

   As of version 2.0.0, the community.kubernetes collection was renamed to kubernetes.core. The community.kubernetes collection has been replaced by deprecated redirects to kubernetes.core. If you use fully qualified collection names (FQCNs) that begin with community.kubernetes, you must update the FQCNs to use kubernetes.core.

Procedure

1. Install Ansible 2.9+:

   $ sudo dnf install ansible

2. Install the OpenShift python client package:

   $ pip3 install openshift

3. Install the Kubernetes Collection using one of the following methods:

   • You can install the collection directly from Ansible Galaxy:

     $ ansible-galaxy collection install community.kubernetes

   • If you have already initialized your Operator, you might have a requirements.yml file at the top level of your project. This file specifies Ansible dependencies that must be installed for your Operator to function. By default, this file installs the community.kubernetes collection as well as the operator_sdk.util collection, which provides modules and plugins for Operator-specific fuctions.

     To install the dependent modules from the requirements.yml file:

     $ ansible-galaxy collection install -r requirements.yml

Procedure

1. In your Ansible-based Operator project directory, modify the roles/<kind>/tasks/main.yml file with the Ansible logic that you want. The roles/<kind>/ directory is created when you use the --generate-role flag while creating an API. The <kind> replaceable matches the kind that you specified for the API.

   The following example creates and deletes a config map based on the value of a variable named state:

   ---
   - name: set ConfigMap example-config to {{ state }}
     community.kubernetes.k8s:
       api_version: v1
       kind: ConfigMap
       name: example-config
       namespace: default 1
       state: "{{ state }}"
     ignore_errors: true 2

   1

   Change this value if you want the config map to be created in a different namespace from default.

   2

   Setting ignore_errors: true ensures that deleting a nonexistent config map does not fail.

2. Modify the roles/<kind>/defaults/main.yml file to set state to present by default:

   ---
   state: present

3. Create an Ansible playbook by creating a playbook.yml file in the top-level of your project directory, and include your <kind> role:

   ---
   - hosts: localhost
     roles:
       - <kind>

4. Run the playbook:

   $ ansible-playbook playbook.yml

   Example output

   [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
   
   PLAY [localhost] ********************************************************************************
   
   TASK [Gathering Facts] ********************************************************************************
   ok: [localhost]
   
   TASK [memcached : set ConfigMap example-config to present] ********************************************************************************
   changed: [localhost]
   
   PLAY RECAP ********************************************************************************
   localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

5. Verify that the config map was created:

   $ oc get configmaps

   Example output

   NAME               DATA   AGE
   example-config     0      2m1s

6. Rerun the playbook setting state to absent:

   $ ansible-playbook playbook.yml --extra-vars state=absent

   Example output

   [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
   
   PLAY [localhost] ********************************************************************************
   
   TASK [Gathering Facts] ********************************************************************************
   ok: [localhost]
   
   TASK [memcached : set ConfigMap example-config to absent] ********************************************************************************
   changed: [localhost]
   
   PLAY RECAP ********************************************************************************
   localhost                  : ok=2    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

7. Verify that the config map was deleted:

   $ oc get configmaps

Procedure

1. Install your custom resource definition (CRD) and proper role-based access control (RBAC) definitions for your custom resource (CR):

   $ make install

   Example output

   /usr/bin/kustomize build config/crd | kubectl apply -f -
   customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created

2. Run the make run command:

   $ make run

   Example output

   /home/user/memcached-operator/bin/ansible-operator run
   {"level":"info","ts":1612739145.2871568,"logger":"cmd","msg":"Version","Go Version":"go1.15.5","GOOS":"linux","GOARCH":"amd64","ansible-operator":"v1.10.1","commit":"1abf57985b43bf6a59dcd18147b3c574fa57d3f6"}
   ...
   {"level":"info","ts":1612739148.347306,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":":8080"}
   {"level":"info","ts":1612739148.3488882,"logger":"watches","msg":"Environment variable not set; using default value","envVar":"ANSIBLE_VERBOSITY_MEMCACHED_CACHE_EXAMPLE_COM","default":2}
   {"level":"info","ts":1612739148.3490262,"logger":"cmd","msg":"Environment variable not set; using default value","Namespace":"","envVar":"ANSIBLE_DEBUG_LOGS","ANSIBLE_DEBUG_LOGS":false}
   {"level":"info","ts":1612739148.3490646,"logger":"ansible-controller","msg":"Watching resource","Options.Group":"cache.example.com","Options.Version":"v1","Options.Kind":"Memcached"}
   {"level":"info","ts":1612739148.350217,"logger":"proxy","msg":"Starting to serve","Address":"127.0.0.1:8888"}
   {"level":"info","ts":1612739148.3506632,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
   {"level":"info","ts":1612739148.350784,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting EventSource","source":"kind source: cache.example.com/v1, Kind=Memcached"}
   {"level":"info","ts":1612739148.5511978,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting Controller"}
   {"level":"info","ts":1612739148.5512562,"logger":"controller-runtime.manager.controller.memcached-controller","msg":"Starting workers","worker count":8}

   With the Operator now watching your CR for events, the creation of a CR will trigger your Ansible role to run.

   Note

   Consider an example config/samples/<gvk>.yaml CR manifest:

   apiVersion: <group>.example.com/v1alpha1
   kind: <kind>
   metadata:
     name: "<kind>-sample"

   Because the spec field is not set, Ansible is invoked with no extra variables. Passing extra variables from a CR to Ansible is covered in another section. It is important to set reasonable defaults for the Operator.

3. Create an instance of your CR with the default variable state set to present:

   $ oc apply -f config/samples/<gvk>.yaml

4. Check that the example-config config map was created:

   $ oc get configmaps

   Example output

   NAME                    STATUS    AGE
   example-config          Active    3s

5. Modify your config/samples/<gvk>.yaml file to set the state field to absent. For example:

   apiVersion: cache.example.com/v1
   kind: Memcached
   metadata:
     name: memcached-sample
   spec:
     state: absent

6. Apply the changes:

   $ oc apply -f config/samples/<gvk>.yaml

7. Confirm that the config map is deleted:

   $ oc get configmap

Procedure

1. Run the following make commands to build and push the Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

   1. Build the image:

      $ make docker-build IMG=<registry>/<user>/<image_name>:<tag>

      Note

      The Dockerfile generated by the SDK for the Operator explicitly references GOARCH=amd64 for go build. This can be amended to GOARCH=$TARGETARCH for non-AMD64 architectures. Docker will automatically set the environment variable to the value specified by –platform. With Buildah, the –build-arg will need to be used for the purpose. For more information, see Multiple Architectures.

   2. Push the image to a repository:

      $ make docker-push IMG=<registry>/<user>/<image_name>:<tag>

      Note

      The name and tag of the image, for example IMG=<registry>/<user>/<image_name>:<tag>, in both the commands can also be set in your Makefile. Modify the IMG ?= controller:latest value to set your default image name.

2. Run the following command to deploy the Operator:

   $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

   By default, this command creates a namespace with the name of your Operator project in the form <project_name>-system and is used for the deployment. This command also installs the RBAC manifests from config/rbac.

3. Run the following command to verify that the Operator is running:

   $ oc get deployment -n <project_name>-system

   Example output

   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   <project_name>-controller-manager       1/1     1            1           8m

Procedure

1. Update the watches.yaml file with a manageStatus field set to false:

   - version: v1
     group: api.example.com
     kind: <kind>
     role: <role>
     manageStatus: false

2. Use the operator_sdk.util.k8s_status Ansible module to update the subresource. For example, to update with key test and value data, operator_sdk.util can be used as shown:

   - operator_sdk.util.k8s_status:
       api_version: app.example.com/v1
       kind: <kind>
       name: "{{ ansible_operator_meta.name }}"
       namespace: "{{ ansible_operator_meta.namespace }}"
       status:
         test: data

3. You can declare collections in the meta/main.yml file for the role, which is included for scaffolded Ansible-based Operators:

   collections:
     - operator_sdk.util

4. After declaring collections in the role meta, you can invoke the k8s_status module directly:

   k8s_status:
     ...
     status:
       key1: value1

Procedure

1. Create a project.

   1. Create your project directory:

      $ mkdir nginx-operator

   2. Change into the project directory:

      $ cd nginx-operator

   3. Run the operator-sdk init command with the helm plugin to initialize the project:

      $ operator-sdk init \
          --plugins=helm

2. Create an API.

   Create a simple Nginx API:

   $ operator-sdk create api \
       --group demo \
       --version v1 \
       --kind Nginx

   This API uses the built-in Helm chart boilerplate from the helm create command.

3. Build and push the Operator image.

   Use the default Makefile targets to build and push your Operator. Set IMG with a pull spec for your image that uses a registry you can push to:

   $ make docker-build docker-push IMG=<registry>/<user>/<image_name>:<tag>

4. Run the Operator.

   1. Install the CRD:

      $ make install

   2. Deploy the project to the cluster. Set IMG to the image that you pushed:

      $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

5. Add a security context constraint (SCC).

   The Nginx service account requires privileged access to run in OpenShift Container Platform. Add the following SCC to the service account for the nginx-sample pod:

   $ oc adm policy add-scc-to-user \
       anyuid system:serviceaccount:nginx-operator-system:nginx-sample

6. Create a sample custom resource (CR).

   1. Create a sample CR:

      $ oc apply -f config/samples/demo_v1_nginx.yaml \
          -n nginx-operator-system

   2. Watch for the CR to reconcile the Operator:

      $ oc logs deployment.apps/nginx-operator-controller-manager \
          -c manager \
          -n nginx-operator-system

7. Delete a CR

   Delete a CR by running the following command:

   $ oc delete -f config/samples/demo_v1_nginx -n nginx-operator-system

8. Clean up.

   Run the following command to clean up the resources that have been created as part of this procedure:

   $ make undeploy

Procedure

1. Create a directory for the project:

   $ mkdir -p $HOME/projects/nginx-operator

2. Change to the directory:

   $ cd $HOME/projects/nginx-operator

3. Run the operator-sdk init command with the helm plugin to initialize the project:

   $ operator-sdk init \
       --plugins=helm \
       --domain=example.com \
       --group=demo \
       --version=v1 \
       --kind=Nginx

   Note

   By default, the helm plugin initializes a project using a boilerplate Helm chart. You can use additional flags, such as the --helm-chart flag, to initialize a project using an existing Helm chart.

   The init command creates the nginx-operator project specifically for watching a resource with API version example.com/v1 and kind Nginx.

4. For Helm-based projects, the init command generates the RBAC rules in the config/rbac/role.yaml file based on the resources that would be deployed by the default manifest for the chart. Verify that the rules generated in this file meet the permission requirements of the Operator.

Procedure

1. Edit the watches.yaml file to include overrides based on an environment variable by adding the overrideValues field:

   ...
   - group: demo.example.com
     version: v1alpha1
     kind: Nginx
     chart: helm-charts/nginx
     overrideValues:
       proxy.http: $HTTP_PROXY
   ...

2. Add the proxy.http value in the helm-charts/nginx/values.yaml file:

   ...
   proxy:
     http: ""
     https: ""
     no_proxy: ""

3. To make sure the chart template supports using the variables, edit the chart template in the helm-charts/nginx/templates/deployment.yaml file to contain the following:

   containers:
     - name: {{ .Chart.Name }}
       securityContext:
         - toYaml {{ .Values.securityContext | nindent 12 }}
       image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
       imagePullPolicy: {{ .Values.image.pullPolicy }}
       env:
         - name: http_proxy
           value: "{{ .Values.proxy.http }}"

4. Set the environment variable on the Operator deployment by adding the following to the config/manager/manager.yaml file:

   containers:
    - args:
      - --leader-elect
      - --leader-election-id=ansible-proxy-demo
      image: controller:latest
      name: manager
      env:
        - name: "HTTP_PROXY"
          value: "http_proxy_test"

Procedure

1. Change to the namespace where your Operator is installed. For example, if you deployed the Operator using the make deploy command:

   $ oc project nginx-operator-system

2. Edit the sample Nginx CR manifest at config/samples/demo_v1_nginx.yaml to contain the following specification:

   apiVersion: demo.example.com/v1
   kind: Nginx
   metadata:
     name: nginx-sample
   ...
   spec:
   ...
     replicaCount: 3

3. The Nginx service account requires privileged access to run in OpenShift Container Platform. Add the following security context constraint (SCC) to the service account for the nginx-sample pod:

   $ oc adm policy add-scc-to-user \
       anyuid system:serviceaccount:nginx-operator-system:nginx-sample

4. Create the CR:

   $ oc apply -f config/samples/demo_v1_nginx.yaml

5. Ensure that the Nginx Operator creates the deployment for the sample CR with the correct size:

   $ oc get deployments

   Example output

   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   nginx-operator-controller-manager       1/1     1            1           8m
   nginx-sample                            3/3     3            3           1m

6. Check the pods and CR status to confirm the status is updated with the Nginx pod names.

   1. Check the pods:

      $ oc get pods

      Example output

      NAME                                  READY     STATUS    RESTARTS   AGE
      nginx-sample-6fd7c98d8-7dqdr          1/1       Running   0          1m
      nginx-sample-6fd7c98d8-g5k7v          1/1       Running   0          1m
      nginx-sample-6fd7c98d8-m7vn7          1/1       Running   0          1m

   2. Check the CR status:

      $ oc get nginx/nginx-sample -o yaml

      Example output

      apiVersion: demo.example.com/v1
      kind: Nginx
      metadata:
      ...
        name: nginx-sample
      ...
      spec:
        replicaCount: 3
      status:
        nodes:
        - nginx-sample-6fd7c98d8-7dqdr
        - nginx-sample-6fd7c98d8-g5k7v
        - nginx-sample-6fd7c98d8-m7vn7

7. Update the deployment size.

   1. Update config/samples/demo_v1_nginx.yaml file to change the spec.size field in the Nginx CR from 3 to 5:

      $ oc patch nginx nginx-sample \
          -p '{"spec":{"replicaCount": 5}}' \
          --type=merge

   2. Confirm that the Operator changes the deployment size:

      $ oc get deployments

      Example output

      NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
      nginx-operator-controller-manager       1/1     1            1           10m
      nginx-sample                            5/5     5            5           3m

8. Delete the CR by running the following command:

   $ oc delete -f config/samples/demo_v1_nginx.yaml

9. Clean up the resources that have been created as part of this tutorial.

   • If you used the make deploy command to test the Operator, run the following command:

     $ make undeploy

   • If you used the operator-sdk run bundle command to test the Operator, run the following command:

     $ operator-sdk cleanup <project_name>

Procedure

1. Make the following changes to the config/default/manager_auth_proxy_patch.yaml file:

   ...
   spec:
     template:
       spec:
         containers:
         - name: kube-rbac-proxy
           image: registry.redhat.io/openshift4/ose-kube-rbac-proxy:v4.11 1
           args:
           - "--secure-listen-address=0.0.0.0:8443"
           - "--upstream=http://127.0.0.1:8080/"
           - "--logtostderr=true"
           - "--v=0" 2
   ...
   resources:
     limits:
       cpu: 500m
       memory: 128Mi
     requests:
       cpu: 5m
       memory: 64Mi 3

   1

   Update the tag version from v4.10 to v4.11.

   2

   Reduce the debugging log level from --v=10 to --v=0.

   3

   Add resource requests and limits.

2. Make the following changes to your Makefile:

   1. Enable support for image digests by adding the following environment variables to your Makefile:

      Old Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      ...

      New Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      
      # BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
      BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
      
      # USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
      # You can enable this value if you would like to use SHA Based Digests
      # To enable set flag to true
      USE_IMAGE_DIGESTS ?= false
      ifeq ($(USE_IMAGE_DIGESTS), true)
      	BUNDLE_GEN_FLAGS += --use-image-digests
      endif

   2. Edit your Makefile to replace the bundle target with the BUNDLE_GEN_FLAGS environment variable:

      Old Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)

      New Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle $(BUNDLE_GEN_FLAGS)

   3. Edit your Makefile to update opm to version 1.23.0:

      .PHONY: opm
      OPM = ./bin/opm
      opm: ## Download opm locally if necessary.
      ifeq (,$(wildcard $(OPM)))
      ifeq (,$(shell which opm 2>/dev/null))
      	@{ \
      	set -e ;\
      	mkdir -p $(dir $(OPM)) ;\
      	OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \
      	curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.23.0/$${OS}-$${ARCH}-opm ;\ 1
      	chmod +x $(OPM) ;\
      	}
      else
      OPM = $(shell which opm)
      endif
      endif

      1

      Replace v1.19.1 with v1.23.0.

   4. Apply the changes to your Makefile and rebuild your Operator by entering the following command:

      $ make

3. Update the image tag in your Operator’s Dockerfile as shown in the following example:

   Example Dockerfile

   FROM registry.redhat.io/openshift4/ose-helm-operator:v4.11 1

   1

   Update the version tag to v4.11.

Procedure

1. Create a directory for the project:

   $ mkdir -p $HOME/github.com/example/memcached-operator

2. Change to the directory:

   $ cd $HOME/github.com/example/memcached-operator

3. Run the operator-sdk init command to initialize the project. Use a domain of example.com so that all API groups are <group>.example.com:

   $ operator-sdk init \
       --plugins=hybrid.helm.sdk.operatorframework.io \
       --project-version="3" \
       --domain example.com \
       --repo=github.com/example/memcached-operator

   The init command generates the RBAC rules in the config/rbac/role.yaml file based on the resources that would be deployed by the chart’s default manifests. Verify that the rules generated in the config/rbac/role.yaml file meet your Operator’s permission requirements.

Procedure

1. Run the following command to create a Go API with group cache, version v1, and kind MemcachedBackup:

   $ operator-sdk create api \
       --group=cache \
       --version v1 \
       --kind MemcachedBackup \
       --resource \
       --controller \
       --plugins=go/v3

2. When prompted, enter y for creating both resource and controller:

   $ Create Resource [y/n]
   y
   Create Controller [y/n]
   y

Procedure

1. Run the following make commands to build and push the Operator image. Modify the IMG argument in the following steps to reference a repository that you have access to. You can obtain an account for storing containers at repository sites such as Quay.io.

   1. Build the image:

      $ make docker-build IMG=<registry>/<user>/<image_name>:<tag>

      Note

      The Dockerfile generated by the SDK for the Operator explicitly references GOARCH=amd64 for go build. This can be amended to GOARCH=$TARGETARCH for non-AMD64 architectures. Docker will automatically set the environment variable to the value specified by –platform. With Buildah, the –build-arg will need to be used for the purpose. For more information, see Multiple Architectures.

   2. Push the image to a repository:

      $ make docker-push IMG=<registry>/<user>/<image_name>:<tag>

      Note

      The name and tag of the image, for example IMG=<registry>/<user>/<image_name>:<tag>, in both the commands can also be set in your Makefile. Modify the IMG ?= controller:latest value to set your default image name.

2. Run the following command to deploy the Operator:

   $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

   By default, this command creates a namespace with the name of your Operator project in the form <project_name>-system and is used for the deployment. This command also installs the RBAC manifests from config/rbac.

3. Run the following command to verify that the Operator is running:

   $ oc get deployment -n <project_name>-system

   Example output

   NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
   <project_name>-controller-manager       1/1     1            1           8m

Procedure

1. Change to the namespace where your Operator is installed:

   $ oc project <project_name>-system

2. Update the sample Memcached CR manifest at the config/samples/cache_v1_memcached.yaml file by updating the replicaCount field to 3:

   Example 5.7. Example config/samples/cache_v1_memcached.yaml file

   apiVersion: cache.my.domain/v1
   kind: Memcached
   metadata:
     name: memcached-sample
   spec:
     # Default values copied from <project_dir>/helm-charts/memcached/values.yaml
     affinity: {}
     autoscaling:
       enabled: false
       maxReplicas: 100
       minReplicas: 1
       targetCPUUtilizationPercentage: 80
     fullnameOverride: ""
     image:
       pullPolicy: IfNotPresent
       repository: nginx
       tag: ""
     imagePullSecrets: []
     ingress:
       annotations: {}
       className: ""
       enabled: false
       hosts:
       - host: chart-example.local
         paths:
         - path: /
           pathType: ImplementationSpecific
       tls: []
     nameOverride: ""
     nodeSelector: {}
     podAnnotations: {}
     podSecurityContext: {}
     replicaCount: 3
     resources: {}
     securityContext: {}
     service:
       port: 80
       type: ClusterIP
     serviceAccount:
       annotations: {}
       create: true
       name: ""
     tolerations: []

3. Create the Memcached CR:

   $ oc apply -f config/samples/cache_v1_memcached.yaml

4. Ensure that the Memcached Operator creates the deployment for the sample CR with the correct size:

   $ oc get pods

   Example output

   NAME                                  READY     STATUS    RESTARTS   AGE
   memcached-sample-6fd7c98d8-7dqdr      1/1       Running   0          18m
   memcached-sample-6fd7c98d8-g5k7v      1/1       Running   0          18m
   memcached-sample-6fd7c98d8-m7vn7      1/1       Running   0          18m

5. Update the sample MemcachedBackup CR manifest at the config/samples/cache_v1_memcachedbackup.yaml file by updating the size to 2:

   Example 5.8. Example config/samples/cache_v1_memcachedbackup.yaml file

   apiVersion: cache.my.domain/v1
   kind: MemcachedBackup
   metadata:
     name: memcachedbackup-sample
   spec:
     size: 2

6. Create the MemcachedBackup CR:

   $ oc apply -f config/samples/cache_v1_memcachedbackup.yaml

7. Ensure that the count of memcachedbackup pods is the same as specified in the CR:

   $ oc get pods

   Example output

   NAME                                        READY     STATUS    RESTARTS   AGE
   memcachedbackup-sample-8649699989-4bbzg     1/1       Running   0          22m
   memcachedbackup-sample-8649699989-mq6mx     1/1       Running   0          22m

8. You can update the spec in each of the above CRs, and then apply them again. The controller reconciles again and ensures that the size of the pods is as specified in the spec of the respective CRs.

9. Clean up the resources that have been created as part of this tutorial:

   1. Delete the Memcached resource:

      $ oc delete -f config/samples/cache_v1_memcached.yaml

   2. Delete the MemcachedBackup resource:

      $ oc delete -f config/samples/cache_v1_memcachedbackup.yaml

   3. If you used the make deploy command to test the Operator, run the following command:

      $ make undeploy

Procedure

1. Make the following changes to the config/default/manager_auth_proxy_patch.yaml file:

   ...
   spec:
     template:
       spec:
         containers:
         - name: kube-rbac-proxy
           image: registry.redhat.io/openshift4/ose-kube-rbac-proxy:v4.11 1
           args:
           - "--secure-listen-address=0.0.0.0:8443"
           - "--upstream=http://127.0.0.1:8080/"
           - "--logtostderr=true"
           - "--v=0" 2
   ...
   resources:
     limits:
       cpu: 500m
       memory: 128Mi
     requests:
       cpu: 5m
       memory: 64Mi 3

   1

   Update the tag version from v4.10 to v4.11.

   2

   Reduce the debugging log level from --v=10 to --v=0.

   3

   Add resource requests and limits.

2. Make the following changes to your Makefile:

   1. Enable support for image digests by adding the following environment variables to your Makefile:

      Old Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      ...

      New Makefile

      BUNDLE_IMG ?= $(IMAGE_TAG_BASE)-bundle:v$(VERSION)
      
      # BUNDLE_GEN_FLAGS are the flags passed to the operator-sdk generate bundle command
      BUNDLE_GEN_FLAGS ?= -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)
      
      # USE_IMAGE_DIGESTS defines if images are resolved via tags or digests
      # You can enable this value if you would like to use SHA Based Digests
      # To enable set flag to true
      USE_IMAGE_DIGESTS ?= false
      ifeq ($(USE_IMAGE_DIGESTS), true)
      	BUNDLE_GEN_FLAGS += --use-image-digests
      endif

   2. Edit your Makefile to replace the bundle target with the BUNDLE_GEN_FLAGS environment variable:

      Old Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle -q --overwrite --version $(VERSION) $(BUNDLE_METADATA_OPTS)

      New Makefile

      $(KUSTOMIZE) build config/manifests | operator-sdk generate bundle $(BUNDLE_GEN_FLAGS)

   3. Edit your Makefile to update opm to version 1.23.0:

      .PHONY: opm
      OPM = ./bin/opm
      opm: ## Download opm locally if necessary.
      ifeq (,$(wildcard $(OPM)))
      ifeq (,$(shell which opm 2>/dev/null))
      	@{ \
      	set -e ;\
      	mkdir -p $(dir $(OPM)) ;\
      	OS=$(shell go env GOOS) && ARCH=$(shell go env GOARCH) && \
      	curl -sSLo $(OPM) https://github.com/operator-framework/operator-registry/releases/download/v1.23.0/$${OS}-$${ARCH}-opm ;\ 1
      	chmod +x $(OPM) ;\
      	}
      else
      OPM = $(shell which opm)
      endif
      endif

      1

      Replace v1.19.1 with v1.23.0.

   4. Update ENVTEST_K8S_VERSION and controller-gen fields in your Makefile to support Kubernetes 1.24:

      ...
      ENVTEST_K8S_VERSION = 1.24 1
      ...
      sigs.k8s.io/controller-tools/cmd/controller-gen@v0.9.0 2

      1

      Update version 1.22 to 1.24.

      2

      Update version 0.7.0 to 0.9.0.

   5. Apply the changes to your Makefile and rebuild your Operator by entering the following command:

      $ make

3. Make the following changes to the go.mod file to update Go and its dependencies:

   go 1.18 1
   
   require (
     github.com/onsi/ginkgo v1.16.5 2
     github.com/onsi/gomega v1.18.1 3
     k8s.io/api v0.24.0 4
     k8s.io/apimachinery v0.24.0 5
     k8s.io/client-go v0.24.0 6
     sigs.k8s.io/controller-runtime v0.12.1 7
   )

   1

   Update version 1.16 to 1.18.

   2

   Update version v1.16.4 to v1.16.5.

   3

   Update version v1.15.0 to v1.18.1.

   4 5 6

   Update version v0.22.1 to v0.24.0.

   7

   Update version v0.10.0 to v0.12.1.

4. Edit your go.mod file to update the Helm Operator plugins:

   github.com/operator-framework/helm-operator-plugins v0.0.11 1

   1

   Update version v0.0.8 to v0.0.11.

5. Make the following changes to your Dockerfile to update Go to version 1.18:

   Old dockerfile.go file

   const dockerfileTemplate = `# Build the manager binary
   FROM golang:1.17 as builder

   New dockerfile.go file

   const dockerfileTemplate = `# Build the manager binary
   FROM golang:1.18 as builder

6. Download and clean up the dependencies by entering the following command:

   $ go mod tidy

Procedure

1. Create a project.

   1. Create your project directory:

      $ mkdir memcached-operator

   2. Change into the project directory:

      $ cd memcached-operator

   3. Run the operator-sdk init command with the quarkus plugin to initialize the project:

      $ operator-sdk init \
          --plugins=quarkus \
          --domain=example.com \
          --project-name=memcached-operator

2. Create an API.

   Create a simple Memcached API:

   $ operator-sdk create api \
       --plugins quarkus \
       --group cache \
       --version v1 \
       --kind Memcached

3. Build and push the Operator image.

   Use the default Makefile targets to build and push your Operator. Set IMG with a pull spec for your image that uses a registry you can push to:

   $ make docker-build docker-push IMG=<registry>/<user>/<image_name>:<tag>

4. Run the Operator.

   1. Install the CRD:

      $ make install

   2. Deploy the project to the cluster. Set IMG to the image that you pushed:

      $ make deploy IMG=<registry>/<user>/<image_name>:<tag>

5. Create a sample custom resource (CR).

   1. Create a sample CR:

      $ oc apply -f config/samples/cache_v1_memcached.yaml \
          -n memcached-operator-system

   2. Watch for the CR to reconcile the Operator:

      $ oc logs deployment.apps/memcached-operator-controller-manager \
          -c manager \
          -n memcached-operator-system

6. Delete a CR

   Delete a CR by running the following command:

   $ oc delete -f config/samples/cache_v1_memcached -n memcached-operator-system

7. Clean up.

   Run the following command to clean up the resources that have been created as part of this procedure:

   $ make undeploy

Procedure

1. Create a directory for the project:

   $ mkdir -p $HOME/projects/memcached-operator

2. Change to the directory:

   $ cd $HOME/projects/memcached-operator

3. Run the operator-sdk init command with the quarkus plugin to initialize the project:

   $ operator-sdk init \
       --plugins=quarkus \
       --domain=example.com \
       --project-name=memcached-operator

Procedure

1. Run the following command to create an API:

   $ operator-sdk create api \
       --plugins=quarkus \ 1
       --group=cache \ 2
       --version=v1 \ 3
       --kind=Memcached 4

   1

   Set the plugin flag to quarkus.

   2

   Set the group flag to cache.

   3

   Set the version flag to v1.

   4

   Set the kind flag to Memcached.

Verification

1. Run the tree command to view the file structure:

   $ tree

   Example output

   .
   ├── Makefile
   ├── PROJECT
   ├── pom.xml
   └── src
       └── main
           ├── java
           │   └── com
           │       └── example
           │           ├── Memcached.java
           │           ├── MemcachedReconciler.java
           │           ├── MemcachedSpec.java
           │           └── MemcachedStatus.java
           └── resources
               └── application.properties
   
   6 directories, 8 files

Procedure

1. Append the following dependency to the pom.xml file:

       <dependency>
         <groupId>commons-collections</groupId>
         <artifactId>commons-collections</artifactId>
         <version>3.2.2</version>
       </dependency>

2. For this example, replace the generated controller file MemcachedReconciler.java with following example implementation:

   Example 5.9. Example MemcachedReconciler.java

   package com.example;
   
   import io.fabric8.kubernetes.client.KubernetesClient;
   import io.javaoperatorsdk.operator.api.reconciler.Context;
   import io.javaoperatorsdk.operator.api.reconciler.Reconciler;
   import io.javaoperatorsdk.operator.api.reconciler.UpdateControl;
   import io.fabric8.kubernetes.api.model.ContainerBuilder;
   import io.fabric8.kubernetes.api.model.ContainerPortBuilder;
   import io.fabric8.kubernetes.api.model.LabelSelectorBuilder;
   import io.fabric8.kubernetes.api.model.ObjectMetaBuilder;
   import io.fabric8.kubernetes.api.model.OwnerReferenceBuilder;
   import io.fabric8.kubernetes.api.model.Pod;
   import io.fabric8.kubernetes.api.model.PodSpecBuilder;
   import io.fabric8.kubernetes.api.model.PodTemplateSpecBuilder;
   import io.fabric8.kubernetes.api.model.apps.Deployment;
   import io.fabric8.kubernetes.api.model.apps.DeploymentBuilder;
   import io.fabric8.kubernetes.api.model.apps.DeploymentSpecBuilder;
   import org.apache.commons.collections.CollectionUtils;
   import java.util.HashMap;
   import java.util.List;
   import java.util.Map;
   import java.util.stream.Collectors;
   
   public class MemcachedReconciler implements Reconciler<Memcached> {
     private final KubernetesClient client;
   
     public MemcachedReconciler(KubernetesClient client) {
       this.client = client;
     }
   
     // TODO Fill in the rest of the reconciler
   
     @Override
     public UpdateControl<Memcached> reconcile(
         Memcached resource, Context context) {
         // TODO: fill in logic
         Deployment deployment = client.apps()
                 .deployments()
                 .inNamespace(resource.getMetadata().getNamespace())
                 .withName(resource.getMetadata().getName())
                 .get();
   
         if (deployment == null) {
             Deployment newDeployment = createMemcachedDeployment(resource);
             client.apps().deployments().create(newDeployment);
             return UpdateControl.noUpdate();
         }
   
         int currentReplicas = deployment.getSpec().getReplicas();
         int requiredReplicas = resource.getSpec().getSize();
   
         if (currentReplicas != requiredReplicas) {
             deployment.getSpec().setReplicas(requiredReplicas);
             client.apps().deployments().createOrReplace(deployment);
             return UpdateControl.noUpdate();
         }
   
         List<Pod> pods = client.pods()
             .inNamespace(resource.getMetadata().getNamespace())
             .withLabels(labelsForMemcached(resource))
             .list()
             .getItems();
   
         List<String> podNames =
             pods.stream().map(p -> p.getMetadata().getName()).collect(Collectors.toList());
   
   
         if (resource.getStatus() == null
                  || !CollectionUtils.isEqualCollection(podNames, resource.getStatus().getNodes())) {
              if (resource.getStatus() == null) resource.setStatus(new MemcachedStatus());
              resource.getStatus().setNodes(podNames);
              return UpdateControl.updateResource(resource);
         }
   
         return UpdateControl.noUpdate();
     }
   
     private Map<String, String> labelsForMemcached(Memcached m) {
       Map<String, String> labels = new HashMap<>();
       labels.put("app", "memcached");
       labels.put("memcached_cr", m.getMetadata().getName());
       return labels;
     }
   
     private Deployment createMemcachedDeployment(Memcached m) {
         Deployment deployment = new DeploymentBuilder()
             .withMetadata(
                 new ObjectMetaBuilder()
                     .withName(m.getMetadata().getName())
                     .withNamespace(m.getMetadata().getNamespace())
                     .build())
             .withSpec(
                 new DeploymentSpecBuilder()
                     .withReplicas(m.getSpec().getSize())
                     .withSelector(
                         new LabelSelectorBuilder().withMatchLabels(labelsForMemcached(m)).build())
                     .withTemplate(
                         new PodTemplateSpecBuilder()
                             .withMetadata(
                                 new ObjectMetaBuilder().withLabels(labelsForMemcached(m)).build())
                             .withSpec(
                                 new PodSpecBuilder()
                                     .withContainers(
                                         new ContainerBuilder()
                                             .withImage("memcached:1.4.36-alpine")
                                             .withName("memcached")
                                             .withCommand("memcached", "-m=64", "-o", "modern", "-v")
                                             .withPorts(
                                                 new ContainerPortBuilder()
                                                     .withContainerPort(11211)
                                                     .withName("memcached")
                                                     .build())
                                             .build())
                                     .build())
                             .build())
                     .build())
             .build();
       deployment.addOwnerReference(m);
       return deployment;
     }
   }

   The example controller runs the following reconciliation logic for each Memcached custom resource (CR):

   • Creates a Memcached deployment if it does not exist.
   • Ensures that the deployment size matches the size specified by the Memcached CR spec.
   • Updates the Memcached CR status with the names of the memcached pods.

Procedure

1. Before marking one of your CRDs as internal, ensure that any debugging information or configuration that might be required to manage the application is reflected on the status or spec block of your CR, if applicable to your Operator.

2. Add the operators.operatorframework.io/internal-objects annotation to the CSV of your Operator to specify any internal objects to hide in the user interface:

   Internal object annotation

   apiVersion: operators.coreos.com/v1alpha1
   kind: ClusterServiceVersion
   metadata:
     name: my-operator-v1.2.3
     annotations:
       operators.operatorframework.io/internal-objects: '["my.internal.crd1.io","my.internal.crd2.io"]' 1
   ...

   1

   Set any internal CRDs as an array of strings.