4.3. Creating Helm-based Operators

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Helm-based, namespace-scoped nginx-operator project, use the following command:

   $ operator-sdk new nginx-operator \
     --api-version=example.com/v1alpha1 \
     --kind=Nginx \
     --type=helm

   Copy to Clipboard Toggle word wrap

   $ cd nginx-operator

   Copy to Clipboard Toggle word wrap

   This creates the nginx-operator project specifically for watching the Nginx resource with API version example.com/v1apha1 and kind Nginx.

2. Customize the Operator logic.

   For this example, the nginx-operator executes the following reconciliation logic for each Nginx custom resource (CR):

   • Create an Nginx deployment if it does not exist.
   • Create an Nginx service if it does not exist.
   • Create an Nginx ingress if it is enabled and does not exist.
   • Ensure that the deployment, service, and optional ingress match the desired configuration (for example, replica count, image, service type) as specified by the Nginx CR.

   By default, the nginx-operator watches Nginx resource events as shown in the watches.yaml file and executes Helm releases using the specified chart:

   - version: v1alpha1
     group: example.com
     kind: Nginx
     chart: /opt/helm/helm-charts/nginx

   Copy to Clipboard Toggle word wrap

   1. Review the Nginx Helm chart.

      When a Helm Operator project is created, the Operator SDK creates an example Helm chart that contains a set of templates for a simple Nginx release.

      For this example, templates are available for deployment, service, and ingress resources, along with a NOTES.txt template, which Helm chart developers use to convey helpful information about a release.

      If you are not already familiar with Helm Charts, review the Helm Chart developer documentation.

   2. Understand the Nginx CR spec.

      Helm uses a concept called values to provide customizations to the defaults of a Helm chart, which are defined in the values.yaml file.

      Override these defaults by setting the desired values in the CR spec. You can use the number of replicas as an example:

      1. First, inspect the helm-charts/nginx/values.yaml file to find that the chart has a value called replicaCount and it is set to 1 by default. To have 2 Nginx instances in your deployment, your CR spec must contain replicaCount: 2.

         Update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file to look like the following:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2

         Copy to Clipboard Toggle word wrap

      2. Similarly, the default service port is set to 80. To instead use 8080, update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file again by adding the service port override:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2
           service:
             port: 8080

         Copy to Clipboard Toggle word wrap

         The Helm Operator applies the entire spec as if it was the contents of a values file, just like the helm install -f ./overrides.yaml command works.

3. Deploy the CRD.

   Before running the Operator, Kubernetes must know about the new custom resource definition (CRD) that the Operator will be watching. Deploy the following CRD:

   $ oc create -f deploy/crds/example_v1alpha1_nginx_crd.yaml

   Copy to Clipboard Toggle word wrap

4. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the nginx-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/nginx-operator:v0.0.1

         Copy to Clipboard Toggle word wrap

         $ podman push quay.io/example/nginx-operator:v0.0.1

         Copy to Clipboard Toggle word wrap

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      3. Deploy the nginx-operator manifests:

         $ oc create -f deploy/service_account.yaml

         Copy to Clipboard Toggle word wrap

         $ oc create -f deploy/role.yaml

         Copy to Clipboard Toggle word wrap

         $ oc create -f deploy/role_binding.yaml

         Copy to Clipboard Toggle word wrap

         $ oc create -f deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      4. Verify that the nginx-operator deployment is up and running:

         $ oc get deployment

         Copy to Clipboard Toggle word wrap

         Example output

         NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         nginx-operator       1         1         1            1           1m

         Copy to Clipboard Toggle word wrap

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      It is important that the chart path referenced in the watches.yaml file exists on your machine. By default, the watches.yaml file is scaffolded to work with an Operator image built with the operator-sdk build command. When developing and testing your Operator with the operator-sdk run --local command, the SDK looks in your local file system for this path.

      1. Create a symlink at this location to point to the path of your Helm chart:

         $ sudo mkdir -p /opt/helm/helm-charts

         Copy to Clipboard Toggle word wrap

         $ sudo ln -s $PWD/helm-charts/nginx /opt/helm/helm-charts/nginx

         Copy to Clipboard Toggle word wrap

      2. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         Copy to Clipboard Toggle word wrap

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=<path_to_config>

         Copy to Clipboard Toggle word wrap

5. Deploy the Nginx CR.

   Apply the Nginx CR that you modified earlier:

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   Ensure that the nginx-operator creates the deployment for the CR:

   $ oc get deployment

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        2         2         2            2           1m

   Copy to Clipboard Toggle word wrap

   Check the pods to confirm two replicas were created:

   $ oc get pods

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                                      READY     STATUS    RESTARTS   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-fjcr9   1/1       Running   0          1m
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-ljbzl   1/1       Running   0          1m

   Copy to Clipboard Toggle word wrap

   Check that the service port is set to 8080:

   $ oc get service

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        8080/TCP   1m

   Copy to Clipboard Toggle word wrap

6. Update the replicaCount and remove the port.

   Change the spec.replicaCount field from 2 to 3, remove the spec.service field, and apply the change:

   $ cat deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   Example output

   apiVersion: "example.com/v1alpha1"
   kind: "Nginx"
   metadata:
     name: "example-nginx"
   spec:
     replicaCount: 3

   Copy to Clipboard Toggle word wrap

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   Confirm that the Operator changes the deployment size:

   $ oc get deployment

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        3         3         3            3           1m

   Copy to Clipboard Toggle word wrap

   Check that the service port is set to the default 80:

   $ oc get service

   Copy to Clipboard Toggle word wrap

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)  AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        80/TCP   1m

   Copy to Clipboard Toggle word wrap

7. Clean up the resources:

   $ oc delete -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   $ oc delete -f deploy/operator.yaml

   Copy to Clipboard Toggle word wrap

   $ oc delete -f deploy/role_binding.yaml

   Copy to Clipboard Toggle word wrap

   $ oc delete -f deploy/role.yaml

   Copy to Clipboard Toggle word wrap

   $ oc delete -f deploy/service_account.yaml

   Copy to Clipboard Toggle word wrap

   $ oc delete -f deploy/crds/example_v1alpha1_nginx_crd.yaml

   Copy to Clipboard Toggle word wrap