Chapter 12. Operator SDK

Procedure

1. Create a new project.

   Use the CLI to create a new memcached-operator project:

   $ mkdir -p $GOPATH/src/github.com/example-inc/
   $ cd $GOPATH/src/github.com/example-inc/
   $ operator-sdk new memcached-operator
   $ cd memcached-operator

   Copy to Clipboard Toggle word wrap

2. Add a new Custom Resource Definition (CRD).

   1. Use the CLI to add a new CRD API called Memcached, with APIVersion set to cache.example.com/v1apha1 and Kind set to Memcached:

      $ operator-sdk add api \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      Copy to Clipboard Toggle word wrap

      This scaffolds the Memcached resource API under pkg/apis/cache/v1alpha1/.

   2. Modify the spec and status of the Memcached Custom Resource (CR) at the pkg/apis/cache/v1alpha1/memcached_types.go file:

      type MemcachedSpec struct {
      	// Size is the size of the memcached deployment
      	Size int32 `json:"size"`
      }
      type MemcachedStatus struct {
      	// Nodes are the names of the memcached pods
      	Nodes []string `json:"nodes"`
      }

      Copy to Clipboard Toggle word wrap

   3. After modifying the *_types.go file, always run the following command to update the generated code for that resource type:

      $ operator-sdk generate k8s

      Copy to Clipboard Toggle word wrap

3. Optional: Add custom validation to your CRD.

   OpenAPI v3.0 schemas are added to CRD manifests in the spec.validation block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached CR when it is created or updated.

   Additionally, a pkg/apis/<group>/<version>/zz_generated.openapi.go file is generated. This file contains the Go representation of this validation block if the +k8s:openapi-gen=true annotation is present above the Kind type declaration, which is present by default. This auto-generated code is your Go Kind type’s OpenAPI model, from which you can create a full OpenAPI Specification and generate a client.

   As an Operator author, you can use Kubebuilder markers (annotations) to configure custom validations for your API. These markers must always have a +kubebuilder:validation prefix. For example, adding an enum-type specification can be done by adding the following marker:

   // +kubebuilder:validation:Enum=Lion;Wolf;Dragon
   type Alias string

   Copy to Clipboard Toggle word wrap

   Usage of markers in API code is discussed in the Kubebuilder Generating CRDs and Markers for Config/Code Generation documentation. A full list of OpenAPIv3 validation markers is also available in the Kubebuilder CRD Validation documentation.

   If you add any custom validations, run the following command to update the OpenAPI validation section in the CRD’s deploy/crds/cache.example.com_memcacheds_crd.yaml file:

   $ operator-sdk generate crds

   Copy to Clipboard Toggle word wrap

   Example generated YAML

   spec:
     validation:
       openAPIV3Schema:
         properties:
           spec:
             properties:
               size:
                 format: int32
                 type: integer

   Copy to Clipboard Toggle word wrap

4. Add a new Controller.

   1. Add a new Controller to the project to watch and reconcile the Memcached resource:

      $ operator-sdk add controller \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      Copy to Clipboard Toggle word wrap

      This scaffolds a new Controller implementation under pkg/controller/memcached/.

   2. For this example, replace the generated controller file pkg/controller/memcached/memcached_controller.go with the example implementation.

      The example controller executes the following reconciliation logic for each Memcached CR:

      • Create a Memcached Deployment if it does not exist.
      • Ensure that the Deployment size is the same as specified by the Memcached CR spec.
      • Update the Memcached CR status with the names of the Memcached pods.

      The next two sub-steps inspect how the Controller watches resources and how the reconcile loop is triggered. You can skip these steps to go directly to building and running the Operator.

   3. Inspect the Controller implementation at the pkg/controller/memcached/memcached_controller.go file to see how the Controller watches resources.

      The first watch is for the Memcached type as the primary resource. For each Add, Update, or Delete event, the reconcile loop is sent a reconcile Request (a <namespace>:<name> key) for that Memcached object:

      err := c.Watch(
        &source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})

      Copy to Clipboard Toggle word wrap

      The next watch is for Deployments, but the event handler maps each event to a reconcile Request for the owner of the Deployment. In this case, this is the Memcached object for which the Deployment was created. This allows the controller to watch Deployments as a secondary resource:

      err := c.Watch(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{
      		IsController: true,
      		OwnerType:    &cachev1alpha1.Memcached{},
      	})

      Copy to Clipboard Toggle word wrap

   4. Every Controller has a Reconciler object with a Reconcile() method that implements the reconcile loop. The reconcile loop is passed the Request argument which is a <namespace>:<name> key used to lookup the primary resource object, Memcached, from the cache:

      func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error) {
        // Lookup the Memcached instance for this reconcile request
        memcached := &cachev1alpha1.Memcached{}
        err := r.client.Get(context.TODO(), request.NamespacedName, memcached)
        ...
      }

      Copy to Clipboard Toggle word wrap

      Based on the return value of Reconcile() the reconcile Request may be requeued and the loop may be triggered again:

      // Reconcile successful - don't requeue
      return reconcile.Result{}, nil
      // Reconcile failed due to error - requeue
      return reconcile.Result{}, err
      // Requeue for any reason other than error
      return reconcile.Result{Requeue: true}, nil

      Copy to Clipboard Toggle word wrap

5. Build and run the Operator.

   1. Before running the Operator, the CRD must be registered with the Kubernetes API server:

      $ oc create \
          -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

      Copy to Clipboard Toggle word wrap

   2. After registering the CRD, there are two options for running the Operator:

      • As a Deployment inside a Kubernetes cluster
      • As Go program outside a cluster

      Choose one of the following methods.

      1. Option A: Running as a Deployment inside the cluster.

         1. Build the memcached-operator image and push it to a registry:

            $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

            Copy to Clipboard Toggle word wrap

         2. The Deployment manifest is generated at deploy/operator.yaml. Update the Deployment image as follows since the default is just a placeholder:

            $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

            Copy to Clipboard Toggle word wrap
         3. Ensure you have an account on Quay.io for the next step, or substitute your preferred container registry. On the registry, create a new public image repository named memcached-operator.

         4. Push the image to the registry:

            $ podman push quay.io/example/memcached-operator:v0.0.1

            Copy to Clipboard Toggle word wrap

         5. Setup RBAC and deploy memcached-operator:

            $ oc create -f deploy/role.yaml
            $ oc create -f deploy/role_binding.yaml
            $ oc create -f deploy/service_account.yaml
            $ oc create -f deploy/operator.yaml

            Copy to Clipboard Toggle word wrap

         6. Verify that memcached-operator is up and running:

            $ oc get deployment
            NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
            memcached-operator       1         1         1            1           1m

            Copy to Clipboard Toggle word wrap

      2. Option B: Running locally outside the cluster.

         This method is preferred during development cycle to deploy and test faster.

         Run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local --namespace=default

         Copy to Clipboard Toggle word wrap

         You can use a specific kubeconfig using the flag --kubeconfig=<path/to/kubeconfig>.

6. Verify that the Operator can deploy a Memcached application by creating a Memcached CR.

   1. Create the example Memcached CR that was generated at deploy/crds/cache_v1alpha1_memcached_cr.yaml:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Copy to Clipboard Toggle word wrap

   2. Ensure that memcached-operator creates the Deployment for the CR:

      $ oc get deployment
      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

      Copy to Clipboard Toggle word wrap

   3. Check the pods and CR status to confirm the status is updated with the memcached pod names:

      $ oc get pods
      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m
      
      $ oc get memcached/example-memcached -o yaml
      apiVersion: cache.example.com/v1alpha1
      kind: Memcached
      metadata:
        clusterName: ""
        creationTimestamp: 2018-03-31T22:51:08Z
        generation: 0
        name: example-memcached
        namespace: default
        resourceVersion: "245453"
        selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached
        uid: 0026cc97-3536-11e8-bd83-0800274106a1
      spec:
        size: 3
      status:
        nodes:
        - example-memcached-6fd7c98d8-7dqdr
        - example-memcached-6fd7c98d8-g5k7v
        - example-memcached-6fd7c98d8-m7vn7

      Copy to Clipboard Toggle word wrap

7. Verify that the Operator can manage a deployed Memcached application by updating the size of the deployment.

   1. Change the spec.size field in the memcached CR from 3 to 4:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

      Copy to Clipboard Toggle word wrap

   2. Apply the change:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Copy to Clipboard Toggle word wrap

   3. Confirm that the Operator changes the Deployment size:

      $ oc get deployment
      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

      Copy to Clipboard Toggle word wrap

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml
   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/service_account.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. Generate an Operator manifest.

   An Operator manifest describes how to display, create, and manage the application, in this case Memcached, as a whole. It is defined by a ClusterServiceVersion (CSV) object and is required for the OLM to function.

   From the memcached-operator/ directory that was created when you built the Memcached Operator, generate the CSV manifest:

   $ operator-sdk generate csv --csv-version 0.0.1

   Copy to Clipboard Toggle word wrap
   Note

   See Building a CSV for the Operator Framework for more information on manually defining a manifest file.

2. Create an OperatorGroup that specifies the namespaces that the Operator will target. Create the following OperatorGroup in the namespace where you will create the CSV. In this example, the default namespace is used:

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: memcached-operator-group
     namespace: default
   spec:
     targetNamespaces:
     - default

   Copy to Clipboard Toggle word wrap

3. Deploy the Operator. Use the files that were generated into the deploy/ directory by the Operator SDK when you built the Memcached Operator.

   1. Apply the Operator’s CSV manifest to the specified namespace in the cluster:

      $ oc apply -f deploy/olm-catalog/memcached-operator/0.0.1/memcached-operator.v0.0.1.clusterserviceversion.yaml

      Copy to Clipboard Toggle word wrap

      When you apply this manifest, the cluster does not immediately update because it does not yet meet the requirements specified in the manifest.

   2. Create the role, role binding, and service account to grant resource permissions to the Operator, and the Custom Resource Definition (CRD) to create the Memcached type that the Operator manages:

      $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml
      $ oc create -f deploy/service_account.yaml
      $ oc create -f deploy/role.yaml
      $ oc create -f deploy/role_binding.yaml

      Copy to Clipboard Toggle word wrap

      Because the OLM creates Operators in a particular namespace when a manifest is applied, administrators can leverage the native Kubernetes RBAC permission model to restrict which users are allowed to install Operators.

4. Create an application instance.

   The Memcached Operator is now running in the default namespace. Users interact with Operators via instances of CustomResources; in this case, the resource has the kind Memcached. Native Kubernetes RBAC also applies to CustomResources, providing administrators control over who can interact with each Operator.

   Creating instances of Memcached in this namespace will now trigger the Memcached Operator to instantiate pods running the memcached server that are managed by the Operator. The more CustomResources you create, the more unique instances of Memcached are managed by the Memcached Operator running in this namespace.

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-wordpress"
   spec:
     size: 1
   EOF
   
   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-drupal"
   spec:
     size: 1
   EOF
   
   $ oc get Memcached
   NAME                      AGE
   memcached-for-drupal      22s
   memcached-for-wordpress   27s
   
   $ oc get pods
   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          14m
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          3s
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          8s

   Copy to Clipboard Toggle word wrap

5. Update an application.

   Manually apply an update to the Operator by creating a new Operator manifest with a replaces field that references the old Operator manifest. The OLM ensures that all resources being managed by the old Operator have their ownership moved to the new Operator without fear of any programs stopping execution. It is up to the Operators themselves to execute any data migrations required to upgrade resources to run under a new version of the Operator.

   The following command demonstrates applying a new Operator manifest file using a new version of the Operator and shows that the pods remain executing:

   $ curl -Lo memcachedoperator.0.0.2.csv.yaml https://raw.githubusercontent.com/operator-framework/getting-started/master/memcachedoperator.0.0.2.csv.yaml
   $ oc apply -f memcachedoperator.0.0.2.csv.yaml
   $ oc get pods
   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          3s
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          14m
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          14m

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Ansible-based, namespace-scoped memcached-operator project and change to its directory, use the following commands:

   $ operator-sdk new memcached-operator \
       --api-version=cache.example.com/v1alpha1 \
       --kind=Memcached \
       --type=ansible
   $ cd memcached-operator

   Copy to Clipboard Toggle word wrap

   This creates the memcached-operator project specifically for watching the Memcached resource with APIVersion example.com/v1apha1 and Kind Memcached.

2. Customize the Operator logic.

   For this example, the memcached-operator executes the following reconciliation logic for each Memcached Custom Resource (CR):

   • Create a memcached Deployment if it does not exist.
   • Ensure that the Deployment size is the same as specified by the Memcached CR.

   By default, the memcached-operator watches Memcached resource events as shown in the watches.yaml file and executes the Ansible role Memcached:

   - version: v1alpha1
     group: cache.example.com
     kind: Memcached

   Copy to Clipboard Toggle word wrap

   You can optionally customize the following logic in the watches.yaml file:

   1. Specifying a role option configures the Operator to use this specified path when launching ansible-runner with an Ansible role. By default, the new command fills in an absolute path to where your role should go:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        role: /opt/ansible/roles/memcached

      Copy to Clipboard Toggle word wrap

   2. Specifying a playbook option in the watches.yaml file configures the Operator to use this specified path when launching ansible-runner with an Ansible playbook:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        playbook: /opt/ansible/playbook.yaml

      Copy to Clipboard Toggle word wrap

3. Build the Memcached Ansible role.

   Modify the generated Ansible role under the roles/memcached/ directory. This Ansible role controls the logic that is executed when a resource is modified.

   1. Define the Memcached spec.

      Defining the spec for an Ansible-based Operator can be done entirely in Ansible. The Ansible Operator passes all key-value pairs listed in the CR spec field along to Ansible as variables. The names of all variables in the spec field are converted to snake case (lowercase with an underscore) by the Operator before running Ansible. For example, serviceAccount in the spec becomes service_account in Ansible.

      Tip

      You should perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

      In case the user does not set the spec field, set a default by modifying the roles/memcached/defaults/main.yml file:

      size: 1

      Copy to Clipboard Toggle word wrap

   2. Define the Memcached Deployment.

      With the Memcached spec now defined, you can define what Ansible is actually executed on resource changes. Because this is an Ansible role, the default behavior executes the tasks in the roles/memcached/tasks/main.yml file.

      The goal is for Ansible to create a Deployment if it does not exist, which runs the memcached:1.4.36-alpine image. Ansible 2.7+ supports the k8s Ansible module, which this example leverages to control the Deployment definition.

      Modify the roles/memcached/tasks/main.yml to match the following:

      - name: start memcached
        k8s:
          definition:
            kind: Deployment
            apiVersion: apps/v1
            metadata:
              name: '{{ meta.name }}-memcached'
              namespace: '{{ meta.namespace }}'
            spec:
              replicas: "{{size}}"
              selector:
                matchLabels:
                  app: memcached
              template:
                metadata:
                  labels:
                    app: memcached
                spec:
                  containers:
                  - name: memcached
                    command:
                    - memcached
                    - -m=64
                    - -o
                    - modern
                    - -v
                    image: "docker.io/memcached:1.4.36-alpine"
                    ports:
                      - containerPort: 11211

      Copy to Clipboard Toggle word wrap
      Note

      This example used the size variable to control the number of replicas of the Memcached Deployment. This example sets the default to 1, but any user can create a CR that overwrites the default.

4. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new Custom Resource Definition (CRD) the Operator will be watching. Deploy the Memcached CRD:

   $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

   Copy to Clipboard Toggle word wrap

5. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a Pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a Pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the memcached-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/memcached-operator:v0.0.1
         $ podman push quay.io/example/memcached-operator:v0.0.1

         Copy to Clipboard Toggle word wrap

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      3. Deploy the memcached-operator:

         $ oc create -f deploy/service_account.yaml
         $ oc create -f deploy/role.yaml
         $ oc create -f deploy/role_binding.yaml
         $ oc create -f deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      4. Verify that the memcached-operator is up and running:

         $ oc get deployment
         NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         memcached-operator       1         1         1            1           1m

         Copy to Clipboard Toggle word wrap

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      Ensure that Ansible Runner and Ansible Runner HTTP Plug-in are installed or else you will see unexpected errors from Ansible Runner when a CR is created.

      It is also important that the role path referenced in the watches.yaml file exists on your machine. Because normally a container is used where the role is put on disk, the role must be manually copied to the configured Ansible roles path (for example /etc/ansible/roles).

      1. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         Copy to Clipboard Toggle word wrap

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=config

         Copy to Clipboard Toggle word wrap

6. Create a Memcached CR.

   1. Modify the deploy/crds/cache_v1alpha1_memcached_cr.yaml file as shown and create a Memcached CR:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Copy to Clipboard Toggle word wrap

   2. Ensure that the memcached-operator creates the Deployment for the CR:

      $ oc get deployment
      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

      Copy to Clipboard Toggle word wrap

   3. Check the pods to confirm three replicas were created:

      $ oc get pods
      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

      Copy to Clipboard Toggle word wrap

7. Update the size.

   1. Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Copy to Clipboard Toggle word wrap

   2. Confirm that the Operator changes the Deployment size:

      $ oc get deployment
      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

      Copy to Clipboard Toggle word wrap

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/service_account.yaml
   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. To track CR status manually from your application, update the Watches file with a manageStatus field set to false:

   - version: v1
     group: api.example.com
     kind: Foo
     role: Foo
     manageStatus: false

   Copy to Clipboard Toggle word wrap

2. Then, use the operator_sdk.util.k8s_status Ansible module to update the subresource. For example, to update with key foo and value bar, operator_sdk.util can be used as shown:

   - operator_sdk.util.k8s_status:
       api_version: app.example.com/v1
       kind: Foo
       name: "{{ meta.name }}"
       namespace: "{{ meta.namespace }}"
       status:
         foo: bar

   Copy to Clipboard Toggle word wrap

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Helm-based, namespace-scoped nginx-operator project, use the following command:

   $ operator-sdk new nginx-operator \
     --api-version=example.com/v1alpha1 \
     --kind=Nginx \
     --type=helm
   $ cd nginx-operator

   Copy to Clipboard Toggle word wrap

   This creates the nginx-operator project specifically for watching the Nginx resource with APIVersion example.com/v1apha1 and Kind Nginx.

2. Customize the Operator logic.

   For this example, the nginx-operator executes the following reconciliation logic for each Nginx Custom Resource (CR):

   • Create a Nginx Deployment if it does not exist.
   • Create a Nginx Service if it does not exist.
   • Create a Nginx Ingress if it is enabled and does not exist.
   • Ensure that the Deployment, Service, and optional Ingress match the desired configuration (for example, replica count, image, service type) as specified by the Nginx CR.

   By default, the nginx-operator watches Nginx resource events as shown in the watches.yaml file and executes Helm releases using the specified chart:

   - version: v1alpha1
     group: example.com
     kind: Nginx
     chart: /opt/helm/helm-charts/nginx

   Copy to Clipboard Toggle word wrap

   1. Review the Nginx Helm chart.

      When a Helm Operator project is created, the Operator SDK creates an example Helm chart that contains a set of templates for a simple Nginx release.

      For this example, templates are available for Deployment, Service, and Ingress resources, along with a NOTES.txt template, which Helm chart developers use to convey helpful information about a release.

      If you are not already familiar with Helm Charts, take a moment to review the Helm Chart developer documentation.

   2. Understand the Nginx CR spec.

      Helm uses a concept called values to provide customizations to a Helm chart’s defaults, which are defined in the Helm chart’s values.yaml file.

      Override these defaults by setting the desired values in the CR spec. You can use the number of replicas as an example:

      1. First, inspect the helm-charts/nginx/values.yaml file to find that the chart has a value called replicaCount and it is set to 1 by default. To have 2 Nginx instances in your deployment, your CR spec must contain replicaCount: 2.

         Update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file to look like the following:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2

         Copy to Clipboard Toggle word wrap

      2. Similarly, the default service port is set to 80. To instead use 8080, update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file again by adding the service port override:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2
           service:
             port: 8080

         Copy to Clipboard Toggle word wrap

         The Helm Operator applies the entire spec as if it was the contents of a values file, just like the helm install -f ./overrides.yaml command works.

3. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new custom resource definition (CRD) the operator will be watching. Deploy the following CRD:

   $ oc create -f deploy/crds/example_v1alpha1_nginx_crd.yaml

   Copy to Clipboard Toggle word wrap

4. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a Pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a Pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the nginx-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/nginx-operator:v0.0.1
         $ podman push quay.io/example/nginx-operator:v0.0.1

         Copy to Clipboard Toggle word wrap

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      3. Deploy the nginx-operator:

         $ oc create -f deploy/service_account.yaml
         $ oc create -f deploy/role.yaml
         $ oc create -f deploy/role_binding.yaml
         $ oc create -f deploy/operator.yaml

         Copy to Clipboard Toggle word wrap

      4. Verify that the nginx-operator is up and running:

         $ oc get deployment
         NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         nginx-operator       1         1         1            1           1m

         Copy to Clipboard Toggle word wrap

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      It is important that the chart path referenced in the watches.yaml file exists on your machine. By default, the watches.yaml file is scaffolded to work with an Operator image built with the operator-sdk build command. When developing and testing your operator with the operator-sdk run --local command, the SDK looks in your local file system for this path.

      1. Create a symlink at this location to point to your Helm chart’s path:

         $ sudo mkdir -p /opt/helm/helm-charts
         $ sudo ln -s $PWD/helm-charts/nginx /opt/helm/helm-charts/nginx

         Copy to Clipboard Toggle word wrap

      2. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         Copy to Clipboard Toggle word wrap

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=<path_to_config>

         Copy to Clipboard Toggle word wrap

5. Deploy the Nginx CR.

   Apply the Nginx CR that you modified earlier:

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   Ensure that the nginx-operator creates the Deployment for the CR:

   $ oc get deployment
   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        2         2         2            2           1m

   Copy to Clipboard Toggle word wrap

   Check the pods to confirm two replicas were created:

   $ oc get pods
   NAME                                                      READY     STATUS    RESTARTS   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-fjcr9   1/1       Running   0          1m
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-ljbzl   1/1       Running   0          1m

   Copy to Clipboard Toggle word wrap

   Check that the Service port is set to 8080:

   $ oc get service
   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        8080/TCP   1m

   Copy to Clipboard Toggle word wrap

6. Update the replicaCount and remove the port.

   Change the spec.replicaCount field from 2 to 3, remove the spec.service field, and apply the change:

   $ cat deploy/crds/example.com_v1alpha1_nginx_cr.yaml
   apiVersion: "example.com/v1alpha1"
   kind: "Nginx"
   metadata:
     name: "example-nginx"
   spec:
     replicaCount: 3
   
   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Copy to Clipboard Toggle word wrap

   Confirm that the Operator changes the Deployment size:

   $ oc get deployment
   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        3         3         3            3           1m

   Copy to Clipboard Toggle word wrap

   Check that the Service port is set to the default 80:

   $ oc get service
   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)  AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        80/TCP   1m

   Copy to Clipboard Toggle word wrap

7. Clean up the resources:

   $ oc delete -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/service_account.yaml
   $ oc delete -f deploy/crds/example_v1alpha1_nginx_crd.yaml

   Copy to Clipboard Toggle word wrap

Procedure

1. In your Operator project, configure your CSV composition by modifying the deploy/olm-catalog/csv-config.yaml file, if desired.

2. Generate the CSV:

   $ operator-sdk generate csv --csv-version <version>

   Copy to Clipboard Toggle word wrap
3. In the new CSV generated in the deploy/olm-catalog/ directory, ensure all required, manually-defined fields are set appropriately.