Chapter 4. Developing Operators

Procedure

1. Create a new project.

   Use the CLI to create a new memcached-operator project:

   $ mkdir -p $GOPATH/src/github.com/example-inc/

   $ cd $GOPATH/src/github.com/example-inc/

   $ operator-sdk new memcached-operator

   $ cd memcached-operator

2. Add a new custom resource definition (CRD).

   1. Use the CLI to add a new CRD API called Memcached, with APIVersion set to cache.example.com/v1apha1 and Kind set to Memcached:

      $ operator-sdk add api \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds the Memcached resource API under pkg/apis/cache/v1alpha1/.

   2. Modify the spec and status of the Memcached custom resource (CR) at the pkg/apis/cache/v1alpha1/memcached_types.go file:

      type MemcachedSpec struct {
      	// Size is the size of the memcached deployment
      	Size int32 `json:"size"`
      }
      type MemcachedStatus struct {
      	// Nodes are the names of the memcached pods
      	Nodes []string `json:"nodes"`
      }

   3. After modifying the *_types.go file, always run the following command to update the generated code for that resource type:

      $ operator-sdk generate k8s

3. Optional: Add custom validation to your CRD.

   OpenAPI v3.0 schemas are added to CRD manifests in the spec.validation block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached CR when it is created or updated.

   Additionally, a pkg/apis/<group>/<version>/zz_generated.openapi.go file is generated. This file contains the Go representation of this validation block if the +k8s:openapi-gen=true annotation is present above the Kind type declaration, which is present by default. This auto-generated code is the OpenAPI model of your Go Kind type, from which you can create a full OpenAPI Specification and generate a client.

   As an Operator author, you can use Kubebuilder markers (annotations) to configure custom validations for your API. These markers must always have a +kubebuilder:validation prefix. For example, adding an enum-type specification can be done by adding the following marker:

   // +kubebuilder:validation:Enum=Lion;Wolf;Dragon
   type Alias string

   Usage of markers in API code is discussed in the Kubebuilder Generating CRDs and Markers for Config/Code Generation documentation. A full list of OpenAPIv3 validation markers is also available in the Kubebuilder CRD Validation documentation.

   If you add any custom validations, run the following command to update the OpenAPI validation section in the deploy/crds/cache.example.com_memcacheds_crd.yaml file for the CRD:

   $ operator-sdk generate crds

   Example generated YAML

   spec:
     validation:
       openAPIV3Schema:
         properties:
           spec:
             properties:
               size:
                 format: int32
                 type: integer

4. Add a new controller.

   1. Add a new controller to the project to watch and reconcile the Memcached resource:

      $ operator-sdk add controller \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds a new controller implementation under pkg/controller/memcached/.

   2. For this example, replace the generated controller file pkg/controller/memcached/memcached_controller.go with the example implementation.

      The example controller executes the following reconciliation logic for each Memcached resource:

      • Create a Memcached deployment if it does not exist.
      • Ensure that the Deployment size is the same as specified by the Memcached CR spec.
      • Update the Memcached resource status with the names of the Memcached pods.

      The next two sub-steps inspect how the controller watches resources and how the reconcile loop is triggered. You can skip these steps to go directly to building and running the Operator.

   3. Inspect the controller implementation at the pkg/controller/memcached/memcached_controller.go file to see how the controller watches resources.

      The first watch is for the Memcached type as the primary resource. For each add, update, or delete event, the reconcile loop is sent a reconcile Request (a <namespace>:<name> key) for that Memcached object:

      err := c.Watch(
        &source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})

      The next watch is for Deployment objects, but the event handler maps each event to a reconcile Request for the owner of the deployment. In this case, this is the Memcached object for which the deployment was created. This allows the controller to watch deployments as a secondary resource:

      err := c.Watch(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{
      		IsController: true,
      		OwnerType:    &cachev1alpha1.Memcached{},
      	})

   4. Every controller has a Reconciler object with a Reconcile() method that implements the reconcile loop. The reconcile loop is passed the Request argument which is a <namespace>:<name> key used to lookup the primary resource object, Memcached, from the cache:

      func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error) {
        // Lookup the Memcached instance for this reconcile request
        memcached := &cachev1alpha1.Memcached{}
        err := r.client.Get(context.TODO(), request.NamespacedName, memcached)
        ...
      }

      Based on the return value of the Reconcile() function, the reconcile Request might be requeued, and the loop might be triggered again:

      // Reconcile successful - don't requeue
      return reconcile.Result{}, nil
      // Reconcile failed due to error - requeue
      return reconcile.Result{}, err
      // Requeue for any reason other than error
      return reconcile.Result{Requeue: true}, nil

5. Build and run the Operator.

   1. Before running the Operator, the CRD must be registered with the Kubernetes API server:

      $ oc create \
          -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   2. After registering the CRD, there are two options for running the Operator:

      • As a Deployment inside a Kubernetes cluster
      • As Go program outside a cluster

      Choose one of the following methods.

      1. Option A: Running as a deployment inside the cluster.

         1. Build the memcached-operator image and push it to a registry:

            $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

         2. The deployment manifest is generated at deploy/operator.yaml. Update the deployment image as follows since the default is just a placeholder:

            $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

         3. Ensure you have an account on Quay.io for the next step, or substitute your preferred container registry. On the registry, create a new public image repository named memcached-operator.

         4. Push the image to the registry:

            $ podman push quay.io/example/memcached-operator:v0.0.1

         5. Set up RBAC and create the memcached-operator manifests:

            $ oc create -f deploy/role.yaml

            $ oc create -f deploy/role_binding.yaml

            $ oc create -f deploy/service_account.yaml

            $ oc create -f deploy/operator.yaml

         6. Verify that the memcached-operator deploy is up and running:

            $ oc get deployment

            Example output

            NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
            memcached-operator       1         1         1            1           1m

      2. Option B: Running locally outside the cluster.

         This method is preferred during development cycle to deploy and test faster.

         Run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local --namespace=default

         You can use a specific kubeconfig using the flag --kubeconfig=<path/to/kubeconfig>.

6. Verify that the Operator can deploy a Memcached application by creating a Memcached CR.

   1. Create the example Memcached CR that was generated at deploy/crds/cache_v1alpha1_memcached_cr.yaml.

   2. View the file:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3

   3. Create the object:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   4. Ensure that memcached-operator creates the deployment for the CR:

      $ oc get deployment

      Example output

      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   5. Check the pods and CR to confirm the CR status is updated with the pod names:

      $ oc get pods

      Example output

      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

      $ oc get memcached/example-memcached -o yaml

      Example output

      apiVersion: cache.example.com/v1alpha1
      kind: Memcached
      metadata:
        clusterName: ""
        creationTimestamp: 2018-03-31T22:51:08Z
        generation: 0
        name: example-memcached
        namespace: default
        resourceVersion: "245453"
        selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached
        uid: 0026cc97-3536-11e8-bd83-0800274106a1
      spec:
        size: 3
      status:
        nodes:
        - example-memcached-6fd7c98d8-7dqdr
        - example-memcached-6fd7c98d8-g5k7v
        - example-memcached-6fd7c98d8-m7vn7

7. Verify that the Operator can manage a deployed Memcached application by updating the size of the deployment.

   1. Change the spec.size field in the memcached CR from 3 to 4:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

   2. Apply the change:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   3. Confirm that the Operator changes the deployment size:

      $ oc get deployment

      Example output

      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/service_account.yaml

Procedure

1. Generate an Operator manifest.

   An Operator manifest describes how to display, create, and manage the application, in this case Memcached, as a whole. It is defined by a ClusterServiceVersion (CSV) object and is required for OLM to function.

   From the memcached-operator/ directory that was created when you built the Memcached Operator, generate the CSV manifest:

   $ operator-sdk generate csv --csv-version 0.0.1

   Note

   See Building a CSV for the Operator Framework for more information on manually defining a manifest file.

2. Create an Operator group that specifies the namespaces that the Operator will target. Create the following Operator group in the namespace where you will create the CSV. In this example, the default namespace is used:

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: memcached-operator-group
     namespace: default
   spec:
     targetNamespaces:
     - default

3. Deploy the Operator. Use the files that were generated into the deploy/ directory by the Operator SDK when you built the Memcached Operator.

   1. Edit the generated CSV manifest file by adding displayName fields for each custom resource definition (CRD) kind in the spec.customresourcedefinitions.owned section:

      deploy/olm-catalog/memcached-operator/0.0.1/memcached-operator.v0.0.1.clusterserviceversion.yaml file

      ...
      spec:
        customresourcedefinitions:
          owned:
          - kind: Memcached
            name: memcacheds.cache.example.com
            version: v1alpha1
            description: Memcached is the Schema for the memcacheds API
            displayName: Memcached 1
      ...

      1

      Specify a display name for the CRD.

   2. Apply the CSV manifest to the specified namespace in the cluster:

      $ oc apply -f deploy/olm-catalog/memcached-operator/0.0.1/memcached-operator.v0.0.1.clusterserviceversion.yaml

      When you apply this manifest, the cluster does not immediately update because it does not yet meet the requirements specified in the manifest.

   3. Create the role, role binding, and service account to grant resource permissions to the Operator, and the custom resource definition (CRD) to create the Memcached custom resource that the Operator manages:

      $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

      $ oc create -f deploy/service_account.yaml

      $ oc create -f deploy/role.yaml

      $ oc create -f deploy/role_binding.yaml

      Because OLM creates Operators in a particular namespace when a manifest is applied, administrators can leverage the native Kubernetes RBAC permission model to restrict which users are allowed to install Operators.

4. Create an application instance.

   The Memcached Operator is now running in the default namespace. Users interact with Operators via instances of custom resources; in this case, the resource has the kind Memcached. Native Kubernetes RBAC also applies to custom resources, providing administrators control over who can interact with each Operator.

   Creating instances of Memcached objects in this namespace will now trigger the Memcached Operator to instantiate pods running the memcached server that are managed by the Operator. The more custom resources you create, the more unique Memcached application instances are managed by the Memcached Operator running in this namespace.

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-wordpress"
   spec:
     size: 1
   EOF

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-drupal"
   spec:
     size: 1
   EOF

   $ oc get Memcached

   Example output

   NAME                      AGE
   memcached-for-drupal      22s
   memcached-for-wordpress   27s

   $ oc get pods

   Example output

   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          14m
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          3s
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          8s

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Ansible-based, namespace-scoped memcached-operator project and change to the new directory, use the following commands:

   $ operator-sdk new memcached-operator \
       --api-version=cache.example.com/v1alpha1 \
       --kind=Memcached \
       --type=ansible

   $ cd memcached-operator

   This creates the memcached-operator project specifically for watching the Memcached resource with API version example.com/v1apha1 and kind Memcached.

2. Customize the Operator logic.

   For this example, the memcached-operator executes the following reconciliation logic for each Memcached custom resource (CR):

   • Create a memcached deployment if it does not exist.
   • Ensure that the deployment size is the same as specified by the Memcached CR.

   By default, the memcached-operator watches Memcached resource events as shown in the watches.yaml file and executes the Ansible role Memcached:

   - version: v1alpha1
     group: cache.example.com
     kind: Memcached

   You can optionally customize the following logic in the watches.yaml file:

   1. Specifying a role option configures the Operator to use this specified path when launching ansible-runner with an Ansible role. By default, the operator-sdk new command fills in an absolute path to where your role should go:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        role: /opt/ansible/roles/memcached

   2. Specifying a playbook option in the watches.yaml file configures the Operator to use this specified path when launching ansible-runner with an Ansible playbook:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        playbook: /opt/ansible/playbook.yaml

3. Build the Memcached Ansible role.

   Modify the generated Ansible role under the roles/memcached/ directory. This Ansible role controls the logic that is executed when a resource is modified.

   1. Define the Memcached spec.

      Defining the spec for an Ansible-based Operator can be done entirely in Ansible. The Ansible Operator passes all key-value pairs listed in the CR spec field along to Ansible as variables. The names of all variables in the spec field are converted to snake case (lowercase with an underscore) by the Operator before running Ansible. For example, serviceAccount in the spec becomes service_account in Ansible.

      Tip

      You should perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

      In case the user does not set the spec field, set a default by modifying the roles/memcached/defaults/main.yml file:

      size: 1

   2. Define the Memcached deployment.

      With the Memcached spec now defined, you can define what Ansible is actually executed on resource changes. Because this is an Ansible role, the default behavior executes the tasks in the roles/memcached/tasks/main.yml file.

      The goal is for Ansible to create a deployment if it does not exist, which runs the memcached:1.4.36-alpine image. Ansible 2.7+ supports the k8s Ansible module, which this example leverages to control the deployment definition.

      Modify the roles/memcached/tasks/main.yml to match the following:

      - name: start memcached
        k8s:
          definition:
            kind: Deployment
            apiVersion: apps/v1
            metadata:
              name: '{{ meta.name }}-memcached'
              namespace: '{{ meta.namespace }}'
            spec:
              replicas: "{{size}}"
              selector:
                matchLabels:
                  app: memcached
              template:
                metadata:
                  labels:
                    app: memcached
                spec:
                  containers:
                  - name: memcached
                    command:
                    - memcached
                    - -m=64
                    - -o
                    - modern
                    - -v
                    image: "docker.io/memcached:1.4.36-alpine"
                    ports:
                      - containerPort: 11211

      Note

      This example used the size variable to control the number of replicas of the Memcached deployment. This example sets the default to 1, but any user can create a CR that overwrites the default.

4. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new custom resource definition (CRD) that the Operator will be watching. Deploy the Memcached CRD:

   $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

5. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the memcached-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

         $ podman push quay.io/example/memcached-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

      3. Deploy the memcached-operator manifests:

         $ oc create -f deploy/service_account.yaml

         $ oc create -f deploy/role.yaml

         $ oc create -f deploy/role_binding.yaml

         $ oc create -f deploy/operator.yaml

      4. Verify that the memcached-operator deployment is up and running:

         $ oc get deployment

         NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         memcached-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      Ensure that Ansible Runner and Ansible Runner HTTP Plug-in are installed or else you will see unexpected errors from Ansible Runner when a CR is created.

      It is also important that the role path referenced in the watches.yaml file exists on your machine. Because normally a container is used where the role is put on disk, the role must be manually copied to the configured Ansible roles path (for example /etc/ansible/roles).

      1. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=config

6. Create a Memcached CR.

   1. Modify the deploy/crds/cache_v1alpha1_memcached_cr.yaml file as shown and create a Memcached CR:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Ensure that the memcached-operator creates the deployment for the CR:

      $ oc get deployment

      Example output

      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   3. Check the pods to confirm three replicas were created:

      $ oc get pods

      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

7. Update the size.

   1. Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Confirm that the Operator changes the deployment size:

      $ oc get deployment

      Example output

      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/service_account.yaml

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

Procedure

1. To track CR status manually from your application, update the watches.yaml file with a manageStatus field set to false:

   - version: v1
     group: api.example.com
     kind: Test1
     role: Test1
     manageStatus: false

2. Use the operator_sdk.util.k8s_status Ansible module to update the subresource. For example, to update with key test1 and value test2, operator_sdk.util can be used as shown:

   - operator_sdk.util.k8s_status:
       api_version: app.example.com/v1
       kind: Test1
       name: "{{ meta.name }}"
       namespace: "{{ meta.namespace }}"
       status:
         test1: test2

   Collections can also be declared in the meta/main.yml for the role, which is included for new scaffolded Ansible Operators:

   collections:
     - operator_sdk.util

   Declaring collections in the role meta allows you to invoke the k8s_status module directly:

   k8s_status:
     <snip>
     status:
       test1: test2

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Helm-based, namespace-scoped nginx-operator project, use the following command:

   $ operator-sdk new nginx-operator \
     --api-version=example.com/v1alpha1 \
     --kind=Nginx \
     --type=helm

   $ cd nginx-operator

   This creates the nginx-operator project specifically for watching the Nginx resource with API version example.com/v1apha1 and kind Nginx.

2. Customize the Operator logic.

   For this example, the nginx-operator executes the following reconciliation logic for each Nginx custom resource (CR):

   • Create an Nginx deployment if it does not exist.
   • Create an Nginx service if it does not exist.
   • Create an Nginx ingress if it is enabled and does not exist.
   • Ensure that the deployment, service, and optional ingress match the desired configuration (for example, replica count, image, service type) as specified by the Nginx CR.

   By default, the nginx-operator watches Nginx resource events as shown in the watches.yaml file and executes Helm releases using the specified chart:

   - version: v1alpha1
     group: example.com
     kind: Nginx
     chart: /opt/helm/helm-charts/nginx

   1. Review the Nginx Helm chart.

      When a Helm Operator project is created, the Operator SDK creates an example Helm chart that contains a set of templates for a simple Nginx release.

      For this example, templates are available for deployment, service, and ingress resources, along with a NOTES.txt template, which Helm chart developers use to convey helpful information about a release.

      If you are not already familiar with Helm Charts, review the Helm Chart developer documentation.

   2. Understand the Nginx CR spec.

      Helm uses a concept called values to provide customizations to the defaults of a Helm chart, which are defined in the values.yaml file.

      Override these defaults by setting the desired values in the CR spec. You can use the number of replicas as an example:

      1. First, inspect the helm-charts/nginx/values.yaml file to find that the chart has a value called replicaCount and it is set to 1 by default. To have 2 Nginx instances in your deployment, your CR spec must contain replicaCount: 2.

         Update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file to look like the following:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2

      2. Similarly, the default service port is set to 80. To instead use 8080, update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file again by adding the service port override:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2
           service:
             port: 8080

         The Helm Operator applies the entire spec as if it was the contents of a values file, just like the helm install -f ./overrides.yaml command works.

3. Deploy the CRD.

   Before running the Operator, Kubernetes must know about the new custom resource definition (CRD) that the Operator will be watching. Deploy the following CRD:

   $ oc create -f deploy/crds/example_v1alpha1_nginx_crd.yaml

4. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the nginx-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/nginx-operator:v0.0.1

         $ podman push quay.io/example/nginx-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml

      3. Deploy the nginx-operator manifests:

         $ oc create -f deploy/service_account.yaml

         $ oc create -f deploy/role.yaml

         $ oc create -f deploy/role_binding.yaml

         $ oc create -f deploy/operator.yaml

      4. Verify that the nginx-operator deployment is up and running:

         $ oc get deployment

         Example output

         NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         nginx-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      It is important that the chart path referenced in the watches.yaml file exists on your machine. By default, the watches.yaml file is scaffolded to work with an Operator image built with the operator-sdk build command. When developing and testing your Operator with the operator-sdk run --local command, the SDK looks in your local file system for this path.

      1. Create a symlink at this location to point to the path of your Helm chart:

         $ sudo mkdir -p /opt/helm/helm-charts

         $ sudo ln -s $PWD/helm-charts/nginx /opt/helm/helm-charts/nginx

      2. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=<path_to_config>

5. Deploy the Nginx CR.

   Apply the Nginx CR that you modified earlier:

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Ensure that the nginx-operator creates the deployment for the CR:

   $ oc get deployment

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        2         2         2            2           1m

   Check the pods to confirm two replicas were created:

   $ oc get pods

   Example output

   NAME                                                      READY     STATUS    RESTARTS   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-fjcr9   1/1       Running   0          1m
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-ljbzl   1/1       Running   0          1m

   Check that the service port is set to 8080:

   $ oc get service

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        8080/TCP   1m

6. Update the replicaCount and remove the port.

   Change the spec.replicaCount field from 2 to 3, remove the spec.service field, and apply the change:

   $ cat deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Example output

   apiVersion: "example.com/v1alpha1"
   kind: "Nginx"
   metadata:
     name: "example-nginx"
   spec:
     replicaCount: 3

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Confirm that the Operator changes the deployment size:

   $ oc get deployment

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        3         3         3            3           1m

   Check that the service port is set to the default 80:

   $ oc get service

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)  AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        80/TCP   1m

7. Clean up the resources:

   $ oc delete -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/service_account.yaml

   $ oc delete -f deploy/crds/example_v1alpha1_nginx_crd.yaml

Procedure

1. In your Operator project, configure your CSV composition by modifying the deploy/olm-catalog/csv-config.yaml file, if desired.

2. Generate the CSV:

   $ operator-sdk generate csv --csv-version <version>

3. In the new CSV generated in the deploy/olm-catalog/ directory, ensure all required, manually-defined fields are set appropriately.

Procedure

1. From your Operator project directory, build the bundle image using the Operator SDK:

   $ operator-sdk bundle create \
       quay.io/<namespace>/test-operator:v0.1.0 \1
       -b podman 2

   1

   The image tag that you want the bundle image to have.

   2

   The CLI tool to use for building the container image, either docker (default), podman, or buildah. This example uses podman.

   Note

   If your local manifests are not located in the default <project_root>/deploy/olm-catalog/test-operator/manifests, specify the location with the --directory flag.

2. Log in to the registry where you want to push the bundle image. For example:

   $ podman login quay.io

3. Push the bundle image to the registry:

   $ podman push quay.io/<namespace>/test-operator:v0.1.0

4. Validate the bundle image in the remote registry:

   $ operator-sdk bundle validate \
       quay.io/<namespace>/test-operator:v0.1.0 \
       -b podman

   Example output

   INFO[0000] Unpacked image layers                                 bundle-dir=/tmp/bundle-041168359 container-tool=podman
   INFO[0000] running podman pull                                   bundle-dir=/tmp/bundle-041168359 container-tool=podman
   INFO[0002] running podman save                                   bundle-dir=/tmp/bundle-041168359 container-tool=podman
   INFO[0002] All validation tests have completed successfully      bundle-dir=/tmp/bundle-041168359 container-tool=podman

Procedure

1. Define a .osdk-scorecard.yaml configuration file in your Operator project.
2. Create the namespace defined in the RBAC files (role_binding).

3. Run the scorecard from the root directory of your Operator project:

   $ operator-sdk scorecard

   The scorecard return code is 1 if any of the executed texts did not pass and 0 if all selected tests passed.

Procedure

1. The scorecard requires a proxy container in the deployment pod of the Operator to read Operator logs. A few modifications to your CSV and creation of one extra object are required to run the proxy before deploying your Operator with Operator Lifecycle Manager (OLM).

   This step can be performed manually or automated using bash functions. Choose one of the following methods.

   • Manual method:

     1. Create a proxy server secret containing a local kubeconfig file`.

        1. Generate a user name using the namespaced owner reference of the scorecard proxy.

           $ echo '{"apiVersion":"","kind":"","name":"scorecard","uid":"","Namespace":"'<namespace>'"}' | base64 -w 0 1

           1

           Replace <namespace> with the namespace your Operator will deploy in.

        2. Write a Config manifest scorecard-config.yaml using the following template, replacing <username> with the base64 user name generated in the previous step:

           apiVersion: v1
           kind: Config
           clusters:
           - cluster:
               insecure-skip-tls-verify: true
               server: http://<username>@localhost:8889
             name: proxy-server
           contexts:
           - context:
               cluster: proxy-server
               user: admin/proxy-server
             name: <namespace>/proxy-server
           current-context: <namespace>/proxy-server
           preferences: {}
           users:
           - name: admin/proxy-server
             user:
               username: <username>
               password: unused

        3. Encode the Config as base64:

           $ cat scorecard-config.yaml | base64 -w 0

        4. Create a Secret manifest scorecard-secret.yaml:

           apiVersion: v1
           kind: Secret
           metadata:
             name: scorecard-kubeconfig
             namespace: <namespace> 1
           data:
             kubeconfig: <kubeconfig_base64> 2

           1

           Replace <namespace> with the namespace your Operator will deploy in.

           2

           Replace <kubeconfig_base64> with the Config encoded as base64.

        5. Apply the secret:

           $ oc apply -f scorecard-secret.yaml

        6. Insert a volume referring to the secret into the deployment for the Operator:

           spec:
             install:
               spec:
                 deployments:
                 - name: memcached-operator
                   spec:
                     ...
                     template:
                       ...
                       spec:
                         containers:
                         ...
                         volumes:
                         - name: scorecard-kubeconfig 1
                           secret:
                             secretName: scorecard-kubeconfig
                             items:
                             - key: kubeconfig
                               path: config

           1

           Scorecard kubeconfig volume.

     2. Insert a volume mount and KUBECONFIG environment variable into each container in the deployment of your Operator:

        spec:
          install:
            spec:
              deployments:
              - name: memcached-operator
                spec:
                  ...
                  template:
                    ...
                    spec:
                      containers:
                      - name: container1
                        ...
                        volumeMounts:
                        - name: scorecard-kubeconfig 1
                          mountPath: /scorecard-secret
                        env:
                        - name: KUBECONFIG 2
                          value: /scorecard-secret/config
                      - name: container2 3
                        ...

        1

        Scorecard kubeconfig volume mount.

        2

        Scorecard kubeconfig environment variable.

        3

        Repeat the same for this and all other containers.

     3. Insert the scorecard proxy container into the deployment of your Operator:

        spec:
          install:
            spec:
              deployments:
              - name: memcached-operator
                spec:
                  ...
                  template:
                    ...
                    spec:
                      containers:
                      ...
                      - name: scorecard-proxy 1
                        command:
                        - scorecard-proxy
                        env:
                        - name: WATCH_NAMESPACE
                          valueFrom:
                            fieldRef:
                              apiVersion: v1
                              fieldPath: metadata.namespace
                        image: quay.io/operator-framework/scorecard-proxy:master
                        imagePullPolicy: Always
                        ports:
                        - name: proxy
                          containerPort: 8889

        1

        Scorecard proxy container.

   • Automated method:

     The community-operators repository has several bash functions that can perform the previous steps in the procedure for you.

     1. Run the following curl command:

        $ curl -Lo csv-manifest-modifiers.sh \
            https://raw.githubusercontent.com/operator-framework/community-operators/master/scripts/lib/file

     2. Source the csv-manifest-modifiers.sh file:

        $ . ./csv-manifest-modifiers.sh

     3. Create the kubeconfig secret file:

        $ create_kubeconfig_secret_file scorecard-secret.yaml "<namespace>" 1

        1

        Replace <namespace> with the namespace your Operator will deploy in.

     4. Apply the secret:

        $ oc apply -f scorecard-secret.yaml

     5. Insert the kubeconfig volume:

        $ insert_kubeconfig_volume "<csv_file>" 1

        1

        Replace <csv_file> with the path to your CSV manifest.

     6. Insert the kubeconfig secret mount:

        $ insert_kubeconfig_secret_mount "<csv_file>"

     7. Insert the proxy container:

        $ insert_proxy_container "<csv_file>" "quay.io/operator-framework/scorecard-proxy:master"

2. After inserting the proxy container, follow the steps in the Getting started with the Operator SDK guide to bundle your CSV and custom resource definitions (CRDs) and deploy your Operator on OLM.
3. After your Operator has been deployed on OLM, define a .osdk-scorecard.yaml configuration file in your Operator project and ensure both the csv-path: <csv_manifest_path> and olm-deployed options are set.

4. Run the scorecard with both the csv-path: <csv_manifest_path> and olm-deployed options set in your scorecard configuration file:

   $ operator-sdk scorecard