Operators

Procedure

1. Navigate in the web console to the Operators → OperatorHub page.

2. Scroll or type a keyword into the Filter by keyword box (in this case, Couchbase) to find the Operator you want.

   Figure 4.1. Filter Operators by keyword

   
3. Select the Operator. For a Community Operator, you are warned that Red Hat does not certify those Operators. You must acknowledge that warning before continuing. Information about the Operator is displayed.
4. Read the information about the Operator and click Install.

5. On the Create Operator Subscription page:

   1. Select one of the following:

      • All namespaces on the cluster (default) installs the Operator in the default openshift-operators namespace to watch and be made available to all namespaces in the cluster. This option is not always available.
      • A specific namespace on the cluster allows you to choose a specific, single namespace in which to install the Operator. The Operator will only watch and be made available for use in this single namespace.
   2. Select an Update Channel (if more than one is available).
   3. Select Automatic or Manual approval strategy, as described earlier.

6. Click Subscribe to make the Operator available to the selected namespaces on this OpenShift Container Platform cluster.

   1. If you selected a Manual approval strategy, the Subscription’s upgrade status will remain Upgrading until you review and approve its Install Plan.

      Figure 4.2. Manually approving from the Install Plan page

      

      After approving on the Install Plan page, the Subscription upgrade status moves to Up to date.

   2. If you selected an Automatic approval strategy, the upgrade status should resolve to Up to date without intervention.

      Figure 4.3. Subscription upgrade status Up to date

      

7. After the Subscription’s upgrade status is Up to date, select Operators → Installed Operators to verify that the Couchbase ClusterServiceVersion (CSV) eventually shows up and its Status ultimately resolves to InstallSucceeded in the relevant namespace.

   Note

   For the All namespaces…​ Installation Mode, the status resolves to InstallSucceeded in the openshift-operators namespace, but the status is Copied if you check in other namespaces.

   If it does not:

   1. Check the logs in any Pods in the openshift-operators project (or other relevant namespace if A specific namespace…​ Installation Mode was selected) on the Workloads → Pods page that are reporting issues to troubleshoot further.

Procedure

1. View the list of Operators available to the cluster from the OperatorHub.

   $ oc get packagemanifests -n openshift-marketplace
   NAME                               CATALOG               AGE
   3scale-operator                    Red Hat Operators     91m
   amq-online                         Red Hat Operators     91m
   amq-streams                        Red Hat Operators     91m
   ...
   couchbase-enterprise-certified     Certified Operators   91m
   mariadb                            Certified Operators   91m
   mongodb-enterprise                 Certified Operators   91m
   ...
   etcd                               Community Operators   91m
   jaeger                             Community Operators   91m
   kubefed                            Community Operators   91m
   ...

   Note the CatalogSource(s) for your desired Operator(s).

2. Inspect your desired Operator to verify its supported InstallModes and available Channels:

   $ oc describe packagemanifests <operator_name> -n openshift-marketplace

3. The namespace to which you subscribe the Operator must have an OperatorGroup that matches the Operator’s InstallMode, either the AllNamespaces or SingleNamespace mode. If the Operator you intend to install uses the AllNamespaces, then the openshift-operators namespace already has an appropriate OperatorGroup in place.

   However, if the Operator uses the SingleNamespace mode and you do not already have an appropriate OperatorGroup in place, you must create one.

   Note

   The web console version of this procedure handles the creation of the OperatorGroup and Subscription objects automatically behind the scenes for you when choosing SingleNamespace mode.

   1. Create an OperatorGroup object YAML file, for example operatorgroup.yaml:

      Example OperatorGroup

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: <operatorgroup_name>
        namespace: <namespace>
      spec:
        targetNamespaces:
        - <namespace>

   2. Create the OperatorGroup object:

      $ oc apply -f operatorgroup.yaml

4. Create a Subscription object YAML file to subscribe a namespace to an Operator, for example sub.yaml:

   Example Subscription

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: <operator_name>
     namespace: openshift-operators 1
   spec:
     channel: alpha
     name: <operator_name> 2
     source: redhat-operators 3
     sourceNamespace: openshift-marketplace 4

   1

   For AllNamespaces InstallMode usage, specify the openshift-operators namespace. Otherwise, specify the relevant single namespace for SingleNamespace InstallMode usage.

   2

   Name of the Operator to subscribe to.

   3

   Name of the CatalogSource that provides the Operator.

   4

   Namespace of the CatalogSource. Use openshift-marketplace for the default OperatorHub CatalogSources.

5. Create the Subscription object:

   $ oc apply -f sub.yaml

   At this point, the OLM is now aware of the selected Operator. A ClusterServiceVersion (CSV) for the Operator should appear in the target namespace, and APIs provided by the Operator should be available for creation.

1. The given Subscription object is picked up by OLM.
2. OLM fetches the OperatorGroup tied to this Subscription.
3. OLM determines that the OperatorGroup has a service account specified.
4. OLM creates a client scoped to the service account and uses the scoped client to install the Operator. This ensures that any permission requested by the Operator is always confined to that of the service account in the OperatorGroup.
5. OLM creates a new service account with the set of permissions specified in the CSV and assigns it to the Operator. The Operator runs as the assigned service account.

1. Create a YAML file that contains the following field types:

   Example YAML file for a CRD

   apiVersion: apiextensions.k8s.io/v1beta1 1
   kind: CustomResourceDefinition
   metadata:
     name: crontabs.stable.example.com 2
   spec:
     group: stable.example.com 3
     version: v1 4
     scope: Namespaced 5
     names:
       plural: crontabs 6
       singular: crontab 7
       kind: CronTab 8
       shortNames:
       - ct 9

   1

   Use the apiextensions.k8s.io/v1beta1 API.

   2

   Specify a name for the definition. This must be in the <plural-name>.<group> format using the values from the group and plural fields.

   3

   Specify a group name for the API. An API group is a collection of objects that are logically related. For example, all batch objects like Job or ScheduledJob could be in the batch API Group (such as batch.api.example.com). A good practice is to use a fully-qualified-domain name of your organization.

   4

   Specify a version name to be used in the URL. Each API Group can exist in multiple versions. For example: v1alpha, v1beta, v1.

   5

   Specify whether the custom objects are available to a project (Namespaced) or all projects in the cluster (Cluster).

   6

   Specify the plural name to use in the URL. The plural field is the same as a resource in an API URL.

   7

   Specify a singular name to use as an alias on the CLI and for display.

   8

   Specify the kind of objects that can be created. The type can be in CamelCase.

   9

   Specify a shorter string to match your resource on the CLI.

   Note

   By default, a CRD is cluster-scoped and available to all projects.

2. Create the CRD object:

   $ oc create -f <file_name>.yaml

   A new RESTful API endpoint is created at:

   /apis/<spec:group>/<spec:version>/<scope>/*/<names-plural>/...

   For example, using the example file, the following endpoint is created:

   /apis/stable.example.com/v1/namespaces/*/crontabs/...

   You can now use this endpoint URL to create and manage CRs. The object Kind is based on the spec.kind field of the CRD object you created.

Procedure

1. Create a cluster role definition file for the CRD. The cluster role definition is a YAML file that contains the rules that apply to each cluster role. The OpenShift Container Platform controller adds the rules that you specify to the default cluster roles.

   Example YAML file for a cluster role definition

   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1 1
   metadata:
     name: aggregate-cron-tabs-admin-edit 2
     labels:
       rbac.authorization.k8s.io/aggregate-to-admin: "true" 3
       rbac.authorization.k8s.io/aggregate-to-edit: "true" 4
   rules:
   - apiGroups: ["stable.example.com"] 5
     resources: ["crontabs"] 6
     verbs: ["get", "list", "watch", "create", "update", "patch", "delete", "deletecollection"] 7
   ---
   kind: ClusterRole
   apiVersion: rbac.authorization.k8s.io/v1
   metadata:
     name: aggregate-cron-tabs-view 8
     labels:
       # Add these permissions to the "view" default role.
       rbac.authorization.k8s.io/aggregate-to-view: "true" 9
       rbac.authorization.k8s.io/aggregate-to-cluster-reader: "true" 10
   rules:
   - apiGroups: ["stable.example.com"] 11
     resources: ["crontabs"] 12
     verbs: ["get", "list", "watch"] 13

   1

   Use the rbac.authorization.k8s.io/v1 API.

   2 8

   Specify a name for the definition.

   3

   Specify this label to grant permissions to the admin default role.

   4

   Specify this label to grant permissions to the edit default role.

   5 11

   Specify the group name of the CRD.

   6 12

   Specify the plural name of the CRD that these rules apply to.

   7 13

   Specify the verbs that represent the permissions that are granted to the role. For example, apply read and write permissions to the admin and edit roles and only read permission to the view role.

   9

   Specify this label to grant permissions to the view default role.

   10

   Specify this label to grant permissions to the cluster-reader default role.

2. Create the cluster role:

   $ oc create -f <file_name>.yaml

Procedure

1. Create a YAML file for the CR. In the following example definition, the cronSpec and image custom fields are set in a CR of Kind: CronTab. The Kind comes from the spec.kind field of the CRD object.

   Example YAML file for a CR

   apiVersion: "stable.example.com/v1" 1
   kind: CronTab 2
   metadata:
     name: my-new-cron-object 3
     finalizers: 4
     - finalizer.stable.example.com
   spec: 5
     cronSpec: "* * * * /5"
     image: my-awesome-cron-image

   1

   Specify the group name and API version (name/version) from the Custom Resource Definition.

   2

   Specify the type in the CRD.

   3

   Specify a name for the object.

   4

   Specify the finalizers for the object, if any. Finalizers allow controllers to implement conditions that must be completed before the object can be deleted.

   5

   Specify conditions specific to the type of object.

2. After you create the file, create the object:

   $ oc create -f <file_name>.yaml

Procedure

1. To get information on a specific Kind of a CR, run:

   $ oc get <kind>

   For example:

   $ oc get crontab
   
   NAME                 KIND
   my-new-cron-object   CronTab.v1.stable.example.com

   Resource names are not case-sensitive, and you can use either the singular or plural forms defined in the CRD, as well as any short name. For example:

   $ oc get crontabs
   $ oc get crontab
   $ oc get ct

2. You can also view the raw YAML data for a CR:

   $ oc get <kind> -o yaml

   $ oc get ct -o yaml
   
   apiVersion: v1
   items:
   - apiVersion: stable.example.com/v1
     kind: CronTab
     metadata:
       clusterName: ""
       creationTimestamp: 2017-05-31T12:56:35Z
       deletionGracePeriodSeconds: null
       deletionTimestamp: null
       name: my-new-cron-object
       namespace: default
       resourceVersion: "285"
       selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
       uid: 9423255b-4600-11e7-af6a-28d2447dc82b
     spec:
       cronSpec: '* * * * /5' 1
       image: my-awesome-cron-image 2

   1 2

   Custom data from the YAML that you used to create the object displays.

Procedure

1. Create a YAML file for the CR. In the following example definition, the cronSpec and image custom fields are set in a CR of Kind: CronTab. The Kind comes from the spec.kind field of the CRD object.

   Example YAML file for a CR

   apiVersion: "stable.example.com/v1" 1
   kind: CronTab 2
   metadata:
     name: my-new-cron-object 3
     finalizers: 4
     - finalizer.stable.example.com
   spec: 5
     cronSpec: "* * * * /5"
     image: my-awesome-cron-image

   1

   Specify the group name and API version (name/version) from the Custom Resource Definition.

   2

   Specify the type in the CRD.

   3

   Specify a name for the object.

   4

   Specify the finalizers for the object, if any. Finalizers allow controllers to implement conditions that must be completed before the object can be deleted.

   5

   Specify conditions specific to the type of object.

2. After you create the file, create the object:

   $ oc create -f <file_name>.yaml

Procedure

1. To get information on a specific Kind of a CR, run:

   $ oc get <kind>

   For example:

   $ oc get crontab
   
   NAME                 KIND
   my-new-cron-object   CronTab.v1.stable.example.com

   Resource names are not case-sensitive, and you can use either the singular or plural forms defined in the CRD, as well as any short name. For example:

   $ oc get crontabs
   $ oc get crontab
   $ oc get ct

2. You can also view the raw YAML data for a CR:

   $ oc get <kind> -o yaml

   $ oc get ct -o yaml
   
   apiVersion: v1
   items:
   - apiVersion: stable.example.com/v1
     kind: CronTab
     metadata:
       clusterName: ""
       creationTimestamp: 2017-05-31T12:56:35Z
       deletionGracePeriodSeconds: null
       deletionTimestamp: null
       name: my-new-cron-object
       namespace: default
       resourceVersion: "285"
       selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
       uid: 9423255b-4600-11e7-af6a-28d2447dc82b
     spec:
       cronSpec: '* * * * /5' 1
       image: my-awesome-cron-image 2

   1 2

   Custom data from the YAML that you used to create the object displays.

Procedure

1. Create a new project.

   Use the CLI to create a new memcached-operator project:

   $ mkdir -p $GOPATH/src/github.com/example-inc/
   $ cd $GOPATH/src/github.com/example-inc/
   $ operator-sdk new memcached-operator --dep-manager dep
   $ cd memcached-operator

2. Add a new Custom Resource Definition (CRD).

   1. Use the CLI to add a new CRD API called Memcached, with APIVersion set to cache.example.com/v1apha1 and Kind set to Memcached:

      $ operator-sdk add api \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds the Memcached resource API under pkg/apis/cache/v1alpha1/.

   2. Modify the spec and status of the Memcached Custom Resource (CR) at the pkg/apis/cache/v1alpha1/memcached_types.go file:

      type MemcachedSpec struct {
      	// Size is the size of the memcached deployment
      	Size int32 `json:"size"`
      }
      type MemcachedStatus struct {
      	// Nodes are the names of the memcached pods
      	Nodes []string `json:"nodes"`
      }

   3. After modifying the *_types.go file, always run the following command to update the generated code for that resource type:

      $ operator-sdk generate k8s

3. Add a new Controller.

   1. Add a new Controller to the project to watch and reconcile the Memcached resource:

      $ operator-sdk add controller \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds a new Controller implementation under pkg/controller/memcached/.

   2. For this example, replace the generated controller file pkg/controller/memcached/memcached_controller.go with the example implementation.

      The example controller executes the following reconciliation logic for each Memcached CR:

      • Create a Memcached Deployment if it does not exist.
      • Ensure that the Deployment size is the same as specified by the Memcached CR spec.
      • Update the Memcached CR status with the names of the Memcached Pods.

      The next two sub-steps inspect how the Controller watches resources and how the reconcile loop is triggered. You can skip these steps to go directly to building and running the Operator.

   3. Inspect the Controller implementation at the pkg/controller/memcached/memcached_controller.go file to see how the Controller watches resources.

      The first watch is for the Memcached type as the primary resource. For each Add, Update, or Delete event, the reconcile loop is sent a reconcile Request (a <namespace>:<name> key) for that Memcached object:

      err := c.Watch(
        &source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})

      The next watch is for Deployments, but the event handler maps each event to a reconcile Request for the owner of the Deployment. In this case, this is the Memcached object for which the Deployment was created. This allows the controller to watch Deployments as a secondary resource:

      err := c.Watch(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{
      		IsController: true,
      		OwnerType:    &cachev1alpha1.Memcached{},
      	})

   4. Every Controller has a Reconciler object with a Reconcile() method that implements the reconcile loop. The reconcile loop is passed the Request argument which is a <namespace>:<name> key used to lookup the primary resource object, Memcached, from the cache:

      func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error) {
        // Lookup the Memcached instance for this reconcile request
        memcached := &cachev1alpha1.Memcached{}
        err := r.client.Get(context.TODO(), request.NamespacedName, memcached)
        ...
      }

      Based on the return value of Reconcile() the reconcile Request may be requeued and the loop may be triggered again:

      // Reconcile successful - don't requeue
      return reconcile.Result{}, nil
      // Reconcile failed due to error - requeue
      return reconcile.Result{}, err
      // Requeue for any reason other than error
      return reconcile.Result{Requeue: true}, nil

4. Build and run the Operator.

   1. Before running the Operator, the CRD must be registered with the Kubernetes API server:

      $ oc create \
          -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   2. After registering the CRD, there are two options for running the Operator:

      • As a Deployment inside a Kubernetes cluster
      • As Go program outside a cluster

      Choose one of the following methods.

      1. Option A: Running as a Deployment inside the cluster.

         1. Build the memcached-operator image and push it to a registry:

            $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

         2. The Deployment manifest is generated at deploy/operator.yaml. Update the Deployment image as follows since the default is just a placeholder:

            $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

         3. Ensure you have an account on Quay.io for the next step, or substitute your preferred container registry. On the registry, create a new public image repository named memcached-operator.

         4. Push the image to the registry:

            $ docker push quay.io/example/memcached-operator:v0.0.1

         5. Setup RBAC and deploy memcached-operator:

            $ oc create -f deploy/role.yaml
            $ oc create -f deploy/role_binding.yaml
            $ oc create -f deploy/service_account.yaml
            $ oc create -f deploy/operator.yaml

         6. Verify that memcached-operator is up and running:

            $ oc get deployment
            NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
            memcached-operator       1         1         1            1           1m

      2. Option B: Running locally outside the cluster.

         This method is preferred during development cycle to deploy and test faster.

         Run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk up local --namespace=default

         You can use a specific kubeconfig using the flag --kubeconfig=<path/to/kubeconfig>.

5. Verify that the Operator can deploy a Memcached application by creating a Memcached CR.

   1. Create the example Memcached CR that was generated at deploy/crds/cache_v1alpha1_memcached_cr.yaml:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Ensure that memcached-operator creates the Deployment for the CR:

      $ oc get deployment
      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   3. Check the Pods and CR status to confirm the status is updated with the memcached Pod names:

      $ oc get pods
      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m
      
      $ oc get memcached/example-memcached -o yaml
      apiVersion: cache.example.com/v1alpha1
      kind: Memcached
      metadata:
        clusterName: ""
        creationTimestamp: 2018-03-31T22:51:08Z
        generation: 0
        name: example-memcached
        namespace: default
        resourceVersion: "245453"
        selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached
        uid: 0026cc97-3536-11e8-bd83-0800274106a1
      spec:
        size: 3
      status:
        nodes:
        - example-memcached-6fd7c98d8-7dqdr
        - example-memcached-6fd7c98d8-g5k7v
        - example-memcached-6fd7c98d8-m7vn7

6. Verify that the Operator can manage a deployed Memcached application by updating the size of the deployment.

   1. Change the spec.size field in the memcached CR from 3 to 4:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

   2. Apply the change:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   3. Confirm that the Operator changes the Deployment size:

      $ oc get deployment
      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

7. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml
   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/service_account.yaml

Procedure

1. Generate an Operator manifest.

   An Operator manifest describes how to display, create, and manage the application, in this case Memcached, as a whole. It is defined by a ClusterServiceVersion (CSV) object and is required for the OLM to function.

   You can use the following command to generate CSV manifests:

   $ operator-sdk olm-catalog gen-csv --csv-version 0.0.1

   Note

   This command is run from the memcached-operator/ directory that was created when you built the Memcached Operator.

   For the purpose of this guide, we will continue with this predefined manifest file for the next steps. You can alter the image field within this manifest to reflect the image you built in previous steps, but it is unnecessary.

   Note

   See Building a CSV for the Operator Framework for more information on manually defining a manifest file.

2. Deploy the Operator.

   1. Create an OperatorGroup that specifies the namespaces that the Operator will target. Create the following OperatorGroup in the namespace where you will create the CSV. In this example, the default namespace is used:

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: memcached-operator-group
        namespace: default
      spec:
        targetNamespaces:
        - default

   2. Apply the Operator’s CSV manifest to the specified namespace in the cluster:

      $ curl -Lo memcachedoperator.0.0.1.csv.yaml https://raw.githubusercontent.com/operator-framework/getting-started/master/memcachedoperator.0.0.1.csv.yaml
      $ oc apply -f memcachedoperator.0.0.1.csv.yaml
      $ oc get csv memcachedoperator.v0.0.1 -n default -o json | jq '.status'

      When you apply this manifest, the cluster does not immediately update because it does not yet meet the requirements specified in the manifest.

   3. Create the role, role binding, and service account to grant resource permissions to the Operator, and the Custom Resource Definition (CRD) to create the Memcached type that the Operator manages:

      $ oc create -f deploy/crds/cache_v1alpha1_memcached_crd.yaml
      $ oc create -f deploy/service_account.yaml
      $ oc create -f deploy/role.yaml
      $ oc create -f deploy/role_binding.yaml

      Note

      These files were generated into the deploy/ directory by the Operator SDK when you built the Memcached Operator.

      Because the OLM creates Operators in a particular namespace when a manifest is applied, administrators can leverage the native Kubernetes RBAC permission model to restrict which users are allowed to install Operators.

3. Create an application instance.

   The Memcached Operator is now running in the default namespace. Users interact with Operators via instances of CustomResources; in this case, the resource has the kind Memcached. Native Kubernetes RBAC also applies to CustomResources, providing administrators control over who can interact with each Operator.

   Creating instances of Memcached in this namespace will now trigger the Memcached Operator to instantiate pods running the memcached server that are managed by the Operator. The more CustomResources you create, the more unique instances of Memcached are managed by the Memcached Operator running in this namespace.

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-wordpress"
   spec:
     size: 1
   EOF
   
   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-drupal"
   spec:
     size: 1
   EOF
   
   $ oc get Memcached
   NAME                      AGE
   memcached-for-drupal      22s
   memcached-for-wordpress   27s
   
   $ oc get pods
   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          14m
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          3s
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          8s

4. Update an application.

   Manually apply an update to the Operator by creating a new Operator manifest with a replaces field that references the old Operator manifest. The OLM ensures that all resources being managed by the old Operator have their ownership moved to the new Operator without fear of any programs stopping execution. It is up to the Operators themselves to execute any data migrations required to upgrade resources to run under a new version of the Operator.

   The following command demonstrates applying a new Operator manifest file using a new version of the Operator and shows that the pods remain executing:

   $ curl -Lo memcachedoperator.0.0.2.csv.yaml https://raw.githubusercontent.com/operator-framework/getting-started/master/memcachedoperator.0.0.2.csv.yaml
   $ oc apply -f memcachedoperator.0.0.2.csv.yaml
   $ oc get pods
   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          3s
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          14m
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          14m

Procedure

1. Create a new Operator project, either namespace-scoped or cluster-scoped, using the operator-sdk new command. Choose one of the following:

   1. A namespace-scoped Operator (the default) watches and manages resources in a single namespace. Namespace-scoped operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

      To create a new Ansible-based, namespace-scoped memcached-operator project and change to its directory, use the following commands:

      $ operator-sdk new memcached-operator \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached \
          --type=ansible
      $ cd memcached-operator

      This creates the memcached-operator project specifically for watching the Memcached resource with APIVersion example.com/v1apha1 and Kind Memcached.

   2. A cluster-scoped Operator watches and manages resources cluster-wide, which can be useful in certain cases. For example, the cert-manager operator is often deployed with cluster-scoped permissions and watches so that it can manage issuing certificates for an entire cluster.

      To create your memcached-operator project to be cluster-scoped and change to its directory, use the following commands:

      $ operator-sdk new memcached-operator \
          --cluster-scoped \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached \
          --type=ansible
      $ cd memcached-operator

      Using the --cluster-scoped flag scaffolds the new Operator with the following modifications:

      • deploy/operator.yaml: Set WATCH_NAMESPACE="" instead of setting it to the Pod’s namespace.
      • deploy/role.yaml: Use ClusterRole instead of Role.

      • deploy/role_binding.yaml:

        • Use ClusterRoleBinding instead of RoleBinding.
        • Set the subject namespace to REPLACE_NAMESPACE. This must be changed to the namespace in which the Operator is deployed.

2. Customize the Operator logic.

   For this example, the memcached-operator executes the following reconciliation logic for each Memcached Custom Resource (CR):

   • Create a memcached Deployment if it does not exist.
   • Ensure that the Deployment size is the same as specified by the Memcached CR.

   By default, the memcached-operator watches Memcached resource events as shown in the watches.yaml file and executes the Ansible role Memcached:

   - version: v1alpha1
     group: cache.example.com
     kind: Memcached

   You can optionally customize the following logic in the watches.yaml file:

   1. Specifying a role option configures the Operator to use this specified path when launching ansible-runner with an Ansible role. By default, the new command fills in an absolute path to where your role should go:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        role: /opt/ansible/roles/memcached

   2. Specifying a playbook option in the watches.yaml file configures the Operator to use this specified path when launching ansible-runner with an Ansible playbook:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        playbook: /opt/ansible/playbook.yaml

3. Build the Memcached Ansible role.

   Modify the generated Ansible role under the roles/memcached/ directory. This Ansible role controls the logic that is executed when a resource is modified.

   1. Define the Memcached spec.

      Defining the spec for an Ansible-based Operator can be done entirely in Ansible. The Ansible Operator passes all key-value pairs listed in the CR spec field along to Ansible as variables. The names of all variables in the spec field are converted to snake case (lowercase with an underscore) by the Operator before running Ansible. For example, serviceAccount in the spec becomes service_account in Ansible.

      Tip

      You should perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

      In case the user does not set the spec field, set a default by modifying the roles/memcached/defaults/main.yml file:

      size: 1

   2. Define the Memcached Deployment.

      With the Memcached spec now defined, you can define what Ansible is actually executed on resource changes. Because this is an Ansible role, the default behavior executes the tasks in the roles/memcached/tasks/main.yml file.

      The goal is for Ansible to create a Deployment if it does not exist, which runs the memcached:1.4.36-alpine image. Ansible 2.7+ supports the k8s Ansible module, which this example leverages to control the Deployment definition.

      Modify the roles/memcached/tasks/main.yml to match the following:

      - name: start memcached
        k8s:
          definition:
            kind: Deployment
            apiVersion: apps/v1
            metadata:
              name: '{{ meta.name }}-memcached'
              namespace: '{{ meta.namespace }}'
            spec:
              replicas: "{{size}}"
              selector:
                matchLabels:
                  app: memcached
              template:
                metadata:
                  labels:
                    app: memcached
                spec:
                  containers:
                  - name: memcached
                    command:
                    - memcached
                    - -m=64
                    - -o
                    - modern
                    - -v
                    image: "docker.io/memcached:1.4.36-alpine"
                    ports:
                      - containerPort: 11211

      Note

      This example used the size variable to control the number of replicas of the Memcached Deployment. This example sets the default to 1, but any user can create a CR that overwrites the default.

4. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new Custom Resource Definition (CRD) the Operator will be watching. Deploy the Memcached CRD:

   $ oc create -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

5. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a Pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a Pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the memcached-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/memcached-operator:v0.0.1
         $ podman push quay.io/example/memcached-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

      3. If you created your Operator using the --cluster-scoped=true flag, update the service account namespace in the generated ClusterRoleBinding to match where you are deploying your Operator:

         $ export OPERATOR_NAMESPACE=$(oc config view --minify -o jsonpath='{.contexts[0].context.namespace}')
         $ sed -i "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml

         If you are performing these steps on OSX, use the following commands instead:

         $ sed -i "" 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml
         $ sed -i "" "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml

      4. Deploy the memcached-operator:

         $ oc create -f deploy/service_account.yaml
         $ oc create -f deploy/role.yaml
         $ oc create -f deploy/role_binding.yaml
         $ oc create -f deploy/operator.yaml

      5. Verify that the memcached-operator is up and running:

         $ oc get deployment
         NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         memcached-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      Ensure that Ansible Runner and Ansible Runner HTTP Plug-in are installed or else you will see unexpected errors from Ansible Runner when a CR is created.

      It is also important that the role path referenced in the watches.yaml file exists on your machine. Because normally a container is used where the role is put on disk, the role must be manually copied to the configured Ansible roles path (for example /etc/ansible/roles).

      1. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk up local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk up local --kubeconfig=config

6. Create a Memcached CR.

   1. Modify the deploy/crds/cache_v1alpha1_memcached_cr.yaml file as shown and create a Memcached CR:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Ensure that the memcached-operator creates the Deployment for the CR:

      $ oc get deployment
      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   3. Check the Pods to confirm three replicas were created:

      $ oc get pods
      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

7. Update the size.

   1. Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml
      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4
      
      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Confirm that the Operator changes the Deployment size:

      $ oc get deployment
      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/service_account.yaml
   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

Procedure

1. To track CR status manually from your application, update the Watches file with a manageStatus field set to false:

   - version: v1
     group: api.example.com
     kind: Foo
     role: /opt/ansible/roles/Foo
     manageStatus: false

2. Then, use the k8s_status Ansible module to update the subresource. For example, to update with key foo and value bar, k8s_status can be used as shown:

   - k8s_status:
     api_version: app.example.com/v1
     kind: Foo
     name: "{{ meta.name }}"
     namespace: "{{ meta.namespace }}"
     status:
       foo: bar

Procedure

1. Create a new Operator project, either namespace-scoped or cluster-scoped, using the operator-sdk new command. Choose one of the following:

   1. A namespace-scoped Operator (the default) watches and manages resources in a single namespace. Namespace-scoped operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

      To create a new Helm-based, namespace-scoped nginx-operator project, use the following command:

      $ operator-sdk new nginx-operator \
        --api-version=example.com/v1alpha1 \
        --kind=Nginx \
        --type=helm
      $ cd nginx-operator

      This creates the nginx-operator project specifically for watching the Nginx resource with APIVersion example.com/v1apha1 and Kind Nginx.

   2. A cluster-scoped Operator watches and manages resources cluster-wide, which can be useful in certain cases. For example, the cert-manager operator is often deployed with cluster-scoped permissions and watches so that it can manage issuing certificates for an entire cluster.

      To create your nginx-operator project to be cluster-scoped, use the following command:

      $ operator-sdk new nginx-operator \
          --cluster-scoped \
          --api-version=example.com/v1alpha1 \
          --kind=Nginx \
          --type=helm

      Using the --cluster-scoped flag scaffolds the new Operator with the following modifications:

      • deploy/operator.yaml: Set WATCH_NAMESPACE="" instead of setting it to the Pod’s namespace.
      • deploy/role.yaml: Use ClusterRole instead of Role.

      • deploy/role_binding.yaml:

        • Use ClusterRoleBinding instead of RoleBinding.
        • Set the subject namespace to REPLACE_NAMESPACE. This must be changed to the namespace in which the Operator is deployed.

2. Customize the Operator logic.

   For this example, the nginx-operator executes the following reconciliation logic for each Nginx Custom Resource (CR):

   • Create a Nginx Deployment if it does not exist.
   • Create a Nginx Service if it does not exist.
   • Create a Nginx Ingress if it is enabled and does not exist.
   • Ensure that the Deployment, Service, and optional Ingress match the desired configuration (for example, replica count, image, service type) as specified by the Nginx CR.

   By default, the nginx-operator watches Nginx resource events as shown in the watches.yaml file and executes Helm releases using the specified chart:

   - version: v1alpha1
     group: example.com
     kind: Nginx
     chart: /opt/helm/helm-charts/nginx

   1. Review the Nginx Helm chart.

      When a Helm Operator project is created, the Operator SDK creates an example Helm chart that contains a set of templates for a simple Nginx release.

      For this example, templates are available for Deployment, Service, and Ingress resources, along with a NOTES.txt template, which Helm chart developers use to convey helpful information about a release.

      If you are not already familiar with Helm Charts, take a moment to review the Helm Chart developer documentation.

   2. Understand the Nginx CR spec.

      Helm uses a concept called values to provide customizations to a Helm chart’s defaults, which are defined in the Helm chart’s values.yaml file.

      Override these defaults by setting the desired values in the CR spec. You can use the number of replicas as an example:

      1. First, inspect the helm-charts/nginx/values.yaml file to find that the chart has a value called replicaCount and it is set to 1 by default. To have 2 Nginx instances in your deployment, your CR spec must contain replicaCount: 2.

         Update the deploy/crds/example_v1alpha1_nginx_cr.yaml file to look like the following:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2

      2. Similarly, the default service port is set to 80. To instead use 8080, update the deploy/crds/example_v1alpha1_nginx_cr.yaml file again by adding the service port override:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2
           service:
             port: 8080

         The Helm Operator applies the entire spec as if it was the contents of a values file, just like the helm install -f ./overrides.yaml command works.

3. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new custom resource definition (CRD) the operator will be watching. Deploy the following CRD:

   $ oc create -f deploy/crds/example_v1alpha1_nginx_crd.yaml

4. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a Pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a Pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the nginx-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/nginx-operator:v0.0.1
         $ docker push quay.io/example/nginx-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml

      3. If you created your Operator using the --cluster-scoped=true flag, update the service account namespace in the generated ClusterRoleBinding to match where you are deploying your Operator:

         $ export OPERATOR_NAMESPACE=$(oc config view --minify -o jsonpath='{.contexts[0].context.namespace}')
         $ sed -i "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml

         If you are performing these steps on OSX, use the following commands instead:

         $ sed -i "" 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml
         $ sed -i "" "s|REPLACE_NAMESPACE|$OPERATOR_NAMESPACE|g" deploy/role_binding.yaml

      4. Deploy the nginx-operator:

         $ oc create -f deploy/service_account.yaml
         $ oc create -f deploy/role.yaml
         $ oc create -f deploy/role_binding.yaml
         $ oc create -f deploy/operator.yaml

      5. Verify that the nginx-operator is up and running:

         $ oc get deployment
         NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         nginx-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      It is important that the chart path referenced in the watches.yaml file exists on your machine. By default, the watches.yaml file is scaffolded to work with an Operator image built with the operator-sdk build command. When developing and testing your operator with the operator-sdk up local command, the SDK looks in your local file system for this path.

      1. Create a symlink at this location to point to your Helm chart’s path:

         $ sudo mkdir -p /opt/helm/helm-charts
         $ sudo ln -s $PWD/helm-charts/nginx /opt/helm/helm-charts/nginx

      2. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk up local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk up local --kubeconfig=<path_to_config>

5. Deploy the Nginx CR.

   Apply the Nginx CR that you modified earlier:

   $ oc apply -f deploy/crds/example_v1alpha1_nginx_cr.yaml

   Ensure that the nginx-operator creates the Deployment for the CR:

   $ oc get deployment
   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        2         2         2            2           1m

   Check the Pods to confirm two replicas were created:

   $ oc get pods
   NAME                                                      READY     STATUS    RESTARTS   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-fjcr9   1/1       Running   0          1m
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-ljbzl   1/1       Running   0          1m

   Check that the Service port is set to 8080:

   $ oc get service
   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        8080/TCP   1m

6. Update the replicaCount and remove the port.

   Change the spec.replicaCount field from 2 to 3, remove the spec.service field, and apply the change:

   $ cat deploy/crds/example_v1alpha1_nginx_cr.yaml
   apiVersion: "example.com/v1alpha1"
   kind: "Nginx"
   metadata:
     name: "example-nginx"
   spec:
     replicaCount: 3
   
   $ oc apply -f deploy/crds/example_v1alpha1_nginx_cr.yaml

   Confirm that the Operator changes the Deployment size:

   $ oc get deployment
   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        3         3         3            3           1m

   Check that the Service port is set to the default 80:

   $ oc get service
   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)  AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        80/TCP   1m

7. Clean up the resources:

   $ oc delete -f deploy/crds/example_v1alpha1_nginx_cr.yaml
   $ oc delete -f deploy/operator.yaml
   $ oc delete -f deploy/role_binding.yaml
   $ oc delete -f deploy/role.yaml
   $ oc delete -f deploy/service_account.yaml
   $ oc delete -f deploy/crds/example_v1alpha1_nginx_crd.yaml

Procedure

1. In your Operator project, configure your CSV composition by modifying the deploy/olm-catalog/csv-config.yaml file, if desired.

2. Generate the CSV:

   $ operator-sdk olm-catalog gen-csv --csv-version <version>

3. In the new CSV generated in the deploy/olm-catalog/ directory, ensure all required, manually-defined fields are set appropriately.