Operators

Procedure

1. Create a new project in the OpenShift Container Platform web console for this procedure. This example uses a project called my-etcd.

2. Navigate to the Operators → Installed Operators page. The Operators that have been installed to the cluster by the cluster administrator and are available for use are shown here as a list of cluster service versions (CSVs). CSVs are used to launch and manage the software provided by the Operator.

   Tip

   You can get this list from the CLI using:

   $ oc get csv

3. On the Installed Operators page, click Copied, and then click the etcd Operator to view more details and available actions.

   As shown under Provided APIs, this Operator makes available three new resource types, including one for an etcd Cluster (the EtcdCluster resource). These objects work similar to the built-in native Kubernetes ones, such as Deployment or ReplicaSet, but contain logic specific to managing etcd.

4. Create a new etcd cluster:

   1. In the etcd Cluster API box, click Create New.
   2. The next screen allows you to make any modifications to the minimal starting template of an EtcdCluster object, such as the size of the cluster. For now, click Create to finalize. This triggers the Operator to start up the pods, services, and other components of the new etcd cluster.

5. Click the Resources tab to see that your project now contains a number of resources created and configured automatically by the Operator.

   Verify that a Kubernetes service has been created that allows you to access the database from other pods in your project.

6. All users with the edit role in a given project can create, manage, and delete application instances (an etcd cluster, in this example) managed by Operators that have already been created in the project, in a self-service manner, just like a cloud service. If you want to enable additional users with this ability, project administrators can add the role using the following command:

   $ oc policy add-role-to-user edit <user> -n <target_project>

Procedure

1. Navigate in the web console to the Operators → OperatorHub page.

2. Scroll or type a keyword into the Filter by keyword box to find the Operator you want. For example, type advanced to find the Advanced Cluster Management for Kubernetes Operator.

   You can also filter options by Infrastructure Features. For example, select Disconnected if you want to see Operators that work in disconnected environments, also known as restricted network environments.

3. Select the Operator to display additional information.

   Note

   Choosing a Community Operator warns that Red Hat does not certify Community Operators; you must acknowledge the warning before continuing.

4. Read the information about the Operator and click Install.

5. On the Install Operator page:

   1. Choose a specific, single namespace in which to install the Operator. The Operator will only watch and be made available for use in this single namespace.
   2. Select an Update Channel (if more than one is available).
   3. Select Automatic or Manual approval strategy, as described earlier.

6. Click Install to make the Operator available to the selected namespaces on this OpenShift Container Platform cluster.

   1. If you selected a Manual approval strategy, the upgrade status of the subscription remains Upgrading until you review and approve the install plan.

      After approving on the Install Plan page, the subscription upgrade status moves to Up to date.

   2. If you selected an Automatic approval strategy, the upgrade status should resolve to Up to date without intervention.

7. After the upgrade status of the subscription is Up to date, select Operators → Installed Operators to verify that the cluster service version (CSV) of the installed Operator eventually shows up. The Status should ultimately resolve to InstallSucceeded in the relevant namespace.

   Note

   For the All namespaces…​ installation mode, the status resolves to InstallSucceeded in the openshift-operators namespace, but the status is Copied if you check in other namespaces.

   If it does not:

   1. Check the logs in any pods in the openshift-operators project (or other relevant namespace if A specific namespace…​ installation mode was selected) on the Workloads → Pods page that are reporting issues to troubleshoot further.

Procedure

1. View the list of Operators available to the cluster from OperatorHub:

   $ oc get packagemanifests -n openshift-marketplace

   Example output

   NAME                               CATALOG               AGE
   3scale-operator                    Red Hat Operators     91m
   advanced-cluster-management        Red Hat Operators     91m
   amq7-cert-manager                  Red Hat Operators     91m
   ...
   couchbase-enterprise-certified     Certified Operators   91m
   crunchy-postgres-operator          Certified Operators   91m
   mongodb-enterprise                 Certified Operators   91m
   ...
   etcd                               Community Operators   91m
   jaeger                             Community Operators   91m
   kubefed                            Community Operators   91m
   ...

   Note the catalog for your desired Operator.

2. Inspect your desired Operator to verify its supported install modes and available channels:

   $ oc describe packagemanifests <operator_name> -n openshift-marketplace

3. An Operator group, defined by an OperatorGroup object, selects target namespaces in which to generate required RBAC access for all Operators in the same namespace as the Operator group.

   The namespace to which you subscribe the Operator must have an Operator group that matches the install mode of the Operator, either the AllNamespaces or SingleNamespace mode. If the Operator you intend to install uses the AllNamespaces, then the openshift-operators namespace already has an appropriate Operator group in place.

   However, if the Operator uses the SingleNamespace mode and you do not already have an appropriate Operator group in place, you must create one.

   Note

   The web console version of this procedure handles the creation of the OperatorGroup and Subscription objects automatically behind the scenes for you when choosing SingleNamespace mode.

   1. Create an OperatorGroup object YAML file, for example operatorgroup.yaml:

      Example OperatorGroup object

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: <operatorgroup_name>
        namespace: <namespace>
      spec:
        targetNamespaces:
        - <namespace>

   2. Create the OperatorGroup object:

      $ oc apply -f operatorgroup.yaml

4. Create a Subscription object YAML file to subscribe a namespace to an Operator, for example sub.yaml:

   Example Subscription object

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: <subscription_name>
     namespace: openshift-operators 1
   spec:
     channel: <channel_name> 2
     name: <operator_name> 3
     source: redhat-operators 4
     sourceNamespace: openshift-marketplace 5

   1

   For AllNamespaces install mode usage, specify the openshift-operators namespace. Otherwise, specify the relevant single namespace for SingleNamespace install mode usage.

   2

   Name of the channel to subscribe to.

   3

   Name of the Operator to subscribe to.

   4

   Name of the catalog source that provides the Operator.

   5

   Namespace of the catalog source. Use openshift-marketplace for the default OperatorHub catalog sources.

5. Create the Subscription object:

   $ oc apply -f sub.yaml

   At this point, OLM is now aware of the selected Operator. A cluster service version (CSV) for the Operator should appear in the target namespace, and APIs provided by the Operator should be available for creation.

Procedure

1. Create a Subscription object YAML file that subscribes a namespace to an Operator with a specific version by setting the startingCSV field. Set the installPlanApproval field to Manual to prevent the Operator from automatically upgrading if a later version exists in the catalog.

   For example, the following sub.yaml file can be used to install the Red Hat Quay Operator specifically to version 3.4.0:

   Subscription with a specific starting Operator version

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: quay-operator
     namespace: quay
   spec:
     channel: quay-v3.4
     installPlanApproval: Manual 1
     name: quay-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     startingCSV: quay-operator.v3.4.0 2

   1

   Set the approval strategy to Manual in case your specified version is superseded by a later version in the catalog. This plan prevents an automatic upgrade to a later version and requires manual approval before the starting CSV can complete the installation.

   2

   Set a specific version of an Operator CSV.

2. Create the Subscription object:

   $ oc apply -f sub.yaml

3. Manually approve the pending install plan to complete the Operator installation.

Procedure

1. Navigate in the web console to the Operators → OperatorHub page.

2. Scroll or type a keyword into the Filter by keyword box to find the Operator you want. For example, type advanced to find the Advanced Cluster Management for Kubernetes Operator.

   You can also filter options by Infrastructure Features. For example, select Disconnected if you want to see Operators that work in disconnected environments, also known as restricted network environments.

3. Select the Operator to display additional information.

   Note

   Choosing a Community Operator warns that Red Hat does not certify Community Operators; you must acknowledge the warning before continuing.

4. Read the information about the Operator and click Install.

5. On the Install Operator page:

   1. Select one of the following:

      • All namespaces on the cluster (default) installs the Operator in the default openshift-operators namespace to watch and be made available to all namespaces in the cluster. This option is not always available.
      • A specific namespace on the cluster allows you to choose a specific, single namespace in which to install the Operator. The Operator will only watch and be made available for use in this single namespace.
   2. Choose a specific, single namespace in which to install the Operator. The Operator will only watch and be made available for use in this single namespace.
   3. Select an Update Channel (if more than one is available).
   4. Select Automatic or Manual approval strategy, as described earlier.

6. Click Install to make the Operator available to the selected namespaces on this OpenShift Container Platform cluster.

   1. If you selected a Manual approval strategy, the upgrade status of the subscription remains Upgrading until you review and approve the install plan.

      After approving on the Install Plan page, the subscription upgrade status moves to Up to date.

   2. If you selected an Automatic approval strategy, the upgrade status should resolve to Up to date without intervention.

7. After the upgrade status of the subscription is Up to date, select Operators → Installed Operators to verify that the cluster service version (CSV) of the installed Operator eventually shows up. The Status should ultimately resolve to InstallSucceeded in the relevant namespace.

   Note

   For the All namespaces…​ installation mode, the status resolves to InstallSucceeded in the openshift-operators namespace, but the status is Copied if you check in other namespaces.

   If it does not:

   1. Check the logs in any pods in the openshift-operators project (or other relevant namespace if A specific namespace…​ installation mode was selected) on the Workloads → Pods page that are reporting issues to troubleshoot further.

Procedure

1. View the list of Operators available to the cluster from OperatorHub:

   $ oc get packagemanifests -n openshift-marketplace

   Example output

   NAME                               CATALOG               AGE
   3scale-operator                    Red Hat Operators     91m
   advanced-cluster-management        Red Hat Operators     91m
   amq7-cert-manager                  Red Hat Operators     91m
   ...
   couchbase-enterprise-certified     Certified Operators   91m
   crunchy-postgres-operator          Certified Operators   91m
   mongodb-enterprise                 Certified Operators   91m
   ...
   etcd                               Community Operators   91m
   jaeger                             Community Operators   91m
   kubefed                            Community Operators   91m
   ...

   Note the catalog for your desired Operator.

2. Inspect your desired Operator to verify its supported install modes and available channels:

   $ oc describe packagemanifests <operator_name> -n openshift-marketplace

3. An Operator group, defined by an OperatorGroup object, selects target namespaces in which to generate required RBAC access for all Operators in the same namespace as the Operator group.

   The namespace to which you subscribe the Operator must have an Operator group that matches the install mode of the Operator, either the AllNamespaces or SingleNamespace mode. If the Operator you intend to install uses the AllNamespaces, then the openshift-operators namespace already has an appropriate Operator group in place.

   However, if the Operator uses the SingleNamespace mode and you do not already have an appropriate Operator group in place, you must create one.

   Note

   The web console version of this procedure handles the creation of the OperatorGroup and Subscription objects automatically behind the scenes for you when choosing SingleNamespace mode.

   1. Create an OperatorGroup object YAML file, for example operatorgroup.yaml:

      Example OperatorGroup object

      apiVersion: operators.coreos.com/v1
      kind: OperatorGroup
      metadata:
        name: <operatorgroup_name>
        namespace: <namespace>
      spec:
        targetNamespaces:
        - <namespace>

   2. Create the OperatorGroup object:

      $ oc apply -f operatorgroup.yaml

4. Create a Subscription object YAML file to subscribe a namespace to an Operator, for example sub.yaml:

   Example Subscription object

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: <subscription_name>
     namespace: openshift-operators 1
   spec:
     channel: <channel_name> 2
     name: <operator_name> 3
     source: redhat-operators 4
     sourceNamespace: openshift-marketplace 5

   1

   For AllNamespaces install mode usage, specify the openshift-operators namespace. Otherwise, specify the relevant single namespace for SingleNamespace install mode usage.

   2

   Name of the channel to subscribe to.

   3

   Name of the Operator to subscribe to.

   4

   Name of the catalog source that provides the Operator.

   5

   Namespace of the catalog source. Use openshift-marketplace for the default OperatorHub catalog sources.

5. Create the Subscription object:

   $ oc apply -f sub.yaml

   At this point, OLM is now aware of the selected Operator. A cluster service version (CSV) for the Operator should appear in the target namespace, and APIs provided by the Operator should be available for creation.

Procedure

1. Create a Subscription object YAML file that subscribes a namespace to an Operator with a specific version by setting the startingCSV field. Set the installPlanApproval field to Manual to prevent the Operator from automatically upgrading if a later version exists in the catalog.

   For example, the following sub.yaml file can be used to install the Red Hat Quay Operator specifically to version 3.4.0:

   Subscription with a specific starting Operator version

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: quay-operator
     namespace: quay
   spec:
     channel: quay-v3.4
     installPlanApproval: Manual 1
     name: quay-operator
     source: redhat-operators
     sourceNamespace: openshift-marketplace
     startingCSV: quay-operator.v3.4.0 2

   1

   Set the approval strategy to Manual in case your specified version is superseded by a later version in the catalog. This plan prevents an automatic upgrade to a later version and requires manual approval before the starting CSV can complete the installation.

   2

   Set a specific version of an Operator CSV.

2. Create the Subscription object:

   $ oc apply -f sub.yaml

3. Manually approve the pending install plan to complete the Operator installation.

Procedure

1. In the Administrator perspective of the OpenShift Container Platform web console, navigate to Operators → Installed Operators.
2. Click the name of the Operator you want to change the update channel for.
3. Click the Subscription tab.
4. Click the name of the update channel under Channel.
5. Click the newer update channel that you want to change to, then click Save.

6. For Subscriptions with an Automatic approval strategy, the upgrade begins automatically. Navigate back to the Operators → Installed Operators page to monitor the progress of the upgrade. When complete, the status changes to Succeeded and Up to date.

   For Subscriptions with a Manual approval strategy, you can manually approve the upgrade from the Subscription tab.

Procedure

1. In the Administrator perspective of the OpenShift Container Platform web console, navigate to Operators → Installed Operators.
2. Operators that have a pending upgrade display a status with Upgrade available. Click the name of the Operator you want to upgrade.
3. Click the Subscription tab. Any upgrades requiring approval are displayed next to Upgrade Status. For example, it might display 1 requires approval.
4. Click 1 requires approval, then click Preview Install Plan.
5. Review the resources that are listed as available for upgrade. When satisfied, click Approve.
6. Navigate back to the Operators → Installed Operators page to monitor the progress of the upgrade. When complete, the status changes to Succeeded and Up to date.

Procedure

1. From the Operators → Installed Operators page, scroll or type a keyword into the Filter by name to find the Operator you want. Then, click on it.

2. On the right-hand side of the Operator Details page, select Uninstall Operator from the Actions drop-down menu.

   An Uninstall Operator? dialog box is displayed, reminding you that:

   Removing the Operator will not remove any of its custom resource definitions or managed resources. If your Operator has deployed applications on the cluster or configured off-cluster resources, these will continue to run and need to be cleaned up manually.

   The Operator, any Operator deployments, and pods are removed by this action. Any resources managed by the Operator, including CRDs and CRs, are not removed. The web console enables dashboards and navigation items for some Operators. To remove these after uninstalling the Operator, you might need to manually delete the Operator CRDs.

3. Select Uninstall. This Operator stops running and no longer receives updates.

Procedure

1. Check the current version of the subscribed Operator (for example, jaeger) in the currentCSV field:

   $ oc get subscription jaeger -n openshift-operators -o yaml | grep currentCSV

   Example output

     currentCSV: jaeger-operator.v1.8.2

2. Delete the subscription (for example, jaeger):

   $ oc delete subscription jaeger -n openshift-operators

   Example output

   subscription.operators.coreos.com "jaeger" deleted

3. Delete the CSV for the Operator in the target namespace using the currentCSV value from the previous step:

   $ oc delete clusterserviceversion jaeger-operator.v1.8.2 -n openshift-operators

   Example output

   clusterserviceversion.operators.coreos.com "jaeger-operator.v1.8.2" deleted

Procedure

1. Navigate in the web console to the Operators → OperatorHub page.
2. Select the Operator and click Install.

3. On the Install Operator page, modify the Subscription object to include one or more of the following environment variables in the spec section:

   • HTTP_PROXY
   • HTTPS_PROXY
   • NO_PROXY

   For example:

   Subscription object with proxy setting overrides

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: etcd-config-test
     namespace: openshift-operators
   spec:
     config:
       env:
       - name: HTTP_PROXY
         value: test_http
       - name: HTTPS_PROXY
         value: test_https
       - name: NO_PROXY
         value: test
     channel: clusterwide-alpha
     installPlanApproval: Automatic
     name: etcd
     source: community-operators
     sourceNamespace: openshift-marketplace
     startingCSV: etcdoperator.v0.9.4-clusterwide

   Note

   These environment variables can also be unset using an empty value to remove any previously set cluster-wide or custom proxy settings.

   OLM handles these environment variables as a unit; if at least one of them is set, all three are considered overridden and the cluster-wide defaults are not used for the deployments of the subscribed Operator.

4. Click Install to make the Operator available to the selected namespaces.

5. After the CSV for the Operator appears in the relevant namespace, you can verify that custom proxy environment variables are set in the deployment. For example, using the CLI:

   $ oc get deployment -n openshift-operators \
       etcd-operator -o yaml \
       | grep -i "PROXY" -A 2

   Example output

           - name: HTTP_PROXY
             value: test_http
           - name: HTTPS_PROXY
             value: test_https
           - name: NO_PROXY
             value: test
           image: quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21088a98b93838e284a6086b13917f96b0d9c
   ...

Procedure

1. Create an empty config map in the namespace where the subscription for your Operator exists and include the following label:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: trusted-ca 1
     labels:
       config.openshift.io/inject-trusted-cabundle: "true" 2

   1

   Name of the config map.

   2

   Requests the Cluster Network Operator to inject the merged bundle.

   After creating this config map, it is immediately populated with the certificate contents of the merged bundle.

2. Update your the Subscription object to include a spec.config section that mounts the trusted-ca config map as a volume to each container within a pod that requires a custom CA:

   kind: Subscription
   metadata:
     name: my-operator
   spec:
     package: etcd
     channel: alpha
     config: 1
       selector:
         matchLabels:
           <labels_for_pods> 2
       volumes: 3
       - name: trusted-ca
         configMap:
           name: trusted-ca
           items:
             - key: ca-bundle.crt 4
               path: tls-ca-bundle.pem 5
       volumeMounts: 6
       - name: trusted-ca
         mountPath: /etc/pki/ca-trust/extracted/pem
         readOnly: true

   1

   Add a config section if it does not exist.

   2

   Specify labels to match pods that are owned by the Operator.

   3

   Create a trusted-ca volume.

   4

   ca-bundle.crt is required as the config map key.

   5

   tls-ca-bundle.pem is required as the config map path.

   6

   Create a trusted-ca volume mount.

Procedure

1. List Operator subscriptions:

   $ oc get subs -n <operator_namespace>

2. Use the oc describe command to inspect a Subscription resource:

   $ oc describe sub <subscription_name> -n <operator_namespace>

3. In the command output, find the Conditions section for the status of Operator subscription condition types. In the following example, the CatalogSourcesUnhealthy condition type has a status of false because all available catalog sources are healthy:

   Example output

   Conditions:
      Last Transition Time:  2019-07-29T13:42:57Z
      Message:               all available catalogsources are healthy
      Reason:                AllCatalogSourcesHealthy
      Status:                False
      Type:                  CatalogSourcesUnhealthy

Procedure

1. Create a new namespace:

   $ cat <<EOF | oc create -f -
   apiVersion: v1
   kind: Namespace
   metadata:
     name: scoped
   EOF

2. Allocate permissions that you want the Operator(s) to be confined to. This involves creating a new service account, relevant role(s), and role binding(s).

   $ cat <<EOF | oc create -f -
   apiVersion: v1
   kind: ServiceAccount
   metadata:
     name: scoped
     namespace: scoped
   EOF

   The following example grants the service account permissions to do anything in the designated namespace for simplicity. In a production environment, you should create a more fine-grained set of permissions:

   $ cat <<EOF | oc create -f -
   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: scoped
     namespace: scoped
   rules:
   - apiGroups: ["*"]
     resources: ["*"]
     verbs: ["*"]
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: scoped-bindings
     namespace: scoped
   roleRef:
     apiGroup: rbac.authorization.k8s.io
     kind: Role
     name: scoped
   subjects:
   - kind: ServiceAccount
     name: scoped
     namespace: scoped
   EOF

3. Create an OperatorGroup object in the designated namespace. This Operator group targets the designated namespace to ensure that its tenancy is confined to it.

   In addition, Operator groups allow a user to specify a service account. Specify the service account created in the previous step:

   $ cat <<EOF | oc create -f -
   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: scoped
     namespace: scoped
   spec:
     serviceAccountName: scoped
     targetNamespaces:
     - scoped
   EOF

   Any Operator installed in the designated namespace is tied to this Operator group and therefore to the service account specified.

4. Create a Subscription object in the designated namespace to install an Operator:

   $ cat <<EOF | oc create -f -
   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: etcd
     namespace: scoped
   spec:
     channel: singlenamespace-alpha
     name: etcd
     source: <catalog_source_name> 1
     sourceNamespace: <catalog_source_namespace> 2
   EOF

   1

   Specify a catalog source that already exists in the designated namespace or one that is in the global catalog namespace.

   2

   Specify a namespace where the catalog source was created.

   Any Operator tied to this Operator group is confined to the permissions granted to the specified service account. If the Operator requests permissions that are outside the scope of the service account, the installation fails with relevant errors.

Procedure

1. Review the Subscription object. Its status has an object reference installPlanRef that points to the InstallPlan object that attempted to create the necessary [Cluster]Role[Binding] object(s) for the Operator:

   apiVersion: operators.coreos.com/v1
   kind: Subscription
   metadata:
     name: etcd
     namespace: scoped
   status:
     installPlanRef:
       apiVersion: operators.coreos.com/v1
       kind: InstallPlan
       name: install-4plp8
       namespace: scoped
       resourceVersion: "117359"
       uid: 2c1df80e-afea-11e9-bce3-5254009c9c23

2. Check the status of the InstallPlan object for any errors:

   apiVersion: operators.coreos.com/v1
   kind: InstallPlan
   status:
     conditions:
     - lastTransitionTime: "2019-07-26T21:13:10Z"
       lastUpdateTime: "2019-07-26T21:13:10Z"
       message: 'error creating clusterrole etcdoperator.v0.9.4-clusterwide-dsfx4: clusterroles.rbac.authorization.k8s.io
         is forbidden: User "system:serviceaccount:scoped:scoped" cannot create resource
         "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope'
       reason: InstallComponentFailed
       status: "False"
       type: Installed
     phase: Failed

   The error message tells you:

   • The type of resource it failed to create, including the API group of the resource. In this case, it was clusterroles in the rbac.authorization.k8s.io group.
   • The name of the resource.
   • The type of error: is forbidden tells you that the user does not have enough permission to do the operation.
   • The name of the user who attempted to create or update the resource. In this case, it refers to the service account specified in the Operator group.

   • The scope of the operation: cluster scope or not.

     The user can add the missing permission to the service account and then iterate.

     Note

     Operator Lifecycle Manager (OLM) does not currently provide the complete list of errors on the first try.

Procedure

1. On the workstation with unrestricted network access, authenticate with the target mirror registry:

   $ podman login <registry_host_name>

   Also authenticate with registry.redhat.io so that the base image can be pulled during the build:

   $ podman login registry.redhat.io

2. Build a catalog image based on the redhat-operators catalog from Quay.io, tagging and pushing it to your mirror registry:

   $ oc adm catalog build \
       --appregistry-org redhat-operators \1
       --from=registry.redhat.io/openshift4/ose-operator-registry:v4.5 \2
       --filter-by-os="linux/amd64" \3
       --to=<registry_host_name>:<port>/olm/redhat-operators:v1 \4
       [-a ${REG_CREDS}] \5
       [--insecure] \6
       [--auth-token "${AUTH_TOKEN}"] 7

   1

   Organization (namespace) to pull from an App Registry instance.

   2

   Set --from to the ose-operator-registry base image using the tag that matches the target OpenShift Container Platform cluster major and minor version.

   3

   Set --filter-by-os to the operating system and architecture to use for the base image, which must match the target OpenShift Container Platform cluster. Valid values are linux/amd64, linux/ppc64le, and linux/s390x.

   4

   Name your catalog image and include a tag, for example, v1.

   5

   Optional: If required, specify the location of your registry credentials file.

   6

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   7

   Optional: If other application registry catalogs are used that are not public, specify a Quay authentication token.

   Example output

   INFO[0013] loading Bundles                               dir=/var/folders/st/9cskxqs53ll3wdn434vw4cd80000gn/T/300666084/manifests-829192605
   ...
   Pushed sha256:f73d42950021f9240389f99ddc5b0c7f1b533c054ba344654ff1edaf6bf827e3 to example_registry:5000/olm/redhat-operators:v1

   Sometimes invalid manifests are accidentally introduced catalogs provided by Red Hat; when this happens, you might see some errors:

   Example output with errors

   ...
   INFO[0014] directory                                     dir=/var/folders/st/9cskxqs53ll3wdn434vw4cd80000gn/T/300666084/manifests-829192605 file=4.2 load=package
   W1114 19:42:37.876180   34665 builder.go:141] error building database: error loading package into db: fuse-camel-k-operator.v7.5.0 specifies replacement that couldn't be found
   Uploading ... 244.9kB/s

   These errors are usually non-fatal, and if the Operator package mentioned does not contain an Operator you plan to install or a dependency of one, then they can be ignored.

Procedure

1. Disable the default OperatorSource objects by adding disableAllDefaultSources: true to the spec:

   $ oc patch OperatorHub cluster --type json \
       -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'

   This disables the default sources that are configured by default during an OpenShift Container Platform installation.

2. The oc adm catalog mirror command extracts the contents of your custom Operator catalog image to generate the manifests required for mirroring. You can choose to either:

   • Allow the default behavior of the command to automatically mirror all of the image content to your mirror registry after generating manifests, or
   • Add the --manifests-only flag to only generate the manifests required for mirroring, but do not actually mirror the image content to a registry yet. This can be useful for reviewing what will be mirrored, and it allows you to make any changes to the mapping list if you only require a subset of the content. You can then use that file with the oc image mirror command to mirror the modified list of images in a later step.

   On your workstation with unrestricted network access, run the following command:

   $ oc adm catalog mirror \
       <registry_host_name>:<port>/olm/redhat-operators:v1 \1
       <registry_host_name>:<port> \
       [-a ${REG_CREDS}] \2
       [--insecure] \3
       --filter-by-os='.*' \4
       [--manifests-only] 5

   1

   Specify your Operator catalog image.

   2

   Optional: If required, specify the location of your registry credentials file.

   3

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   4

   This flag is currently required due to a known issue with multiple architecture support.

   5

   Optional: Only generate the manifests required for mirroring and do not actually mirror the image content to a registry.

   Warning

   If the --filter-by-os flag remains unset or set to any value other than .*, the command filters out different architectures, which changes the digest of the manifest list, also known as a multi-arch image. The incorrect digest causes deployments of those images and Operators on disconnected clusters to fail. For more information, see BZ#1890951.

   Example output

   using database path mapping: /:/tmp/190214037
   wrote database to /tmp/190214037
   using database at: /tmp/190214037/bundles.db 1
   ...

   1

   Temporary database generated by the command.

   After running the command, a <image_name>-manifests/ directory is created in the current directory and generates the following files:

   • The imageContentSourcePolicy.yaml file defines an ImageContentSourcePolicy object that can configure nodes to translate between the image references stored in Operator manifests and the mirrored registry.
   • The mapping.txt file contains all of the source images and where to map them in the target registry. This file is compatible with the oc image mirror command and can be used to further customize the mirroring configuration.

3. If you used the --manifests-only flag in the previous step and want to mirror only a subset of the content:

   1. Modify the list of images in your mapping.txt file to your specifications. If you are unsure of the exact names and versions of the subset of images you want to mirror, use the following steps to find them:

      1. Run the sqlite3 tool against the temporary database that was generated by the oc adm catalog mirror command to retrieve a list of images matching a general search query. The output helps inform how you will later edit your mapping.txt file.

         For example, to retrieve a list of images that are similar to the string clusterlogging.4.3:

         $ echo "select * from related_image \
             where operatorbundle_name like 'clusterlogging.4.3%';" \
             | sqlite3 -line /tmp/190214037/bundles.db 1

         1

         Refer to the previous output of the oc adm catalog mirror command to find the path of the database file.

         Example output

         image = registry.redhat.io/openshift4/ose-logging-kibana5@sha256:aa4a8b2a00836d0e28aa6497ad90a3c116f135f382d8211e3c55f34fb36dfe61
         operatorbundle_name = clusterlogging.4.3.33-202008111029.p0
         
         image = registry.redhat.io/openshift4/ose-oauth-proxy@sha256:6b4db07f6e6c962fc96473d86c44532c93b146bbefe311d0c348117bf759c506
         operatorbundle_name = clusterlogging.4.3.33-202008111029.p0
         ...

      2. Use the results from the previous step to edit the mapping.txt file to only include the subset of images you want to mirror.

         For example, you can use the image values from the previous example output to find that the following matching lines exist in your mapping.txt file:

         Matching image mappings in mapping.txt

         registry.redhat.io/openshift4/ose-logging-kibana5@sha256:aa4a8b2a00836d0e28aa6497ad90a3c116f135f382d8211e3c55f34fb36dfe61=<registry_host_name>:<port>/openshift4-ose-logging-kibana5:a767c8f0
         registry.redhat.io/openshift4/ose-oauth-proxy@sha256:6b4db07f6e6c962fc96473d86c44532c93b146bbefe311d0c348117bf759c506=<registry_host_name>:<port>/openshift4-ose-oauth-proxy:3754ea2b

         In this example, if you only want to mirror these images, you would then remove all other entries in the mapping.txt file and leave only the above two lines.

   2. Still on your workstation with unrestricted network access, use your modified mapping.txt file to mirror the images to your registry using the oc image mirror command:

      $ oc image mirror \
          [-a ${REG_CREDS}] \
          --filter-by-os='.*' \
          -f ./redhat-operators-manifests/mapping.txt

      Warning

      If the --filter-by-os flag remains unset or set to any value other than .*, the command filters out different architectures, which changes the digest of the manifest list, also known as a multi-arch image. The incorrect digest causes deployments of those images and Operators on disconnected clusters to fail.

4. Apply the ImageContentSourcePolicy object:

   $ oc apply -f ./redhat-operators-manifests/imageContentSourcePolicy.yaml

5. Create a CatalogSource object that references your catalog image.

   1. Modify the following to your specifications and save it as a catalogsource.yaml file:

      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      metadata:
        name: my-operator-catalog
        namespace: openshift-marketplace
      spec:
        sourceType: grpc
        image: <registry_host_name>:<port>/olm/redhat-operators:v1 1
        displayName: My Operator Catalog
        publisher: grpc

      1

      Specify your custom Operator catalog image.

   2. Use the file to create the CatalogSource object:

      $ oc create -f catalogsource.yaml

6. Verify the following resources are created successfully.

   1. Check the pods:

      $ oc get pods -n openshift-marketplace

      Example output

      NAME                                    READY   STATUS    RESTARTS  AGE
      my-operator-catalog-6njx6               1/1     Running   0         28s
      marketplace-operator-d9f549946-96sgr    1/1     Running   0         26h

   2. Check the catalog source:

      $ oc get catalogsource -n openshift-marketplace

      Example output

      NAME                  DISPLAY               TYPE PUBLISHER  AGE
      my-operator-catalog   My Operator Catalog   grpc            5s

   3. Check the package manifest:

      $ oc get packagemanifest -n openshift-marketplace

      Example output

      NAME    CATALOG              AGE
      etcd    My Operator Catalog  34s

Procedure

1. On the workstation with unrestricted network access, authenticate with the target mirror registry:

   $ podman login <registry_host_name>

   Also authenticate with registry.redhat.io so that the base image can be pulled during the build:

   $ podman login registry.redhat.io

2. Build a new catalog image based on the redhat-operators catalog from Quay.io, tagging and pushing it to your mirror registry:

   $ oc adm catalog build \
       --appregistry-org redhat-operators \1
       --from=registry.redhat.io/openshift4/ose-operator-registry:v4.5 \2
       --filter-by-os="linux/amd64" \3
       --to=<registry_host_name>:<port>/olm/redhat-operators:v2 \4
       [-a ${REG_CREDS}] \5
       [--insecure] \6
       [--auth-token "${AUTH_TOKEN}"] 7

   1

   Organization (namespace) to pull from an App Registry instance.

   2

   Set --from to the ose-operator-registry base image using the tag that matches the target OpenShift Container Platform cluster major and minor version.

   3

   Set --filter-by-os to the operating system and architecture to use for the base image, which must match the target OpenShift Container Platform cluster. Valid values are linux/amd64, linux/ppc64le, and linux/s390x.

   4

   Name your catalog image and include a tag, for example, v2 because it is the updated catalog.

   5

   Optional: If required, specify the location of your registry credentials file.

   6

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   7

   Optional: If other application registry catalogs are used that are not public, specify a Quay authentication token.

   Example output

   INFO[0013] loading Bundles                               dir=/var/folders/st/9cskxqs53ll3wdn434vw4cd80000gn/T/300666084/manifests-829192605
   ...
   Pushed sha256:f73d42950021f9240389f99ddc5b0c7f1b533c054ba344654ff1edaf6bf827e3 to example_registry:5000/olm/redhat-operators:v2

3. Mirror the contents of your catalog to your target registry. The following oc adm catalog mirror command extracts the contents of your custom Operator catalog image to generate the manifests required for mirroring and mirrors the images to your registry:

   $ oc adm catalog mirror \
       <registry_host_name>:<port>/olm/redhat-operators:v2 \1
       <registry_host_name>:<port> \
       [-a ${REG_CREDS}] \2
       [--insecure] \3
       --filter-by-os='.*' 4

   1

   Specify your new Operator catalog image.

   2

   Optional: If required, specify the location of your registry credentials file.

   3

   Optional: If you do not want to configure trust for the target registry, add the --insecure flag.

   4

   This flag is currently required due to a known issue with multiple architecture support. If the --filter-by-os flag remains unset or set to any value other than .*, the command filters out different architectures, which changes the digest of the manifest list, also known as a multi-arch image. The incorrect digest causes deployments of those images and Operators on disconnected clusters to fail. For more information, see BZ#1890951.

4. Apply the newly generated manifests:

   $ oc apply -f ./redhat-operators-manifests

   Important

   It is possible that you do not need to apply the imageContentSourcePolicy.yaml manifest. Complete a diff of the files to determine if changes are necessary.

5. Update your CatalogSource object that references your catalog image.

   1. If you have your original catalogsource.yaml file for this CatalogSource object:

      1. Edit your catalogsource.yaml file to reference your new catalog image in the spec.image field:

         apiVersion: operators.coreos.com/v1alpha1
         kind: CatalogSource
         metadata:
           name: my-operator-catalog
           namespace: openshift-marketplace
         spec:
           sourceType: grpc
           image: <registry_host_name>:<port>/olm/redhat-operators:v2 1
           displayName: My Operator Catalog
           publisher: grpc

         1

         Specify your new Operator catalog image.

      2. Use the updated file to replace the CatalogSource object:

         $ oc replace -f catalogsource.yaml

   2. Alternatively, edit the catalog source using the following command and reference your new catalog image in the spec.image parameter:

      $ oc edit catalogsource <catalog_source_name> -n openshift-marketplace

Procedure

1. Pull the Operator catalog image:

   $ podman pull <registry_host_name>:<port>/olm/redhat-operators:v1

2. Run the image:

   $ podman run -p 50051:50051 \
       -it <registry_host_name>:<port>/olm/redhat-operators:v1

3. Query the running image for available packages using grpcurl:

   $ grpcurl -plaintext localhost:50051 api.Registry/ListPackages

   Example output

   {
     "name": "3scale-operator"
   }
   {
     "name": "amq-broker"
   }
   {
     "name": "amq-online"
   }

4. Get the latest Operator bundle in a channel:

   $  grpcurl -plaintext -d '{"pkgName":"kiali-ossm","channelName":"stable"}' localhost:50051 api.Registry/GetBundleForChannel

   Example output

   {
     "csvName": "kiali-operator.v1.0.7",
     "packageName": "kiali-ossm",
     "channelName": "stable",
   ...

5. Get the digest of the image:

   $ podman inspect \
       --format='{{index .RepoDigests 0}}' \
       <registry_host_name>:<port>/olm/redhat-operators:v1

   Example output

   example_registry:5000/olm/redhat-operators@sha256:f73d42950021f9240389f99ddc5b0c7f1b533c054ba344654ff1edaf6bf827e3

6. Assuming an Operator group exists in namespace my-ns that supports your Operator and its dependencies, create a CatalogSource object using the image digest. For example:

   apiVersion: operators.coreos.com/v1alpha1
   kind: CatalogSource
   metadata:
     name: custom-redhat-operators
     namespace: my-ns
   spec:
     sourceType: grpc
     image: example_registry:5000/olm/redhat-operators@sha256:f73d42950021f9240389f99ddc5b0c7f1b533c054ba344654ff1edaf6bf827e3
     displayName: Red Hat Operators

7. Create a subscription that resolves the latest available servicemeshoperator and its dependencies from your catalog image:

   apiVersion: operators.coreos.com/v1alpha1
   kind: Subscription
   metadata:
     name: servicemeshoperator
     namespace: my-ns
   spec:
     source: custom-redhat-operators
     sourceNamespace: my-ns
     name: servicemeshoperator
     channel: "1.0"

Procedure

1. Create a new project.

   Use the CLI to create a new memcached-operator project:

   $ mkdir -p $GOPATH/src/github.com/example-inc/

   $ cd $GOPATH/src/github.com/example-inc/

   $ operator-sdk new memcached-operator

   $ cd memcached-operator

2. Add a new custom resource definition (CRD).

   1. Use the CLI to add a new CRD API called Memcached, with APIVersion set to cache.example.com/v1apha1 and Kind set to Memcached:

      $ operator-sdk add api \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds the Memcached resource API under pkg/apis/cache/v1alpha1/.

   2. Modify the spec and status of the Memcached custom resource (CR) at the pkg/apis/cache/v1alpha1/memcached_types.go file:

      type MemcachedSpec struct {
      	// Size is the size of the memcached deployment
      	Size int32 `json:"size"`
      }
      type MemcachedStatus struct {
      	// Nodes are the names of the memcached pods
      	Nodes []string `json:"nodes"`
      }

   3. After modifying the *_types.go file, always run the following command to update the generated code for that resource type:

      $ operator-sdk generate k8s

3. Optional: Add custom validation to your CRD.

   OpenAPI v3.0 schemas are added to CRD manifests in the spec.validation block when the manifests are generated. This validation block allows Kubernetes to validate the properties in a Memcached CR when it is created or updated.

   Additionally, a pkg/apis/<group>/<version>/zz_generated.openapi.go file is generated. This file contains the Go representation of this validation block if the +k8s:openapi-gen=true annotation is present above the Kind type declaration, which is present by default. This auto-generated code is the OpenAPI model of your Go Kind type, from which you can create a full OpenAPI Specification and generate a client.

   As an Operator author, you can use Kubebuilder markers (annotations) to configure custom validations for your API. These markers must always have a +kubebuilder:validation prefix. For example, adding an enum-type specification can be done by adding the following marker:

   // +kubebuilder:validation:Enum=Lion;Wolf;Dragon
   type Alias string

   Usage of markers in API code is discussed in the Kubebuilder Generating CRDs and Markers for Config/Code Generation documentation. A full list of OpenAPIv3 validation markers is also available in the Kubebuilder CRD Validation documentation.

   If you add any custom validations, run the following command to update the OpenAPI validation section in the deploy/crds/cache.example.com_memcacheds_crd.yaml file for the CRD:

   $ operator-sdk generate crds

   Example generated YAML

   spec:
     validation:
       openAPIV3Schema:
         properties:
           spec:
             properties:
               size:
                 format: int32
                 type: integer

4. Add a new controller.

   1. Add a new controller to the project to watch and reconcile the Memcached resource:

      $ operator-sdk add controller \
          --api-version=cache.example.com/v1alpha1 \
          --kind=Memcached

      This scaffolds a new controller implementation under pkg/controller/memcached/.

   2. For this example, replace the generated controller file pkg/controller/memcached/memcached_controller.go with the example implementation.

      The example controller executes the following reconciliation logic for each Memcached resource:

      • Create a Memcached deployment if it does not exist.
      • Ensure that the Deployment size is the same as specified by the Memcached CR spec.
      • Update the Memcached resource status with the names of the Memcached pods.

      The next two sub-steps inspect how the controller watches resources and how the reconcile loop is triggered. You can skip these steps to go directly to building and running the Operator.

   3. Inspect the controller implementation at the pkg/controller/memcached/memcached_controller.go file to see how the controller watches resources.

      The first watch is for the Memcached type as the primary resource. For each add, update, or delete event, the reconcile loop is sent a reconcile Request (a <namespace>:<name> key) for that Memcached object:

      err := c.Watch(
        &source.Kind{Type: &cachev1alpha1.Memcached{}}, &handler.EnqueueRequestForObject{})

      The next watch is for Deployment objects, but the event handler maps each event to a reconcile Request for the owner of the deployment. In this case, this is the Memcached object for which the deployment was created. This allows the controller to watch deployments as a secondary resource:

      err := c.Watch(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{
      		IsController: true,
      		OwnerType:    &cachev1alpha1.Memcached{},
      	})

   4. Every controller has a Reconciler object with a Reconcile() method that implements the reconcile loop. The reconcile loop is passed the Request argument which is a <namespace>:<name> key used to lookup the primary resource object, Memcached, from the cache:

      func (r *ReconcileMemcached) Reconcile(request reconcile.Request) (reconcile.Result, error) {
        // Lookup the Memcached instance for this reconcile request
        memcached := &cachev1alpha1.Memcached{}
        err := r.client.Get(context.TODO(), request.NamespacedName, memcached)
        ...
      }

      Based on the return value of the Reconcile() function, the reconcile Request might be requeued, and the loop might be triggered again:

      // Reconcile successful - don't requeue
      return reconcile.Result{}, nil
      // Reconcile failed due to error - requeue
      return reconcile.Result{}, err
      // Requeue for any reason other than error
      return reconcile.Result{Requeue: true}, nil

5. Build and run the Operator.

   1. Before running the Operator, the CRD must be registered with the Kubernetes API server:

      $ oc create \
          -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   2. After registering the CRD, there are two options for running the Operator:

      • As a Deployment inside a Kubernetes cluster
      • As Go program outside a cluster

      Choose one of the following methods.

      1. Option A: Running as a deployment inside the cluster.

         1. Build the memcached-operator image and push it to a registry:

            $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

         2. The deployment manifest is generated at deploy/operator.yaml. Update the deployment image as follows since the default is just a placeholder:

            $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

         3. Ensure you have an account on Quay.io for the next step, or substitute your preferred container registry. On the registry, create a new public image repository named memcached-operator.

         4. Push the image to the registry:

            $ podman push quay.io/example/memcached-operator:v0.0.1

         5. Set up RBAC and create the memcached-operator manifests:

            $ oc create -f deploy/role.yaml

            $ oc create -f deploy/role_binding.yaml

            $ oc create -f deploy/service_account.yaml

            $ oc create -f deploy/operator.yaml

         6. Verify that the memcached-operator deploy is up and running:

            $ oc get deployment

            Example output

            NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
            memcached-operator       1         1         1            1           1m

      2. Option B: Running locally outside the cluster.

         This method is preferred during development cycle to deploy and test faster.

         Run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local --namespace=default

         You can use a specific kubeconfig using the flag --kubeconfig=<path/to/kubeconfig>.

6. Verify that the Operator can deploy a Memcached application by creating a Memcached CR.

   1. Create the example Memcached CR that was generated at deploy/crds/cache_v1alpha1_memcached_cr.yaml.

   2. View the file:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3

   3. Create the object:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   4. Ensure that memcached-operator creates the deployment for the CR:

      $ oc get deployment

      Example output

      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   5. Check the pods and CR to confirm the CR status is updated with the pod names:

      $ oc get pods

      Example output

      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

      $ oc get memcached/example-memcached -o yaml

      Example output

      apiVersion: cache.example.com/v1alpha1
      kind: Memcached
      metadata:
        clusterName: ""
        creationTimestamp: 2018-03-31T22:51:08Z
        generation: 0
        name: example-memcached
        namespace: default
        resourceVersion: "245453"
        selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/example-memcached
        uid: 0026cc97-3536-11e8-bd83-0800274106a1
      spec:
        size: 3
      status:
        nodes:
        - example-memcached-6fd7c98d8-7dqdr
        - example-memcached-6fd7c98d8-g5k7v
        - example-memcached-6fd7c98d8-m7vn7

7. Verify that the Operator can manage a deployed Memcached application by updating the size of the deployment.

   1. Change the spec.size field in the memcached CR from 3 to 4:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

   2. Apply the change:

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   3. Confirm that the Operator changes the deployment size:

      $ oc get deployment

      Example output

      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/service_account.yaml

Procedure

1. Generate an Operator manifest.

   An Operator manifest describes how to display, create, and manage the application, in this case Memcached, as a whole. It is defined by a ClusterServiceVersion (CSV) object and is required for OLM to function.

   From the memcached-operator/ directory that was created when you built the Memcached Operator, generate the CSV manifest:

   $ operator-sdk generate csv --csv-version 0.0.1

   Note

   See Building a CSV for the Operator Framework for more information on manually defining a manifest file.

2. Create an Operator group that specifies the namespaces that the Operator will target. Create the following Operator group in the namespace where you will create the CSV. In this example, the default namespace is used:

   apiVersion: operators.coreos.com/v1
   kind: OperatorGroup
   metadata:
     name: memcached-operator-group
     namespace: default
   spec:
     targetNamespaces:
     - default

3. Deploy the Operator. Use the files that were generated into the deploy/ directory by the Operator SDK when you built the Memcached Operator.

   1. Edit the generated CSV manifest file by adding displayName fields for each custom resource definition (CRD) kind in the spec.customresourcedefinitions.owned section:

      deploy/olm-catalog/memcached-operator/0.0.1/memcached-operator.v0.0.1.clusterserviceversion.yaml file

      ...
      spec:
        customresourcedefinitions:
          owned:
          - kind: Memcached
            name: memcacheds.cache.example.com
            version: v1alpha1
            description: Memcached is the Schema for the memcacheds API
            displayName: Memcached 1
      ...

      1

      Specify a display name for the CRD.

   2. Apply the CSV manifest to the specified namespace in the cluster:

      $ oc apply -f deploy/olm-catalog/memcached-operator/0.0.1/memcached-operator.v0.0.1.clusterserviceversion.yaml

      When you apply this manifest, the cluster does not immediately update because it does not yet meet the requirements specified in the manifest.

   3. Create the role, role binding, and service account to grant resource permissions to the Operator, and the custom resource definition (CRD) to create the Memcached custom resource that the Operator manages:

      $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

      $ oc create -f deploy/service_account.yaml

      $ oc create -f deploy/role.yaml

      $ oc create -f deploy/role_binding.yaml

      Because OLM creates Operators in a particular namespace when a manifest is applied, administrators can leverage the native Kubernetes RBAC permission model to restrict which users are allowed to install Operators.

4. Create an application instance.

   The Memcached Operator is now running in the default namespace. Users interact with Operators via instances of custom resources; in this case, the resource has the kind Memcached. Native Kubernetes RBAC also applies to custom resources, providing administrators control over who can interact with each Operator.

   Creating instances of Memcached objects in this namespace will now trigger the Memcached Operator to instantiate pods running the memcached server that are managed by the Operator. The more custom resources you create, the more unique Memcached application instances are managed by the Memcached Operator running in this namespace.

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-wordpress"
   spec:
     size: 1
   EOF

   $ cat <<EOF | oc apply -f -
   apiVersion: "cache.example.com/v1alpha1"
   kind: "Memcached"
   metadata:
     name: "memcached-for-drupal"
   spec:
     size: 1
   EOF

   $ oc get Memcached

   Example output

   NAME                      AGE
   memcached-for-drupal      22s
   memcached-for-wordpress   27s

   $ oc get pods

   Example output

   NAME                                       READY     STATUS    RESTARTS   AGE
   memcached-app-operator-66b5777b79-pnsfj    1/1       Running   0          14m
   memcached-for-drupal-5476487c46-qbd66      1/1       Running   0          3s
   memcached-for-wordpress-65b75fd8c9-7b9x7   1/1       Running   0          8s

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Ansible-based, namespace-scoped memcached-operator project and change to the new directory, use the following commands:

   $ operator-sdk new memcached-operator \
       --api-version=cache.example.com/v1alpha1 \
       --kind=Memcached \
       --type=ansible

   $ cd memcached-operator

   This creates the memcached-operator project specifically for watching the Memcached resource with API version example.com/v1apha1 and kind Memcached.

2. Customize the Operator logic.

   For this example, the memcached-operator executes the following reconciliation logic for each Memcached custom resource (CR):

   • Create a memcached deployment if it does not exist.
   • Ensure that the deployment size is the same as specified by the Memcached CR.

   By default, the memcached-operator watches Memcached resource events as shown in the watches.yaml file and executes the Ansible role Memcached:

   - version: v1alpha1
     group: cache.example.com
     kind: Memcached

   You can optionally customize the following logic in the watches.yaml file:

   1. Specifying a role option configures the Operator to use this specified path when launching ansible-runner with an Ansible role. By default, the operator-sdk new command fills in an absolute path to where your role should go:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        role: /opt/ansible/roles/memcached

   2. Specifying a playbook option in the watches.yaml file configures the Operator to use this specified path when launching ansible-runner with an Ansible playbook:

      - version: v1alpha1
        group: cache.example.com
        kind: Memcached
        playbook: /opt/ansible/playbook.yaml

3. Build the Memcached Ansible role.

   Modify the generated Ansible role under the roles/memcached/ directory. This Ansible role controls the logic that is executed when a resource is modified.

   1. Define the Memcached spec.

      Defining the spec for an Ansible-based Operator can be done entirely in Ansible. The Ansible Operator passes all key-value pairs listed in the CR spec field along to Ansible as variables. The names of all variables in the spec field are converted to snake case (lowercase with an underscore) by the Operator before running Ansible. For example, serviceAccount in the spec becomes service_account in Ansible.

      Tip

      You should perform some type validation in Ansible on the variables to ensure that your application is receiving expected input.

      In case the user does not set the spec field, set a default by modifying the roles/memcached/defaults/main.yml file:

      size: 1

   2. Define the Memcached deployment.

      With the Memcached spec now defined, you can define what Ansible is actually executed on resource changes. Because this is an Ansible role, the default behavior executes the tasks in the roles/memcached/tasks/main.yml file.

      The goal is for Ansible to create a deployment if it does not exist, which runs the memcached:1.4.36-alpine image. Ansible 2.7+ supports the k8s Ansible module, which this example leverages to control the deployment definition.

      Modify the roles/memcached/tasks/main.yml to match the following:

      - name: start memcached
        k8s:
          definition:
            kind: Deployment
            apiVersion: apps/v1
            metadata:
              name: '{{ meta.name }}-memcached'
              namespace: '{{ meta.namespace }}'
            spec:
              replicas: "{{size}}"
              selector:
                matchLabels:
                  app: memcached
              template:
                metadata:
                  labels:
                    app: memcached
                spec:
                  containers:
                  - name: memcached
                    command:
                    - memcached
                    - -m=64
                    - -o
                    - modern
                    - -v
                    image: "docker.io/memcached:1.4.36-alpine"
                    ports:
                      - containerPort: 11211

      Note

      This example used the size variable to control the number of replicas of the Memcached deployment. This example sets the default to 1, but any user can create a CR that overwrites the default.

4. Deploy the CRD.

   Before running the Operator, Kubernetes needs to know about the new custom resource definition (CRD) that the Operator will be watching. Deploy the Memcached CRD:

   $ oc create -f deploy/crds/cache.example.com_memcacheds_crd.yaml

5. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the memcached-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/memcached-operator:v0.0.1

         $ podman push quay.io/example/memcached-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/memcached-operator:v0.0.1|g' deploy/operator.yaml

      3. Deploy the memcached-operator manifests:

         $ oc create -f deploy/service_account.yaml

         $ oc create -f deploy/role.yaml

         $ oc create -f deploy/role_binding.yaml

         $ oc create -f deploy/operator.yaml

      4. Verify that the memcached-operator deployment is up and running:

         $ oc get deployment

         NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         memcached-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      Ensure that Ansible Runner and Ansible Runner HTTP Plug-in are installed or else you will see unexpected errors from Ansible Runner when a CR is created.

      It is also important that the role path referenced in the watches.yaml file exists on your machine. Because normally a container is used where the role is put on disk, the role must be manually copied to the configured Ansible roles path (for example /etc/ansible/roles).

      1. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=config

6. Create a Memcached CR.

   1. Modify the deploy/crds/cache_v1alpha1_memcached_cr.yaml file as shown and create a Memcached CR:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 3

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Ensure that the memcached-operator creates the deployment for the CR:

      $ oc get deployment

      Example output

      NAME                     DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      memcached-operator       1         1         1            1           2m
      example-memcached        3         3         3            3           1m

   3. Check the pods to confirm three replicas were created:

      $ oc get pods

      NAME                                  READY     STATUS    RESTARTS   AGE
      example-memcached-6fd7c98d8-7dqdr     1/1       Running   0          1m
      example-memcached-6fd7c98d8-g5k7v     1/1       Running   0          1m
      example-memcached-6fd7c98d8-m7vn7     1/1       Running   0          1m
      memcached-operator-7cc7cfdf86-vvjqk   1/1       Running   0          2m

7. Update the size.

   1. Change the spec.size field in the memcached CR from 3 to 4 and apply the change:

      $ cat deploy/crds/cache_v1alpha1_memcached_cr.yaml

      Example output

      apiVersion: "cache.example.com/v1alpha1"
      kind: "Memcached"
      metadata:
        name: "example-memcached"
      spec:
        size: 4

      $ oc apply -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   2. Confirm that the Operator changes the deployment size:

      $ oc get deployment

      Example output

      NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
      example-memcached    4         4         4            4           5m

8. Clean up the resources:

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_cr.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/service_account.yaml

   $ oc delete -f deploy/crds/cache_v1alpha1_memcached_crd.yaml

Procedure

1. To track CR status manually from your application, update the watches.yaml file with a manageStatus field set to false:

   - version: v1
     group: api.example.com
     kind: Test1
     role: Test1
     manageStatus: false

2. Use the operator_sdk.util.k8s_status Ansible module to update the subresource. For example, to update with key test1 and value test2, operator_sdk.util can be used as shown:

   - operator_sdk.util.k8s_status:
       api_version: app.example.com/v1
       kind: Test1
       name: "{{ meta.name }}"
       namespace: "{{ meta.namespace }}"
       status:
         test1: test2

   Collections can also be declared in the meta/main.yml for the role, which is included for new scaffolded Ansible Operators:

   collections:
     - operator_sdk.util

   Declaring collections in the role meta allows you to invoke the k8s_status module directly:

   k8s_status:
     <snip>
     status:
       test1: test2

Procedure

1. Create a new Operator project. A namespace-scoped Operator watches and manages resources in a single namespace. Namespace-scoped Operators are preferred because of their flexibility. They enable decoupled upgrades, namespace isolation for failures and monitoring, and differing API definitions.

   To create a new Helm-based, namespace-scoped nginx-operator project, use the following command:

   $ operator-sdk new nginx-operator \
     --api-version=example.com/v1alpha1 \
     --kind=Nginx \
     --type=helm

   $ cd nginx-operator

   This creates the nginx-operator project specifically for watching the Nginx resource with API version example.com/v1apha1 and kind Nginx.

2. Customize the Operator logic.

   For this example, the nginx-operator executes the following reconciliation logic for each Nginx custom resource (CR):

   • Create an Nginx deployment if it does not exist.
   • Create an Nginx service if it does not exist.
   • Create an Nginx ingress if it is enabled and does not exist.
   • Ensure that the deployment, service, and optional ingress match the desired configuration (for example, replica count, image, service type) as specified by the Nginx CR.

   By default, the nginx-operator watches Nginx resource events as shown in the watches.yaml file and executes Helm releases using the specified chart:

   - version: v1alpha1
     group: example.com
     kind: Nginx
     chart: /opt/helm/helm-charts/nginx

   1. Review the Nginx Helm chart.

      When a Helm Operator project is created, the Operator SDK creates an example Helm chart that contains a set of templates for a simple Nginx release.

      For this example, templates are available for deployment, service, and ingress resources, along with a NOTES.txt template, which Helm chart developers use to convey helpful information about a release.

      If you are not already familiar with Helm Charts, review the Helm Chart developer documentation.

   2. Understand the Nginx CR spec.

      Helm uses a concept called values to provide customizations to the defaults of a Helm chart, which are defined in the values.yaml file.

      Override these defaults by setting the desired values in the CR spec. You can use the number of replicas as an example:

      1. First, inspect the helm-charts/nginx/values.yaml file to find that the chart has a value called replicaCount and it is set to 1 by default. To have 2 Nginx instances in your deployment, your CR spec must contain replicaCount: 2.

         Update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file to look like the following:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2

      2. Similarly, the default service port is set to 80. To instead use 8080, update the deploy/crds/example.com_v1alpha1_nginx_cr.yaml file again by adding the service port override:

         apiVersion: example.com/v1alpha1
         kind: Nginx
         metadata:
           name: example-nginx
         spec:
           replicaCount: 2
           service:
             port: 8080

         The Helm Operator applies the entire spec as if it was the contents of a values file, just like the helm install -f ./overrides.yaml command works.

3. Deploy the CRD.

   Before running the Operator, Kubernetes must know about the new custom resource definition (CRD) that the Operator will be watching. Deploy the following CRD:

   $ oc create -f deploy/crds/example_v1alpha1_nginx_crd.yaml

4. Build and run the Operator.

   There are two ways to build and run the Operator:

   • As a pod inside a Kubernetes cluster.
   • As a Go program outside the cluster using the operator-sdk up command.

   Choose one of the following methods:

   1. Run as a pod inside a Kubernetes cluster. This is the preferred method for production use.

      1. Build the nginx-operator image and push it to a registry:

         $ operator-sdk build quay.io/example/nginx-operator:v0.0.1

         $ podman push quay.io/example/nginx-operator:v0.0.1

      2. Deployment manifests are generated in the deploy/operator.yaml file. The deployment image in this file needs to be modified from the placeholder REPLACE_IMAGE to the previous built image. To do this, run:

         $ sed -i 's|REPLACE_IMAGE|quay.io/example/nginx-operator:v0.0.1|g' deploy/operator.yaml

      3. Deploy the nginx-operator manifests:

         $ oc create -f deploy/service_account.yaml

         $ oc create -f deploy/role.yaml

         $ oc create -f deploy/role_binding.yaml

         $ oc create -f deploy/operator.yaml

      4. Verify that the nginx-operator deployment is up and running:

         $ oc get deployment

         Example output

         NAME                 DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
         nginx-operator       1         1         1            1           1m

   2. Run outside the cluster. This method is preferred during the development cycle to speed up deployment and testing.

      It is important that the chart path referenced in the watches.yaml file exists on your machine. By default, the watches.yaml file is scaffolded to work with an Operator image built with the operator-sdk build command. When developing and testing your Operator with the operator-sdk run --local command, the SDK looks in your local file system for this path.

      1. Create a symlink at this location to point to the path of your Helm chart:

         $ sudo mkdir -p /opt/helm/helm-charts

         $ sudo ln -s $PWD/helm-charts/nginx /opt/helm/helm-charts/nginx

      2. To run the Operator locally with the default Kubernetes configuration file present at $HOME/.kube/config:

         $ operator-sdk run --local

         To run the Operator locally with a provided Kubernetes configuration file:

         $ operator-sdk run --local --kubeconfig=<path_to_config>

5. Deploy the Nginx CR.

   Apply the Nginx CR that you modified earlier:

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Ensure that the nginx-operator creates the deployment for the CR:

   $ oc get deployment

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        2         2         2            2           1m

   Check the pods to confirm two replicas were created:

   $ oc get pods

   Example output

   NAME                                                      READY     STATUS    RESTARTS   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-fjcr9   1/1       Running   0          1m
   example-nginx-b9phnoz9spckcrua7ihrbkrt1-f8f9c875d-ljbzl   1/1       Running   0          1m

   Check that the service port is set to 8080:

   $ oc get service

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        8080/TCP   1m

6. Update the replicaCount and remove the port.

   Change the spec.replicaCount field from 2 to 3, remove the spec.service field, and apply the change:

   $ cat deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Example output

   apiVersion: "example.com/v1alpha1"
   kind: "Nginx"
   metadata:
     name: "example-nginx"
   spec:
     replicaCount: 3

   $ oc apply -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   Confirm that the Operator changes the deployment size:

   $ oc get deployment

   Example output

   NAME                                           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1        3         3         3            3           1m

   Check that the service port is set to the default 80:

   $ oc get service

   Example output

   NAME                                      TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)  AGE
   example-nginx-b9phnoz9spckcrua7ihrbkrt1   ClusterIP   10.96.26.3   <none>        80/TCP   1m

7. Clean up the resources:

   $ oc delete -f deploy/crds/example.com_v1alpha1_nginx_cr.yaml

   $ oc delete -f deploy/operator.yaml

   $ oc delete -f deploy/role_binding.yaml

   $ oc delete -f deploy/role.yaml

   $ oc delete -f deploy/service_account.yaml

   $ oc delete -f deploy/crds/example_v1alpha1_nginx_crd.yaml