Chapter 6. Cluster Operators reference

Purpose

The Cluster Baremetal Operator (CBO) deploys all the components necessary to take a bare-metal server to a fully functioning worker node ready to run OpenShift Container Platform compute nodes. The CBO ensures that the metal3 deployment, which consists of the Bare Metal Operator (BMO) and Ironic containers, runs on one of the control plane nodes within the OpenShift Container Platform cluster. The CBO also listens for OpenShift Container Platform updates to resources that it watches and takes appropriate action.

Project

cluster-baremetal-operator

Purpose

The OpenShift Bare Metal Event Relay manages the life-cycle of the Bare Metal Event Relay. The Bare Metal Event Relay enables you to configure the types of cluster event that are monitored using Redfish hardware events.

Configuration objects

You can use this command to edit the configuration after installation: for example, the webhook port. You can edit configuration objects with:

$ oc -n [namespace] edit cm hw-event-proxy-operator-manager-config

apiVersion: controller-runtime.sigs.k8s.io/v1alpha1
kind: ControllerManagerConfig
health:
  healthProbeBindAddress: :8081
metrics:
  bindAddress: 127.0.0.1:8080
webhook:
  port: 9443
leaderElection:
  leaderElect: true
  resourceName: 6e7a703c.redhat-cne.org

Project

hw-event-proxy-operator

CRD

The proxy enables applications running on bare-metal clusters to respond quickly to Redfish hardware changes and failures such as breaches of temperature thresholds, fan failure, disk loss, power outages, and memory failure, reported using the HardwareEvent CR.

hardwareevents.event.redhat-cne.org

Additional resources

Purpose

The Cloud Credential Operator (CCO) manages cloud provider credentials as Kubernetes custom resource definitions (CRDs). The CCO syncs on

CredentialsRequest

By setting different values for the

credentialsMode

install-config.yaml

credentialsMode

""

Project

openshift-cloud-credential-operator

CRDs

Configuration objects

No configuration required.

Additional resources

Purpose

The Cluster Authentication Operator installs and maintains the

Authentication

$ oc get clusteroperator authentication -o yaml

Project

cluster-authentication-operator

Purpose

The Cluster Autoscaler Operator manages deployments of the OpenShift Cluster Autoscaler using the

cluster-api

Project

cluster-autoscaler-operator

CRDs

Purpose

The Cluster Cloud Controller Manager Operator manages and updates the cloud controller managers deployed on top of OpenShift Container Platform. The Operator is based on the Kubebuilder framework and

controller-runtime

It contains the following components:

By default, the Operator exposes Prometheus metrics through the

metrics

Project

cluster-cloud-controller-manager-operator

Purpose

The Cluster CAPI Operator maintains the lifecycle of Cluster API resources. This Operator is responsible for all administrative tasks related to deploying the Cluster API project within an OpenShift Container Platform cluster.

Project

cluster-capi-operator

CRDs

Purpose

The Cluster Config Operator performs the following tasks related to

config.openshift.io

Project

cluster-config-operator

Purpose

The Cluster CSI Snapshot Controller Operator installs and maintains the CSI Snapshot Controller. The CSI Snapshot Controller is responsible for watching the

VolumeSnapshot

Project

cluster-csi-snapshot-controller-operator

Purpose

The Cluster Image Registry Operator manages a singleton instance of the OpenShift image registry. It manages all configuration of the registry, including creating storage.

On initial start up, the Operator creates a default

image-registry

If insufficient information is available to define a complete

image-registry

The Cluster Image Registry Operator runs in the

openshift-image-registry

Project

cluster-image-registry-operator

Purpose

The Cluster Machine Approver Operator automatically approves the CSRs requested for a new worker node after cluster installation.

approve-csr

Project

cluster-machine-approver-operator

Purpose

The Cluster Monitoring Operator manages and updates the Prometheus-based cluster monitoring stack deployed on top of OpenShift Container Platform.

Project

openshift-monitoring

CRDs

Configuration objects

$ oc -n openshift-monitoring edit cm cluster-monitoring-config

Purpose

The Cluster Network Operator installs and upgrades the networking components on an OpenShift Container Platform cluster.

Purpose

The Cluster Samples Operator manages the sample image streams and templates stored in the

openshift

On initial start up, the Operator creates the default samples configuration resource to initiate the creation of the image streams and templates. The configuration object is a cluster scoped object with the key

cluster

configs.samples

The image streams are the Red Hat Enterprise Linux CoreOS (RHCOS)-based OpenShift Container Platform image streams pointing to images on

registry.redhat.io

The Cluster Samples Operator deployment is contained within the

openshift-cluster-samples-operator

registry.redhat.io

openshift

config.json

docker

The image for the Cluster Samples Operator contains image stream and template definitions for the associated OpenShift Container Platform release. After the Cluster Samples Operator creates a sample, it adds an annotation that denotes the OpenShift Container Platform version that it is compatible with. The Operator uses this annotation to ensure that each sample matches the compatible release version. Samples outside of its inventory are ignored, as are skipped samples.

Modifications to any samples that are managed by the Operator are allowed as long as the version annotation is not modified or deleted. However, on an upgrade, as the version annotation will change, those modifications can get replaced as the sample will be updated with the newer version. The Jenkins images are part of the image payload from the installation and are tagged into the image streams directly.

The samples resource includes a finalizer, which cleans up the following upon its deletion:

Upon deletion of the samples resource, the Cluster Samples Operator recreates the resource using the default configuration.

Project

cluster-samples-operator

Purpose

The Cluster Storage Operator sets OpenShift Container Platform cluster-wide storage defaults. It ensures a default storage class exists for OpenShift Container Platform clusters.

Project

cluster-storage-operator

Configuration

No configuration is required.

Notes

Purpose

Cluster Operators manage specific areas of cluster functionality. The Cluster Version Operator (CVO) manages the lifecycle of cluster Operators, many of which are installed in OpenShift Container Platform by default.

The CVO also checks with the OpenShift Update Service to see the valid updates and update paths based on current component versions and information in the graph.

Project

cluster-version-operator

Additional resources

Purpose

The Console Operator installs and maintains the OpenShift Container Platform web console on a cluster.

Project

console-operator

Purpose

The DNS Operator deploys and manages CoreDNS to provide a name resolution service to pods that enables DNS-based Kubernetes Service discovery in OpenShift Container Platform.

The Operator creates a working default deployment based on the cluster’s configuration.

The DNS Operator manages CoreDNS as a Kubernetes daemon set exposed as a service with a static IP. CoreDNS runs on all nodes in the cluster.

Project

cluster-dns-operator

Purpose

The etcd cluster Operator automates etcd cluster scaling, enables etcd monitoring and metrics, and simplifies disaster recovery procedures.

Project

cluster-etcd-operator

CRDs

Configuration objects

$ oc edit etcd cluster

Purpose

The Ingress Operator configures and manages the OpenShift Container Platform router.

Project

openshift-ingress-operator

CRDs

Configuration objects

Notes

The Ingress Operator sets up the router in the

openshift-ingress

$ oc get deployment -n openshift-ingress

The Ingress Operator uses the

clusterNetwork[].cidr

network/cluster

clusterNetwork

cidr

In the following example, Ingress Controllers managed by the Ingress Operator will run in IPv4-only mode because only one cluster network exists and the network is an IPv4

cidr

$ oc get network/cluster -o jsonpath='{.status.clusterNetwork[*]}'

map[cidr:10.128.0.0/14 hostPrefix:23]

Purpose

The Insights Operator gathers OpenShift Container Platform configuration data and sends it to Red Hat. The data is used to produce proactive insights recommendations about potential issues that a cluster might be exposed to. These insights are communicated to cluster administrators through Insights Advisor on console.redhat.com.

Project

insights-operator

Configuration

No configuration is required.

Notes

Insights Operator compliments OpenShift Container Platform Telemetry.

Additional resources

Purpose

The Kubernetes API Server Operator manages and updates the Kubernetes API server deployed on top of OpenShift Container Platform. The Operator is based on the OpenShift Container Platform

library-go

Project

openshift-kube-apiserver-operator

CRDs

Configuration objects

$ oc edit kubeapiserver

Purpose

The Kubernetes Controller Manager Operator manages and updates the Kubernetes Controller Manager deployed on top of OpenShift Container Platform. The Operator is based on OpenShift Container Platform

library-go

It contains the following components:

By default, the Operator exposes Prometheus metrics through the

metrics

Project

cluster-kube-controller-manager-operator

Purpose

The Kubernetes Scheduler Operator manages and updates the Kubernetes Scheduler deployed on top of OpenShift Container Platform. The Operator is based on the OpenShift Container Platform

library-go

The Kubernetes Scheduler Operator contains the following components:

By default, the Operator exposes Prometheus metrics through the metrics service.

Project

cluster-kube-scheduler-operator

Configuration

The configuration for the Kubernetes Scheduler is the result of merging:

All of these are sparse configurations, invalidated JSON snippets which are merged to form a valid configuration at the end.

Purpose

The Kubernetes Storage Version Migrator Operator detects changes of the default storage version, creates migration requests for resource types when the storage version changes, and processes migration requests.

Project

cluster-kube-storage-version-migrator-operator

Purpose

The Machine API Operator manages the lifecycle of specific purpose custom resource definitions (CRD), controllers, and RBAC objects that extend the Kubernetes API. This declares the desired state of machines in a cluster.

Project

machine-api-operator

CRDs

Purpose

The Machine Config Operator manages and applies configuration and updates of the base operating system and container runtime, including everything between the kernel and kubelet.

There are four components:

Project

openshift-machine-config-operator

Purpose

The Marketplace Operator simplifies the process for bringing off-cluster Operators to your cluster by using a set of default Operator Lifecycle Manager (OLM) catalogs on the cluster. When the Marketplace Operator is installed, it creates the

openshift-marketplace

openshift-marketplace

Project

operator-marketplace

Purpose

The Node Tuning Operator helps you manage node-level tuning by orchestrating the TuneD daemon and achieves low latency performance by using the Performance Profile controller. The majority of high-performance applications require some level of kernel tuning. The Node Tuning Operator provides a unified management interface to users of node-level sysctls and more flexibility to add custom tuning specified by user needs.

The Operator manages the containerized TuneD daemon for OpenShift Container Platform as a Kubernetes daemon set. It ensures the custom tuning specification is passed to all containerized TuneD daemons running in the cluster in the format that the daemons understand. The daemons run on all nodes in the cluster, one per node.

Node-level settings applied by the containerized TuneD daemon are rolled back on an event that triggers a profile change or when the containerized TuneD daemon is terminated gracefully by receiving and handling a termination signal.

The Node Tuning Operator uses the Performance Profile controller to implement automatic tuning to achieve low latency performance for OpenShift Container Platform applications. The cluster administrator configures a performance profile to define node-level settings such as the following:

The Node Tuning Operator is part of a standard OpenShift Container Platform installation in version 4.1 and later.

Project

cluster-node-tuning-operator

Additional resources

Purpose

The OpenShift API Server Operator installs and maintains the

openshift-apiserver

Project

openshift-apiserver-operator

CRDs

Purpose

The OpenShift Controller Manager Operator installs and maintains the

OpenShiftControllerManager

$ oc get clusteroperator openshift-controller-manager -o yaml

The custom resource definitino (CRD)

openshiftcontrollermanagers.operator.openshift.io

$ oc get crd openshiftcontrollermanagers.operator.openshift.io -o yaml

Project

cluster-openshift-controller-manager-operator

Purpose

Operator Lifecycle Manager (OLM) helps users install, update, and manage the lifecycle of Kubernetes native applications (Operators) and their associated services running across their OpenShift Container Platform clusters. It is part of the Operator Framework, an open source toolkit designed to manage Operators in an effective, automated, and scalable way.

Figure 6.1. Operator Lifecycle Manager workflow

OLM runs by default in OpenShift Container Platform 4.11, which aids cluster administrators in installing, upgrading, and granting access to Operators running on their cluster. The OpenShift Container Platform web console provides management screens for cluster administrators to install Operators, as well as grant specific projects access to use the catalog of Operators available on the cluster.

For developers, a self-service experience allows provisioning and configuring instances of databases, monitoring, and big data services without having to be subject matter experts, because the Operator has that knowledge baked into it.

CRDs

Operator Lifecycle Manager (OLM) is composed of two Operators: the OLM Operator and the Catalog Operator.

Each of these Operators is responsible for managing the custom resource definitions (CRDs) that are the basis for the OLM framework:

ClusterServiceVersion

csv

InstallPlan

ip

CatalogSource

catsrc

Subscription

sub

OperatorGroup

og

OperatorGroup

Each of these Operators is also responsible for creating the following resources:

Deployments

ServiceAccounts

(Cluster)Roles

(Cluster)RoleBindings

CustomResourceDefinitions

ClusterServiceVersions

OLM Operator

The OLM Operator is responsible for deploying applications defined by CSV resources after the required resources specified in the CSV are present in the cluster.

The OLM Operator is not concerned with the creation of the required resources; you can choose to manually create these resources using the CLI or using the Catalog Operator. This separation of concern allows users incremental buy-in in terms of how much of the OLM framework they choose to leverage for their application.

The OLM Operator uses the following workflow:

Catalog Operator

The Catalog Operator is responsible for resolving and installing cluster service versions (CSVs) and the required resources they specify. It is also responsible for watching catalog sources for updates to packages in channels and upgrading them, automatically if desired, to the latest available versions.

To track a package in a channel, you can create a

Subscription

CatalogSource

InstallPlan

The Catalog Operator uses the following workflow:

Catalog Registry

The Catalog Registry stores CSVs and CRDs for creation in a cluster and stores metadata about packages and channels.

A package manifest is an entry in the Catalog Registry that associates a package identity with sets of CSVs. Within a package, channels point to a particular CSV. Because CSVs explicitly reference the CSV that they replace, a package manifest provides the Catalog Operator with all of the information that is required to update a CSV to the latest version in a channel, stepping through each intermediate version.

Additional resources

Purpose

The OpenShift Service CA Operator mints and manages serving certificates for Kubernetes services.

Project

openshift-service-ca-operator

Purpose

The vSphere Problem Detector Operator checks clusters that are deployed on vSphere for common installation and misconfiguration issues that are related to storage.

Configuration

No configuration is required.

Notes

Additional resources