Search
SupportConsoleDevelopersStart a trialContact

Chapter 6. Installing Logging

download PDF

OpenShift Container Platform Operators use custom resources (CR) to manage applications and their components. High-level configuration and settings are provided by the user within a CR. The Operator translates high-level directives into low-level actions, based on best practices embedded within the Operator’s logic. A custom resource definition (CRD) defines a CR and lists all the configurations available to users of the Operator. Installing an Operator creates the CRDs, which are then used to generate CRs.

Important

You must install the Red Hat OpenShift Logging Operator after the log store Operator.

You deploy logging by installing the Loki Operator or OpenShift Elasticsearch Operator to manage your log store, followed by the Red Hat OpenShift Logging Operator to manage the components of logging. You can use either the OpenShift Container Platform web console or the OpenShift Container Platform CLI to install or configure logging.

Note

The Logging 5.9 release does not contain an updated version of the OpenShift Elasticsearch Operator. If you currently use the OpenShift Elasticsearch Operator released with Logging 5.8, it will continue to work with Logging until the EOL of Logging 5.8. As an alternative to using the OpenShift Elasticsearch Operator to manage the default log storage, you can use the Loki Operator. For more information on the Logging lifecycle dates, see Platform Agnostic Operators.

Tip

You can alternatively apply all example objects.

6.1. Installing Logging with Elasticsearch using the web console

You can use the OpenShift Container Platform web console to install the OpenShift Elasticsearch and Red Hat OpenShift Logging Operators. Elasticsearch is a memory-intensive application. By default, OpenShift Container Platform installs three Elasticsearch nodes with memory requests and limits of 16 GB. This initial set of three OpenShift Container Platform nodes might not have enough memory to run Elasticsearch within your cluster. If you experience memory issues that are related to Elasticsearch, add more Elasticsearch nodes to your cluster rather than increasing the memory on existing nodes.

Note

If you do not want to use the default Elasticsearch log store, you can remove the internal Elasticsearch logStore and Kibana visualization components from the ClusterLogging custom resource (CR). Removing these components is optional but saves resources.

Prerequisites

  • Ensure that you have the necessary persistent storage for Elasticsearch. Note that each Elasticsearch node requires its own storage volume.

    Note

    If you use a local volume for persistent storage, do not use a raw block volume, which is described with volumeMode: block in the LocalVolume object. Elasticsearch cannot use raw block volumes.

Procedure

To install the OpenShift Elasticsearch Operator and Red Hat OpenShift Logging Operator using the OpenShift Container Platform web console:

  1. Install the OpenShift Elasticsearch Operator:

    1. In the OpenShift Container Platform web console, click Operators OperatorHub.
    2. Choose OpenShift Elasticsearch Operator from the list of available Operators, and click Install.
    3. Ensure that the All namespaces on the cluster is selected under Installation Mode.

    4. Ensure that openshift-operators-redhat is selected under Installed Namespace.

      You must specify the openshift-operators-redhat namespace. The openshift-operators namespace might contain Community Operators, which are untrusted and could publish a metric with the same name as an OpenShift Container Platform metric, which would cause conflicts.

    5. Select Enable Operator recommended cluster monitoring on this namespace.

      This option sets the openshift.io/cluster-monitoring: "true" label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes the openshift-operators-redhat namespace.

    6. Select stable-5.y as the Update Channel.

      Note

      The stable channel only provides updates to the most recent release of logging. To continue receiving updates for prior releases, you must change your subscription channel to stable-x.y, where x.y represents the major and minor version of logging you have installed. For example, stable-5.7.

    7. Select an Approval Strategy.

      • The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
      • The Manual strategy requires a user with appropriate credentials to approve the Operator update.
    8. Click Install.
    9. Verify that the OpenShift Elasticsearch Operator installed by switching to the Operators Installed Operators page.
    10. Ensure that OpenShift Elasticsearch Operator is listed in all projects with a Status of Succeeded.

  2. Install the Red Hat OpenShift Logging Operator:

    1. In the OpenShift Container Platform web console, click Operators OperatorHub.
    2. Choose Red Hat OpenShift Logging from the list of available Operators, and click Install.
    3. Ensure that the A specific namespace on the cluster is selected under Installation Mode.
    4. Ensure that Operator recommended namespace is openshift-logging under Installed Namespace.

    5. Select Enable Operator recommended cluster monitoring on this namespace.

      This option sets the openshift.io/cluster-monitoring: "true" label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes the openshift-logging namespace.

    6. Select stable-5.y as the Update Channel.

    7. Select an Approval Strategy.

      • The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
      • The Manual strategy requires a user with appropriate credentials to approve the Operator update.
    8. Click Install.
    9. Verify that the Red Hat OpenShift Logging Operator installed by switching to the Operators Installed Operators page.

    10. Ensure that Red Hat OpenShift Logging is listed in the openshift-logging project with a Status of Succeeded.

      If the Operator does not appear as installed, to troubleshoot further:

      • Switch to the Operators Installed Operators page and inspect the Status column for any errors or failures.
      • Switch to the Workloads Pods page and check the logs in any pods in the openshift-logging project that are reporting issues.

  3. Create an OpenShift Logging instance:

    1. Switch to the Administration Custom Resource Definitions page.
    2. On the Custom Resource Definitions page, click ClusterLogging.
    3. On the Custom Resource Definition details page, select View Instances from the Actions menu.

    4. On the ClusterLoggings page, click Create ClusterLogging.

      You might have to refresh the page to load the data.

    5. In the YAML field, replace the code with the following:

      Note

      This default OpenShift Logging configuration should support a wide array of environments. Review the topics on tuning and configuring logging components for information on modifications you can make to your OpenShift Logging cluster. 

      apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
  name: instance 1
  namespace: openshift-logging
spec:
  managementState: Managed 2
  logStore:
    type: elasticsearch 3
    retentionPolicy: 4
      application:
        maxAge: 1d
      infra:
        maxAge: 7d
      audit:
        maxAge: 7d
    elasticsearch:
      nodeCount: 3 5
      storage:
        storageClassName: <storage_class_name> 6
        size: 200G
      resources: 7
          limits:
            memory: 16Gi
          requests:
            memory: 16Gi
      proxy: 8
        resources:
          limits:
            memory: 256Mi
          requests:
            memory: 256Mi
      redundancyPolicy: SingleRedundancy
  visualization:
    type: kibana 9
    kibana:
      replicas: 1
  collection:
    type: fluentd 10
    fluentd: {}
      1
      The name must be instance.
      2
      The OpenShift Logging management state. In some cases, if you change the OpenShift Logging defaults, you must set this to Unmanaged. However, an unmanaged deployment does not receive updates until OpenShift Logging is placed back into a managed state.
      3
      Settings for configuring Elasticsearch. Using the CR, you can configure shard replication policy and persistent storage.
      4
      Specify the length of time that Elasticsearch should retain each log source. Enter an integer and a time designation: weeks(w), hours(h/H), minutes(m) and seconds(s). For example, 7d for seven days. Logs older than the maxAge are deleted. You must specify a retention policy for each log source or the Elasticsearch indices will not be created for that source.
      5
      Specify the number of Elasticsearch nodes. See the note that follows this list.
      6
      Enter the name of an existing storage class for Elasticsearch storage. For best performance, specify a storage class that allocates block storage. If you do not specify a storage class, OpenShift Logging uses ephemeral storage.
      7
      Specify the CPU and memory requests for Elasticsearch as needed. If you leave these values blank, the OpenShift Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are 16Gi for the memory request and 1 for the CPU request.
      8
      Specify the CPU and memory requests for the Elasticsearch proxy as needed. If you leave these values blank, the OpenShift Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are 256Mi for the memory request and 100m for the CPU request.
      9
      Settings for configuring Kibana. Using the CR, you can scale Kibana for redundancy and configure the CPU and memory for your Kibana nodes. For more information, see Configuring the log visualizer.
      10
      Settings for configuring Fluentd. Using the CR, you can configure Fluentd CPU and memory limits. For more information, see "Configuring Fluentd".
      Note

      The maximum number of master nodes is three. If you specify a nodeCount greater than 3, OpenShift Container Platform creates three Elasticsearch nodes that are Master-eligible nodes, with the master, client, and data roles. The additional Elasticsearch nodes are created as Data-only nodes, using client and data roles. Master nodes perform cluster-wide actions such as creating or deleting an index, shard allocation, and tracking nodes. Data nodes hold the shards and perform data-related operations such as CRUD, search, and aggregations. Data-related operations are I/O-, memory-, and CPU-intensive. It is important to monitor these resources and to add more Data nodes if the current nodes are overloaded.

      For example, if nodeCount=4, the following nodes are created: 

      $ oc get deployment

      Example output

      cluster-logging-operator-66f77ffccb-ppzbg       1/1    Running 0 7m
elasticsearch-cd-tuhduuw-1-f5c885dbf-dlqws      1/1    Running 0 2m4s
elasticsearch-cdm-ftuhduuw-1-ffc4b9566-q6bhp    2/2    Running 0 2m40s
elasticsearch-cdm-ftuhduuw-2-7b4994dbfc-rd2gc   2/2    Running 0 2m36s
elasticsearch-cdm-ftuhduuw-3-84b5ff7ff8-gqnm2   2/2    Running 0 2m4s

    6. Click Create. This creates the logging components, the Elasticsearch custom resource and components, and the Kibana interface.

  4. Verify the install:

    1. Switch to the Workloads Pods page.

    2. Select the openshift-logging project.

      You should see several pods for OpenShift Logging, Elasticsearch, your collector, and Kibana similar to the following list:

      Example output

      cluster-logging-operator-66f77ffccb-ppzbg       1/1     Running   0          7m
elasticsearch-cdm-ftuhduuw-1-ffc4b9566-q6bhp    2/2     Running   0          2m40s
elasticsearch-cdm-ftuhduuw-2-7b4994dbfc-rd2gc   2/2     Running   0          2m36s
elasticsearch-cdm-ftuhduuw-3-84b5ff7ff8-gqnm2   2/2     Running   0          2m4s
collector-587vb                                   1/1     Running   0          2m26s
collector-7mpb9                                   1/1     Running   0          2m30s
collector-flm6j                                   1/1     Running   0          2m33s
collector-gn4rn                                   1/1     Running   0          2m26s
collector-nlgb6                                   1/1     Running   0          2m30s
collector-snpkt                                   1/1     Running   0          2m28s
kibana-d6d5668c5-rppqm                          2/2     Running   0          2m39s

6.2. Installing Logging with Elasticsearch using the CLI

Elasticsearch is a memory-intensive application. By default, OpenShift Container Platform installs three Elasticsearch nodes with memory requests and limits of 16 GB. This initial set of three OpenShift Container Platform nodes might not have enough memory to run Elasticsearch within your cluster. If you experience memory issues that are related to Elasticsearch, add more Elasticsearch nodes to your cluster rather than increasing the memory on existing nodes.

Prerequisites

  • Ensure that you have the necessary persistent storage for Elasticsearch. Note that each Elasticsearch node requires its own storage volume.

    Note

    If you use a local volume for persistent storage, do not use a raw block volume, which is described with volumeMode: block in the LocalVolume object. Elasticsearch cannot use raw block volumes.

Procedure

  1. Create a Namespace object for the OpenShift Elasticsearch Operator:

    Example Namespace object

    apiVersion: v1
kind: Namespace
metadata:
  name: openshift-operators-redhat 1
  annotations:
    openshift.io/node-selector: ""
  labels:
    openshift.io/cluster-monitoring: "true" 2

    1
    You must specify the openshift-operators-redhat namespace. The openshift-operators namespace might contain Community Operators, which are untrusted and could publish a metric with the same name as an OpenShift Container Platform metric, which would cause conflicts.
    2
    A string value that specifies the label as shown to ensure that cluster monitoring scrapes the openshift-operators-redhat namespace.

  2. Apply the Namespace object by running the following command: 

    $ oc apply -f <filename>.yaml

  3. Create a Namespace object for the Red Hat OpenShift Logging Operator:

    Example Namespace object

    apiVersion: v1
kind: Namespace
metadata:
  name: openshift-logging 1
  annotations:
    openshift.io/node-selector: ""
  labels:
    openshift.io/cluster-monitoring: "true"

    1
    You must specify openshift-logging as the namespace for logging versions 5.7 and earlier. For logging 5.8 and later, you can use any namespace.

  4. Apply the Namespace object by running the following command: 

    $ oc apply -f <filename>.yaml

  5. Create an OperatorGroup object for the OpenShift Elasticsearch Operator:

    Example OperatorGroup object

    apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-operators-redhat
  namespace: openshift-operators-redhat 1
spec: {}

    1
    You must specify the openshift-operators-redhat namespace.

  6. Apply the OperatorGroup object by running the following command: 

    $ oc apply -f <filename>.yaml

  7. Create a Subscription object to subscribe a namespace to the OpenShift Elasticsearch Operator:

    Note

    The stable channel only provides updates to the most recent release of logging. To continue receiving updates for prior releases, you must change your subscription channel to stable-x.y, where x.y represents the major and minor version of logging you have installed. For example, stable-5.7.

    Example Subscription object

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: elasticsearch-operator
  namespace: openshift-operators-redhat 1
spec:
  channel: <channel> 2
  installPlanApproval: Automatic 3
  source: redhat-operators 4
  sourceNamespace: openshift-marketplace
  name: elasticsearch-operator

    1
    You must specify the openshift-operators-redhat namespace.
    2
    Specify stable, or stable-<x.y> as the channel.
    3
    Automatic allows the Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available. Manual requires a user with appropriate credentials to approve the Operator update.
    4
    Specify redhat-operators. If your OpenShift Container Platform cluster is installed on a restricted network, also known as a disconnected cluster, specify the name of the CatalogSource object you created when you configured the Operator Lifecycle Manager (OLM)

  8. Apply the subscription by running the following command: 

    $ oc apply -f <filename>.yaml

  9. Verify the Operator installation by running the following command: 

    $ oc get csv --all-namespaces

    Example output

    NAMESPACE                                          NAME                            DISPLAY                            VERSION          REPLACES                        PHASE
default                                            elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
kube-node-lease                                    elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
kube-public                                        elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
kube-system                                        elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-apiserver-operator                       elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-apiserver                                elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-authentication-operator                  elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-authentication                           elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-cloud-controller-manager-operator        elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-cloud-controller-manager                 elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded
openshift-cloud-credential-operator                elasticsearch-operator.v5.8.3   OpenShift Elasticsearch Operator   5.8.3            elasticsearch-operator.v5.8.2   Succeeded

  10. Create an OperatorGroup object for the Red Hat OpenShift Logging Operator:

    Example OperatorGroup object

    apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: cluster-logging
  namespace: openshift-logging 1
spec:
  targetNamespaces:
  - openshift-logging 2

    1
    You must specify openshift-logging as the namespace for logging versions 5.7 and earlier. For logging 5.8 and later, you can use any namespace.
    2
    You must specify openshift-logging as the namespace for logging versions 5.7 and earlier. For logging 5.8 and later, you can use any namespace.

  11. Apply the OperatorGroup object by running the following command: 

    $ oc apply -f <filename>.yaml

  12. Create a Subscription object to subscribe the namespace to the Red Hat OpenShift Logging Operator:

    Example Subscription object

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-logging
  namespace: openshift-logging 1
spec:
  channel: stable 2
  name: cluster-logging
  source: redhat-operators 3
  sourceNamespace: openshift-marketplace

    1
    You must specify the openshift-logging namespace for logging versions 5.7 and older. For logging 5.8 and later versions, you can use any namespace.
    2
    Specify stable or stable-x.y as the channel.
    3
    Specify redhat-operators. If your OpenShift Container Platform cluster is installed on a restricted network, also known as a disconnected cluster, specify the name of the CatalogSource object you created when you configured the Operator Lifecycle Manager (OLM).

  13. Apply the subscription object by running the following command: 

    $ oc apply -f <filename>.yaml

  14. Create a ClusterLogging object as a YAML file:

    Example ClusterLogging object

    apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
  name: instance 1
  namespace: openshift-logging
spec:
  managementState: Managed 2
  logStore:
    type: elasticsearch 3
    retentionPolicy: 4
      application:
        maxAge: 1d
      infra:
        maxAge: 7d
      audit:
        maxAge: 7d
    elasticsearch:
      nodeCount: 3 5
      storage:
        storageClassName: <storage_class_name> 6
        size: 200G
      resources: 7
          limits:
            memory: 16Gi
          requests:
            memory: 16Gi
      proxy: 8
        resources:
          limits:
            memory: 256Mi
          requests:
            memory: 256Mi
      redundancyPolicy: SingleRedundancy
  visualization:
    type: kibana 9
    kibana:
      replicas: 1
  collection:
    type: fluentd 10
    fluentd: {}

    1
    The name must be instance.
    2
    The OpenShift Logging management state. In some cases, if you change the OpenShift Logging defaults, you must set this to Unmanaged. However, an unmanaged deployment does not receive updates until OpenShift Logging is placed back into a managed state.
    3
    Settings for configuring Elasticsearch. Using the CR, you can configure shard replication policy and persistent storage.
    4
    Specify the length of time that Elasticsearch should retain each log source. Enter an integer and a time designation: weeks(w), hours(h/H), minutes(m) and seconds(s). For example, 7d for seven days. Logs older than the maxAge are deleted. You must specify a retention policy for each log source or the Elasticsearch indices will not be created for that source.
    5
    Specify the number of Elasticsearch nodes.
    6
    Enter the name of an existing storage class for Elasticsearch storage. For best performance, specify a storage class that allocates block storage. If you do not specify a storage class, OpenShift Logging uses ephemeral storage.
    7
    Specify the CPU and memory requests for Elasticsearch as needed. If you leave these values blank, the OpenShift Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are 16Gi for the memory request and 1 for the CPU request.
    8
    Specify the CPU and memory requests for the Elasticsearch proxy as needed. If you leave these values blank, the OpenShift Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are 256Mi for the memory request and 100m for the CPU request.
    9
    Settings for configuring Kibana. Using the CR, you can scale Kibana for redundancy and configure the CPU and memory for your Kibana nodes.
    10
    Settings for configuring Fluentd. Using the CR, you can configure Fluentd CPU and memory limits.
    Note

    The maximum number of master nodes is three. If you specify a nodeCount greater than 3, OpenShift Container Platform creates three Elasticsearch nodes that are Master-eligible nodes, with the master, client, and data roles. The additional Elasticsearch nodes are created as Data-only nodes, using client and data roles. Master nodes perform cluster-wide actions such as creating or deleting an index, shard allocation, and tracking nodes. Data nodes hold the shards and perform data-related operations such as CRUD, search, and aggregations. Data-related operations are I/O-, memory-, and CPU-intensive. It is important to monitor these resources and to add more Data nodes if the current nodes are overloaded.

    For example, if nodeCount=4, the following nodes are created: 

    $ oc get deployment

    Example output

    cluster-logging-operator-66f77ffccb-ppzbg       1/1     Running   0          7m
elasticsearch-cdm-ftuhduuw-1-ffc4b9566-q6bhp    2/2     Running   0          2m40s
elasticsearch-cdm-ftuhduuw-2-7b4994dbfc-rd2gc   2/2     Running   0          2m36s
elasticsearch-cdm-ftuhduuw-3-84b5ff7ff8-gqnm2   2/2     Running   0          2m4s

  15. Apply the ClusterLogging CR by running the following command: 

    $ oc apply -f <filename>.yaml

  16. Verify the installation by running the following command: 

    $ oc get pods -n openshift-logging

    Example output

    NAME                                            READY   STATUS    RESTARTS   AGE
cluster-logging-operator-66f77ffccb-ppzbg       1/1     Running   0          7m
elasticsearch-cdm-ftuhduuw-1-ffc4b9566-q6bhp    2/2     Running   0          2m40s
elasticsearch-cdm-ftuhduuw-2-7b4994dbfc-rd2gc   2/2     Running   0          2m36s
elasticsearch-cdm-ftuhduuw-3-84b5ff7ff8-gqnm2   2/2     Running   0          2m4s
collector-587vb                                 1/1     Running   0          2m26s
collector-7mpb9                                 1/1     Running   0          2m30s
collector-flm6j                                 1/1     Running   0          2m33s
collector-gn4rn                                 1/1     Running   0          2m26s
collector-nlgb6                                 1/1     Running   0          2m30s
collector-snpkt                                 1/1     Running   0          2m28s
kibana-d6d5668c5-rppqm                          2/2     Running   0          2m39s

Important

If there is no retention period defined on the s3 bucket or in the LokiStack custom resource (CR), then the logs are not pruned and they stay in the s3 bucket forever, which might fill up the s3 storage.

6.3. Installing Logging and the Loki Operator using the CLI

To install and configure logging on your OpenShift Container Platform cluster, an Operator such as Loki Operator for log storage must be installed first. This can be done from the OpenShift Container Platform CLI.

Prerequisites

  • You have administrator permissions.
  • You installed the OpenShift CLI (oc).
  • You have access to a supported object store. For example: AWS S3, Google Cloud Storage, Azure, Swift, Minio, or OpenShift Data Foundation.
Note

The stable channel only provides updates to the most recent release of logging. To continue receiving updates for prior releases, you must change your subscription channel to stable-x.y, where x.y represents the major and minor version of logging you have installed. For example, stable-5.7.

  1. Create a Namespace object for Loki Operator:

    Example Namespace object

    apiVersion: v1
kind: Namespace
metadata:
  name: openshift-operators-redhat 1
  annotations:
    openshift.io/node-selector: ""
  labels:
    openshift.io/cluster-monitoring: "true" 2

    1
    You must specify the openshift-operators-redhat namespace. To prevent possible conflicts with metrics, you should configure the Prometheus Cluster Monitoring stack to scrape metrics from the openshift-operators-redhat namespace and not the openshift-operators namespace. The openshift-operators namespace might contain community Operators, which are untrusted and could publish a metric with the same name as an OpenShift Container Platform metric, which would cause conflicts.
    2
    A string value that specifies the label as shown to ensure that cluster monitoring scrapes the openshift-operators-redhat namespace.

  2. Apply the Namespace object by running the following command: 

    $ oc apply -f <filename>.yaml

  3. Create a Subscription object for Loki Operator:

    Example Subscription object

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: loki-operator
  namespace: openshift-operators-redhat 1
spec:
  channel: stable 2
  name: loki-operator
  source: redhat-operators 3
  sourceNamespace: openshift-marketplace

    1
    You must specify the openshift-operators-redhat namespace.
    2
    Specify stable, or stable-5.<y> as the channel.
    3
    Specify redhat-operators. If your OpenShift Container Platform cluster is installed on a restricted network, also known as a disconnected cluster, specify the name of the CatalogSource object you created when you configured the Operator Lifecycle Manager (OLM).

  4. Apply the Subscription object by running the following command: 

    $ oc apply -f <filename>.yaml

  5. Create a namespace object for the Red Hat OpenShift Logging Operator:

    Example namespace object

    apiVersion: v1
kind: Namespace
metadata:
  name: openshift-logging 1
annotations:
    openshift.io/node-selector: ""
labels:
    openshift.io/cluster-logging: "true"
    openshift.io/cluster-monitoring: "true" 2

    1
    The Red Hat OpenShift Logging Operator is only deployable to the openshift-logging namespace.
    2
    A string value that specifies the label as shown to ensure that cluster monitoring scrapes the openshift-operators-redhat namespace.

  6. Apply the namespace object by running the following command: 

    $ oc apply -f <filename>.yaml

  7. Create an OperatorGroup object

    Example OperatorGroup object

    apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: cluster-logging
  namespace: openshift-logging 1
spec:
  targetNamespaces:
  - openshift-logging

    1
    You must specify the openshift-logging namespace.

  8. Apply the OperatorGroup object by running the following command: 

    $ oc apply -f <filename>.yaml

  9. Create a Subscription object: 

    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: cluster-logging
  namespace: openshift-logging 1
spec:
  channel: stable 2
  name: cluster-logging
  source: redhat-operators 3
  sourceNamespace: openshift-marketplace
    1
    You must specify the openshift-logging namespace.
    2
    Specify stable, or stable-5.<y> as the channel.
    3
    Specify redhat-operators. If your OpenShift Container Platform cluster is installed on a restricted network, also known as a disconnected cluster, specify the name of the CatalogSource object you created when you configured the Operator Lifecycle Manager (OLM).

  10. Apply the Subscription object by running the following command: 

    $ oc apply -f <filename>.yaml

  11. Create a LokiStack CR:

    Example LokiStack CR

    apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: logging-loki 1
  namespace: openshift-logging 2
spec:
  size: 1x.small 3
  storage:
    schemas:
    - version: v12
      effectiveDate: "2022-06-01"
    secret:
      name: logging-loki-s3 4
      type: s3 5
      credentialMode: 6
  storageClassName: <storage_class_name> 7
  tenants:
    mode: openshift-logging 8

    1
    Use the name logging-loki.
    2
    You must specify the openshift-logging namespace.
    3
    Specify the deployment size. In the logging 5.8 and later versions, the supported size options for production instances of Loki are 1x.extra-small, 1x.small, or 1x.medium.
    4
    Specify the name of your log store secret.
    5
    Specify the corresponding storage type.
    6
    Optional field, logging 5.9 and later. Supported user configured values are as follows: static is the default authentication mode available for all supported object storage types using credentials stored in a Secret. token for short-lived tokens retrieved from a credential source. In this mode the static configuration does not contain credentials needed for the object storage. Instead, they are generated during runtime using a service, which allows for shorter-lived credentials and much more granular control. This authentication mode is not supported for all object storage types. token-cco is the default value when Loki is running on managed STS mode and using CCO on STS/WIF clusters.
    7
    Specify the name of a storage class for temporary storage. For best performance, specify a storage class that allocates block storage. Available storage classes for your cluster can be listed by using the oc get storageclasses command.
    8
    LokiStack defaults to running in multi-tenant mode, which cannot be modified. One tenant is provided for each log type: audit, infrastructure, and application logs. This enables access control for individual users and user groups to different log streams.

  12. Apply the LokiStack CR object by running the following command: 

    $ oc apply -f <filename>.yaml

  13. Create a ClusterLogging CR object:

    Example ClusterLogging CR object

    apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
  name: instance 1
  namespace: openshift-logging 2
spec:
  collection:
    type: vector
  logStore:
    lokistack:
      name: logging-loki
    retentionPolicy:
      application:
        maxAge: 7d
      audit:
        maxAge: 7d
      infra:
        maxAge: 7d
    type: lokistack
  visualization:
    type: ocp-console
    ocpConsole:
      logsLimit: 15
  managementState: Managed

    1
    Name must be instance.
    2
    Namespace must be openshift-logging.

  14. Apply the ClusterLogging CR object by running the following command: 

    $ oc apply -f <filename>.yaml

  15. Verify the installation by running the following command: 

    $ oc get pods -n openshift-logging

    Example output

    $ oc get pods -n openshift-logging
NAME                                               READY   STATUS    RESTARTS   AGE
cluster-logging-operator-fb7f7cf69-8jsbq           1/1     Running   0          98m
collector-222js                                    2/2     Running   0          18m
collector-g9ddv                                    2/2     Running   0          18m
collector-hfqq8                                    2/2     Running   0          18m
collector-sphwg                                    2/2     Running   0          18m
collector-vv7zn                                    2/2     Running   0          18m
collector-wk5zz                                    2/2     Running   0          18m
logging-view-plugin-6f76fbb78f-n2n4n               1/1     Running   0          18m
lokistack-sample-compactor-0                       1/1     Running   0          42m
lokistack-sample-distributor-7d7688bcb9-dvcj8      1/1     Running   0          42m
lokistack-sample-gateway-5f6c75f879-bl7k9          2/2     Running   0          42m
lokistack-sample-gateway-5f6c75f879-xhq98          2/2     Running   0          42m
lokistack-sample-index-gateway-0                   1/1     Running   0          42m
lokistack-sample-ingester-0                        1/1     Running   0          42m
lokistack-sample-querier-6b7b56bccc-2v9q4          1/1     Running   0          42m
lokistack-sample-query-frontend-84fb57c578-gq2f7   1/1     Running   0          42m

6.4. Installing Logging and the Loki Operator using the web console

To install and configure logging on your OpenShift Container Platform cluster, an Operator such as Loki Operator for log storage must be installed first. This can be done from the OperatorHub within the web console.

Prerequisites

  • You have access to a supported object store (AWS S3, Google Cloud Storage, Azure, Swift, Minio, OpenShift Data Foundation).
  • You have administrator permissions.
  • You have access to the OpenShift Container Platform web console.

Procedure

  1. In the OpenShift Container Platform web console Administrator perspective, go to Operators OperatorHub.

  2. Type Loki Operator in the Filter by keyword field. Click Loki Operator in the list of available Operators, and then click Install.

    Important

    The Community Loki Operator is not supported by Red Hat.

  3. Select stable or stable-x.y as the Update channel.

    Note

    The stable channel only provides updates to the most recent release of logging. To continue receiving updates for prior releases, you must change your subscription channel to stable-x.y, where x.y represents the major and minor version of logging you have installed. For example, stable-5.7.

    The Loki Operator must be deployed to the global operator group namespace openshift-operators-redhat, so the Installation mode and Installed Namespace are already selected. If this namespace does not already exist, it is created for you.

  4. Select Enable Operator-recommended cluster monitoring on this namespace.

    This option sets the openshift.io/cluster-monitoring: "true" label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes the openshift-operators-redhat namespace.

  5. For Update approval select Automatic, then click Install.

    If the approval strategy in the subscription is set to Automatic, the update process initiates as soon as a new Operator version is available in the selected channel. If the approval strategy is set to Manual, you must manually approve pending updates.

  6. Install the Red Hat OpenShift Logging Operator:

    1. In the OpenShift Container Platform web console, click Operators OperatorHub.
    2. Choose Red Hat OpenShift Logging from the list of available Operators, and click Install.
    3. Ensure that the A specific namespace on the cluster is selected under Installation Mode.
    4. Ensure that Operator recommended namespace is openshift-logging under Installed Namespace.

    5. Select Enable Operator recommended cluster monitoring on this namespace.

      This option sets the openshift.io/cluster-monitoring: "true" label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes the openshift-logging namespace.

    6. Select stable-5.y as the Update Channel.

    7. Select an Approval Strategy.

      • The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
      • The Manual strategy requires a user with appropriate credentials to approve the Operator update.
    8. Click Install.
  7. Go to the Operators Installed Operators page. Click the All instances tab.
  8. From the Create new drop-down list, select LokiStack.

  9. Select YAML view, and then use the following template to create a LokiStack CR:

    Example LokiStack CR

    apiVersion: loki.grafana.com/v1
kind: LokiStack
metadata:
  name: logging-loki 1
  namespace: openshift-logging 2
spec:
  size: 1x.small 3
  storage:
    schemas:
    - version: v12
      effectiveDate: "2022-06-01"
    secret:
      name: logging-loki-s3 4
      type: s3 5
      credentialMode: 6
  storageClassName: <storage_class_name> 7
  tenants:
    mode: openshift-logging 8

    1
    Use the name logging-loki.
    2
    You must specify the openshift-logging namespace.
    3
    Specify the deployment size. In the logging 5.8 and later versions, the supported size options for production instances of Loki are 1x.extra-small, 1x.small, or 1x.medium.
    4
    Specify the name of your log store secret.
    5
    Specify the corresponding storage type.
    6
    Optional field, logging 5.9 and later. Supported user configured values are as follows: static is the default authentication mode available for all supported object storage types using credentials stored in a Secret. token for short-lived tokens retrieved from a credential source. In this mode the static configuration does not contain credentials needed for the object storage. Instead, they are generated during runtime using a service, which allows for shorter-lived credentials and much more granular control. This authentication mode is not supported for all object storage types. token-cco is the default value when Loki is running on managed STS mode and using CCO on STS/WIF clusters.
    7
    Specify the name of a storage class for temporary storage. For best performance, specify a storage class that allocates block storage. Available storage classes for your cluster can be listed by using the oc get storageclasses command.
    8
    LokiStack defaults to running in multi-tenant mode, which cannot be modified. One tenant is provided for each log type: audit, infrastructure, and application logs. This enables access control for individual users and user groups to different log streams.
    Important

    It is not possible to change the number 1x for the deployment size.

  10. Click Create.

  11. Create an OpenShift Logging instance:

    1. Switch to the Administration Custom Resource Definitions page.
    2. On the Custom Resource Definitions page, click ClusterLogging.
    3. On the Custom Resource Definition details page, select View Instances from the Actions menu.

    4. On the ClusterLoggings page, click Create ClusterLogging.

      You might have to refresh the page to load the data.

    5. In the YAML field, replace the code with the following: 

      apiVersion: logging.openshift.io/v1
kind: ClusterLogging
metadata:
  name: instance 1
  namespace: openshift-logging 2
spec:
  collection:
    type: vector
  logStore:
    lokistack:
      name: logging-loki
    retentionPolicy:
      application:
        maxAge: 7d
      audit:
        maxAge: 7d
      infra:
        maxAge: 7d
    type: lokistack
  visualization:
    type: ocp-console
    ocpConsole:
      logsLimit: 15

  managementState: Managed
      1
      Name must be instance.
      2
      Namespace must be openshift-logging.

Verification

  1. Go to Operators Installed Operators.
  2. Make sure the openshift-logging project is selected.
  3. In the Status column, verify that you see green checkmarks with InstallSucceeded and the text Up to date.
Note

An Operator might display a Failed status before the installation finishes. If the Operator install completes with an InstallSucceeded message, refresh the page.

Additional resources

Red Hat logoGithubRedditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

© 2024 Red Hat, Inc.