第 2 章 Installing cluster logging
You can install cluster logging by deploying the Elasticsearch and Cluster Logging Operators. The Elasticsearch Operator creates and manages the Elasticsearch cluster used by cluster logging. The Cluster Logging Operator creates and manages the components of the logging stack.
The process for deploying cluster logging to OpenShift Container Platform involves:
- Reviewing the cluster logging storage considerations.
- Installing the Elasticsearch Operator and Cluster Logging Operator using the OpenShift Container Platform web console or CLI.
2.1. Installing cluster logging using the web console
You can use the OpenShift Container Platform web console to install the Elasticsearch and Cluster Logging operators.
Prerequisites
Ensure that you have the necessary persistent storage for Elasticsearch. Note that each Elasticsearch node requires its own storage volume.
注意If you use a local volume for persistent storage, do not use a raw block volume, which is described with
volumeMode: block
in theLocalVolume
object. Elasticsearch cannot use raw block volumes.Elasticsearch is a memory-intensive application. By default, OpenShift Container Platform installs three Elasticsearch nodes with memory requests and limits of 16 GB. This initial set of three OpenShift Container Platform nodes might not have enough memory to run Elasticsearch within your cluster. If you experience memory issues that are related to Elasticsearch, add more Elasticsearch nodes to your cluster rather than increasing the memory on existing nodes.
Procedure
To install the Elasticsearch Operator and Cluster Logging Operator using the OpenShift Container Platform web console:
Install the Elasticsearch Operator:
-
In the OpenShift Container Platform web console, click Operators
OperatorHub. - Choose Elasticsearch Operator from the list of available Operators, and click Install.
- Ensure that the All namespaces on the cluster is selected under Installation Mode.
Ensure that openshift-operators-redhat is selected under Installed Namespace.
You must specify the
openshift-operators-redhat
namespace. Theopenshift-operators
namespace might contain Community Operators, which are untrusted and could publish a metric with the same name as an OpenShift Container Platform metric, which would cause conflicts.Select Enable operator recommended cluster monitoring on this namespace.
This option sets the
openshift.io/cluster-monitoring: "true"
label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes theopenshift-operators-redhat
namespace.- Select 4.5 as the Update Channel.
Select an Approval Strategy.
- The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
- The Manual strategy requires a user with appropriate credentials to approve the Operator update.
- Click Install.
-
Verify that the Elasticsearch Operator installed by switching to the Operators
Installed Operators page. - Ensure that Elasticsearch Operator is listed in all projects with a Status of Succeeded.
-
In the OpenShift Container Platform web console, click Operators
Install the Cluster Logging Operator:
-
In the OpenShift Container Platform web console, click Operators
OperatorHub. - Choose Cluster Logging from the list of available Operators, and click Install.
- Ensure that the A specific namespace on the cluster is selected under Installation Mode.
- Ensure that Operator recommended namespace is openshift-logging under Installed Namespace.
Select Enable operator recommended cluster monitoring on this namespace.
This option sets the
openshift.io/cluster-monitoring: "true"
label in the Namespace object. You must select this option to ensure that cluster monitoring scrapes theopenshift-logging
namespace.- Select 4.5 as the Update Channel.
Select an Approval Strategy.
- The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
- The Manual strategy requires a user with appropriate credentials to approve the Operator update.
- Click Install.
-
Verify that the Cluster Logging Operator installed by switching to the Operators
Installed Operators page. Ensure that Cluster Logging is listed in the openshift-logging project with a Status of Succeeded.
If the Operator does not appear as installed, to troubleshoot further:
-
Switch to the Operators
Installed Operators page and inspect the Status column for any errors or failures. -
Switch to the Workloads
Pods page and check the logs in any pods in the openshift-logging
project that are reporting issues.
-
Switch to the Operators
-
In the OpenShift Container Platform web console, click Operators
Create a cluster logging instance:
-
Switch to the Administration
Custom Resource Definitions page. - On the Custom Resource Definitions page, click ClusterLogging.
- On the Custom Resource Definition Overview page, select View Instances from the Actions menu.
On the ClusterLoggings page, click Create ClusterLogging.
You might have to refresh the page to load the data.
In the YAML field, replace the code with the following:
注意This default cluster logging configuration should support a wide array of environments. Review the topics on tuning and configuring the cluster logging components for information on modifications you can make to your cluster logging cluster.
apiVersion: "logging.openshift.io/v1" kind: "ClusterLogging" metadata: name: "instance" 1 namespace: "openshift-logging" spec: managementState: "Managed" 2 logStore: type: "elasticsearch" 3 retentionPolicy: 4 application: maxAge: 1d infra: maxAge: 7d audit: maxAge: 7d elasticsearch: nodeCount: 3 5 storage: storageClassName: "<storage-class-name>" 6 size: 200G resources: 7 requests: memory: "8Gi" proxy: 8 resources: limits: memory: 256Mi requests: memory: 256Mi redundancyPolicy: "SingleRedundancy" visualization: type: "kibana" 9 kibana: replicas: 1 curation: type: "curator" curator: schedule: "30 3 * * *" 10 collection: logs: type: "fluentd" 11 fluentd: {}
- 1
- The name must be
instance
. - 2
- The cluster logging management state. In some cases, if you change the cluster logging defaults, you must set this to
Unmanaged
. However, an unmanaged deployment does not receive updates until the cluster logging is placed back into a managed state. - 3
- Settings for configuring Elasticsearch. Using the CR, you can configure shard replication policy and persistent storage.
- 4
- Specify the length of time that Elasticsearch should retain each log source. Enter an integer and a time designation: weeks(w), hours(h/H), minutes(m) and seconds(s). For example,
7d
for seven days. Logs older than themaxAge
are deleted. You must specify a retention policy for each log source or the Elasticsearch indices will not be created for that source. - 5
- Specify the number of Elasticsearch nodes. See the note that follows this list.
- 6
- Enter the name of an existing storage class for Elasticsearch storage. For best performance, specify a storage class that allocates block storage. If you do not specify a storage class, OpenShift Container Platform deploys cluster logging with ephemeral storage only.
- 7
- Enter the name of an existing storage class for Elasticsearch storage. For best performance, specify a storage class that allocates block storage. If you do not specify a storage class, OpenShift Container Platform deploys cluster logging with ephemeral storage only.
- 8
- Specify the CPU and memory requests for Elasticsearch as needed. If you leave these values blank, the Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are
16G
for the memory request and1
for the CPU request. - 9
- Specify the CPU and memory requests for the Elasticsearch proxy as needed. If you leave these values blank, the Elasticsearch Operator sets default values that should be sufficient for most deployments. The default values are
256Mi
for the memory request and100m
for the CPU request. - 10
- Settings for configuring Kibana. Using the CR, you can scale Kibana for redundancy and configure the CPU and memory for your Kibana nodes. For more information, see Configuring the log visualizer.
- 11
- Settings for configuring the Curator schedule. Curator is used to remove data that is in the Elasticsearch index format prior to OpenShift Container Platform 4.5 and will be removed in a later release.
- Settings for configuring Fluentd. Using the CR, you can configure Fluentd CPU and memory limits. For more information, see Configuring Fluentd.
注意The maximum number of Elasticsearch master nodes is three. If you specify a
nodeCount
greater than3
, OpenShift Container Platform creates three Elasticsearch nodes that are Master-eligible nodes, with the master, client, and data roles. The additional Elasticsearch nodes are created as Data-only nodes, using client and data roles. Master nodes perform cluster-wide actions such as creating or deleting an index, shard allocation, and tracking nodes. Data nodes hold the shards and perform data-related operations such as CRUD, search, and aggregations. Data-related operations are I/O-, memory-, and CPU-intensive. It is important to monitor these resources and to add more Data nodes if the current nodes are overloaded.For example, if
nodeCount=4
, the following nodes are created:$ oc get deployment
Example output
cluster-logging-operator 1/1 1 1 18h elasticsearch-cd-x6kdekli-1 0/1 1 0 6m54s elasticsearch-cdm-x6kdekli-1 1/1 1 1 18h elasticsearch-cdm-x6kdekli-2 0/1 1 0 6m49s elasticsearch-cdm-x6kdekli-3 0/1 1 0 6m44s
The number of primary shards for the index templates is equal to the number of Elasticsearch data nodes.
-
Click Create. This creates the Cluster Logging components, the
Elasticsearch
custom resource and components, and the Kibana interface.
-
Switch to the Administration
Verify the install:
-
Switch to the Workloads
Pods page. Select the openshift-logging project.
You should see several pods for cluster logging, Elasticsearch, Fluentd, and Kibana similar to the following list:
- cluster-logging-operator-cb795f8dc-xkckc
- elasticsearch-cdm-b3nqzchd-1-5c6797-67kfz
- elasticsearch-cdm-b3nqzchd-2-6657f4-wtprv
- elasticsearch-cdm-b3nqzchd-3-588c65-clg7g
- fluentd-2c7dg
- fluentd-9z7kk
- fluentd-br7r2
- fluentd-fn2sb
- fluentd-pb2f8
- fluentd-zqgqx
- kibana-7fb4fd4cc9-bvt4p
-
Switch to the Workloads
Additional resources