3.10. Configuring the log curator
You can configure log retention time. That is, you can specify how long the default Elasticsearch log store keeps indices by configuring a separate retention policy for each of the three log sources: infrastructure logs, application logs, and audit logs. For instructions, see Configuring log retention time.
Configuring log retention time is recommended method for curating log data: It works with both the current data model and the previous data model from OpenShift Container Platform 4.4 and earlier.
Optionally, to remove Elasticsearch indices that use the data model from OpenShift Container Platform 4.4 and earlier, you can also use the Elasticsearch Curator. The following sections explain how to use the Elasticsearch Curator.
The Elasticsearch Curator is deprecated in OpenShift Container Platform 4.7 (OpenShift Logging 5.0) and will be removed in OpenShift Logging 5.1.
3.10.1. Configuring the Curator schedule
You can specify the schedule for Curator using the Cluster Logging custom resource created by the OpenShift Logging installation.
The Elasticsearch Curator is deprecated in OpenShift Container Platform 4.7 (OpenShift Logging 5.0) and will be removed in OpenShift Logging 5.1.
Prerequisites
- Cluster logging and Elasticsearch must be installed.
Procedure
To configure the Curator schedule:
Edit the
ClusterLoggingcustom resource in theopenshift-loggingproject:$ oc edit clusterlogging instance
apiVersion: "logging.openshift.io/v1" kind: "ClusterLogging" metadata: name: "instance" ... curation: curator: schedule: 30 3 * * * 1 type: curator- 1
- Specify the schedule for Curator in cron format.
注意The time zone is set based on the host node where the Curator pod runs.
3.10.2. Configuring Curator index deletion
You can configure Elasticsearch Curator to delete Elasticsearch data that uses the data model prior to OpenShift Container Platform version 4.5. You can configure per-project and global settings. Global settings apply to any project not specified. Per-project settings override global settings.
The Elasticsearch Curator is deprecated in OpenShift Container Platform 4.7 (OpenShift Logging 5.0) and will be removed in OpenShift Logging 5.1.
Prerequisites
- Cluster logging must be installed.
Procedure
To delete indices:
Edit the OpenShift Container Platform custom Curator configuration file:
$ oc edit configmap/curator
Set the following parameters as needed:
config.yaml: | project_name: action unit:valueThe available parameters are:
表 3.2. Project options Variable Name Description project_nameThe actual name of a project, such as myapp-devel. For OpenShift Container Platform operations logs, use the name
.operationsas the project name.actionThe action to take, currently only
deleteis allowed.unitThe period to use for deletion,
days,weeks, ormonths.valueThe number of units.
表 3.3. Filter options Variable Name Description .defaultsUse
.defaultsas theproject_nameto set the defaults for projects that are not specified..regexThe list of regular expressions that match project names.
patternThe valid and properly escaped regular expression pattern enclosed by single quotation marks.
For example, to configure Curator to:
-
Delete indices in the myapp-dev project older than
1 day -
Delete indices in the myapp-qe project older than
1 week -
Delete operations logs older than
8 weeks -
Delete all other projects indices after they are
31 daysold -
Delete indices older than 1 day that are matched by the
^project\..+\-dev.*$regex -
Delete indices older than 2 days that are matched by the
^project\..+\-test.*$regex
Use:
config.yaml: |
.defaults:
delete:
days: 31
.operations:
delete:
weeks: 8
myapp-dev:
delete:
days: 1
myapp-qe:
delete:
weeks: 1
.regex:
- pattern: '^project\..+\-dev\..*$'
delete:
days: 1
- pattern: '^project\..+\-test\..*$'
delete:
days: 2
When you use months as the $UNIT for an operation, Curator starts counting at the first day of the current month, not the current day of the current month. For example, if today is April 15, and you want to delete indices that are 2 months older than today (delete: months: 2), Curator does not delete indices that are dated older than February 15; it deletes indices older than February 1. That is, it goes back to the first day of the current month, then goes back two whole months from that date. If you want to be exact with Curator, it is best to use days (for example, delete: days: 30).
