This documentation is for a release that is no longer maintained
See documentation for the latest supported version 3 or the latest supported version 4.3.9. Configuring systemd-journald and Fluentd
Because Fluentd reads from the journal, and the journal default settings are very low, journal entries can be lost because the journal cannot keep up with the logging rate from system services.
We recommend setting RateLimitInterval=1s and RateLimitBurst=10000 (or even higher if necessary) to prevent the journal from losing entries.
3.9.1. Configuring systemd-journald for cluster logging 复制链接链接已复制到粘贴板!
As you scale up your project, the default logging environment might need some adjustments.
For example, if you are missing logs, you might have to increase the rate limits for journald. You can adjust the number of messages to retain for a specified period of time to ensure that cluster logging does not use excessive resources without dropping logs.
You can also determine if you want the logs compressed, how long to retain logs, how or if the logs are stored, and other settings.
Procedure
Create a
journald.conffile with the required settings:Copy to Clipboard Copied! Toggle word wrap Toggle overflow - 1
- Specify whether you want logs compressed before they are written to the file system. Specify
yesto compress the message ornoto not compress. The default isyes. - 2
- Configure whether to forward log messages. Defaults to
nofor each. Specify:-
ForwardToConsoleto forward logs to the system console. -
ForwardToKsmgto forward logs to the kernel log buffer. -
ForwardToSyslogto forward to a syslog daemon. -
ForwardToWallto forward messages as wall messages to all logged-in users.
-
- 3
- Specify the maximum time to store journal entries. Enter a number to specify seconds. Or include a unit: "year", "month", "week", "day", "h" or "m". Enter
0to disable. The default is1month. - 4
- Configure rate limiting. If, during the time interval defined by
RateLimitIntervalSec, more logs than specified inRateLimitBurstare received, all further messages within the interval are dropped until the interval is over. It is recommended to setRateLimitInterval=1sandRateLimitBurst=10000, which are the defaults. - 5
- Specify how logs are stored. The default is
persistent:-
volatileto store logs in memory in/var/log/journal/. -
persistentto store logs to disk in/var/log/journal/. systemd creates the directory if it does not exist. -
autoto store logs in in/var/log/journal/if the directory exists. If it does not exist, systemd temporarily stores logs in/run/systemd/journal. -
noneto not store logs. systemd drops all logs.
-
- 6
- Specify the timeout before synchronizing journal files to disk for ERR, WARNING, NOTICE, INFO, and DEBUG logs. systemd immediately syncs after receiving a CRIT, ALERT, or EMERG log. The default is
1s. - 7
- Specify the maximum size the journal can use. The default is
8g. - 8
- Specify how much disk space systemd must leave free. The default is
20%. - 9
- Specify the maximum size for individual journal files stored persistently in
/var/log/journal. The default is10M.注意If you are removing the rate limit, you might see increased CPU utilization on the system logging daemons as it processes any messages that would have previously been throttled.
For more information on systemd settings, see https://www.freedesktop.org/software/systemd/man/journald.conf.html. The default settings listed on that page might not apply to OpenShift Container Platform.
Convert the
journal.conffile to base64:export jrnl_cnf=$( cat /journald.conf | base64 -w0 )
$ export jrnl_cnf=$( cat /journald.conf | base64 -w0 )Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create a new
MachineConfigobject for master or worker and add thejournal.confparameters:For example:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Create the machine config:
oc apply -f <filename>.yaml
$ oc apply -f <filename>.yamlCopy to Clipboard Copied! Toggle word wrap Toggle overflow The controller detects the new
MachineConfigobject and generates a newrendered-worker-<hash>version.Monitor the status of the rollout of the new rendered configuration to each node:
oc describe machineconfigpool/worker
$ oc describe machineconfigpool/workerCopy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow