第 4 章 更新集群日志记录
在将 OpenShift Container Platform 集群从 4.3 升级到 4.4 后,必须将集群日志记录从 4.3 升级到 4.4。
4.1. 更新集群日志记录
升级 OpenShift Container Platform 集群后,您可以通过更新 Elasticsearch Operator 和 Cluster Logging Operator 的订阅将集群日志记录从 4.3 升级到 4.4。
从 OpenShift Container Platform 4.3 开始,新引进的日志转发功能带来的变化改变了 out_forward。创建 ConfigMap 来配置 out_forward。任何对 Fluentd ConfigMap 中的 secure-forward.conf
部分的更新都会被删除。
如果使用 out_forward 插件,在更新前,您可以从 Fluentd ConfigMap 中复制当前的 secure-forward.conf
部分,并在创建 secure-forward
ConfigMap 时使用复制的数据。
先决条件
- 将集群从 4.3 更新至 4.4。
确保集群日志记录具有健康状态:
-
所有 pod 都为
Ready
状态。 - Elasticsearch 集群处于健康状态。
-
所有 pod 都为
-
另外,如果您需要创建
secure-forward
ConfigMap,复制当前 Fluentd ConfigMap 中的secure-forward.conf
部分的内容。请参阅上述备注。
流程
更新 Elasticsearch Operator:
-
在 Web 控制台中,点 Operators
Installed Operators。 -
选择
openshift-operators-redhat
项目。 - 点 Elasticsearch Operator。
-
点 Subscription
Channel。 - 在 Change Subscription Update Channel 窗口中,选择 4.4 并点 Save。
等待几秒钟,然后点 Operators
Installed Operators。 Elasticsearch Operator 显示为 4.4 版本。例如:
Elasticsearch Operator 4.4.0-201909201915 provided by Red Hat, Inc
-
在 Web 控制台中,点 Operators
更新 Cluster Logging Operator:
-
在 Web 控制台中,点 Operators
Installed Operators。 -
选择
openshift-logging
项目。 - 点 Cluster Logging Operator。
-
点 Subscription
Channel。 - 在 Change Subscription Update Channel 窗口中,选择 4.4 并点 Save。
等待几秒钟,然后点 Operators
Installed Operators。 Cluster Logging Operator 显示为 4.4 版本。例如:
Cluster Logging 4.4.0-201909201915 provided by Red Hat, Inc
-
在 Web 控制台中,点 Operators
检查日志记录组件:
确保 Elasticsearch Pod 使用的是 4.4 镜像:
$ oc get pod -o yaml -n openshift-logging --selector component=elasticsearch |grep 'image:'
image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-elasticsearch5@sha256:4e081ff048c3a56113bc672af5dfb8d29ea2ddca1fd79a3332a4446a461944f5 image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
确保所有 Elasticsearch pod 都处于 Ready 状态:
$ oc get pod -n openshift-logging --selector component=elasticsearch NAME READY STATUS RESTARTS AGE elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk 2/2 Running 0 31m elasticsearch-cdm-1pbrl44l-2-5c6d87589f-gx5hk 2/2 Running 0 30m elasticsearch-cdm-1pbrl44l-3-88df5d47-m45jc 2/2 Running 0 29m
确保 Elasticsearch 集群健康:
oc exec -n openshift-logging -c elasticsearch elasticsearch-cdm-1pbrl44l-1-55b7546f4c-mshhk -- es_cluster_health { "cluster_name" : "elasticsearch", "status" : "green", ....
确保日志记录收集器 Pod 使用的是 4.4 镜像:
$ oc get pod -n openshift-logging --selector logging-infra=fluentd -o yaml |grep 'image:'
image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483 image: registry.redhat.io/openshift4/ose-logging-fluentd@sha256:20f92b37685bc1003bb490002f7d8a1abee2dd2d157e8532afa3830ce8da3483
确保 Kibana Pod 使用的是 4.4 镜像:
$ oc get pod -n openshift-logging --selector logging-infra=kibana -o yaml |grep 'image:'
image: registry.redhat.io/openshift4/ose-logging-kibana5@sha256:3d657e3b90fae604a8351b1923250f93c04529b36e6ada0aba7c0a038ffef56e image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2 image: registry.redhat.io/openshift4/ose-logging-kibana5@sha256:3d657e3b90fae604a8351b1923250f93c04529b36e6ada0aba7c0a038ffef56e image: registry.redhat.io/openshift4/ose-oauth-proxy@sha256:5fe478210770b21c1eb26c1570bcbda40bc5a79011580ff5ebd4c701a5b04eb2
确保 Curator CronJob 使用的是 4.4 镜像:
$ oc get CronJob curator -n openshift-logging -o yaml |grep 'image:'
image: registry.redhat.io/openshift4/ose-logging-curator5@sha256:330c3499e790d0e184414125a4843cd48849c601eb9f19ff82f30794c858b0bc