8.5. Troubleshooting Operator 的问题

流程

1. 列出 Operator 订阅：

   $ oc get subs -n <operator_namespace>

2. 使用 oc describe 命令检查 Subscription 资源：

   $ oc describe sub <subscription_name> -n <operator_namespace>

3. 在命令输出中，找到 Operator 订阅状况类型的 Conditions 部分。在以下示例中，CatalogSourcesUnhealthy 条件类型具有 false 状态，因为所有可用目录源都健康：

   输出示例

   Name:         cluster-logging
   Namespace:    openshift-logging
   Labels:       operators.coreos.com/cluster-logging.openshift-logging=
   Annotations:  <none>
   API Version:  operators.coreos.com/v1alpha1
   Kind:         Subscription
   # ...
   Conditions:
      Last Transition Time:  2019-07-29T13:42:57Z
      Message:               all available catalogsources are healthy
      Reason:                AllCatalogSourcesHealthy
      Status:                False
      Type:                  CatalogSourcesUnhealthy
   # ...

流程

1. 列出命名空间中的目录源。例如，您可以检查 openshift-marketplace 命名空间，该命名空间用于集群范围的目录源：

   $ oc get catalogsources -n openshift-marketplace

   输出示例

   NAME                  DISPLAY               TYPE   PUBLISHER   AGE
   certified-operators   Certified Operators   grpc   Red Hat     55m
   community-operators   Community Operators   grpc   Red Hat     55m
   example-catalog       Example Catalog       grpc   Example Org 2m25s
   redhat-operators      Red Hat Operators     grpc   Red Hat     55m

2. 使用 oc describe 命令获取有关目录源的详情和状态：

   $ oc describe catalogsource example-catalog -n openshift-marketplace

   输出示例

   Name:         example-catalog
   Namespace:    openshift-marketplace
   Labels:       <none>
   Annotations:  operatorframework.io/managed-by: marketplace-operator
                 target.workload.openshift.io/management: {"effect": "PreferredDuringScheduling"}
   API Version:  operators.coreos.com/v1alpha1
   Kind:         CatalogSource
   # ...
   Status:
     Connection State:
       Address:              example-catalog.openshift-marketplace.svc:50051
       Last Connect:         2021-09-09T17:07:35Z
       Last Observed State:  TRANSIENT_FAILURE
     Registry Service:
       Created At:         2021-09-09T17:05:45Z
       Port:               50051
       Protocol:           grpc
       Service Name:       example-catalog
       Service Namespace:  openshift-marketplace
   # ...

   在上例的输出中，最后观察到的状态是 TRANSIENT_FAILURE。此状态表示目录源建立连接时出现问题。

3. 列出创建目录源的命名空间中的 pod：

   $ oc get pods -n openshift-marketplace

   输出示例

   NAME                                    READY   STATUS             RESTARTS   AGE
   certified-operators-cv9nn               1/1     Running            0          36m
   community-operators-6v8lp               1/1     Running            0          36m
   marketplace-operator-86bfc75f9b-jkgbc   1/1     Running            0          42m
   example-catalog-bwt8z                   0/1     ImagePullBackOff   0          3m55s
   redhat-operators-smxx8                  1/1     Running            0          36m

   在命名空间中创建目录源时，会在该命名空间中为目录源创建一个 pod。在前面的示例中，example-catalog-bwt8z pod 的状态是 ImagePullBackOff。此状态表示拉取目录源的索引镜像存在问题。

4. 使用 oc describe 命令检查 pod 以获取更多详细信息：

   $ oc describe pod example-catalog-bwt8z -n openshift-marketplace

   输出示例

   Name:         example-catalog-bwt8z
   Namespace:    openshift-marketplace
   Priority:     0
   Node:         ci-ln-jyryyg2-f76d1-ggdbq-worker-b-vsxjd/10.0.128.2
   ...
   Events:
     Type     Reason          Age                From               Message
     ----     ------          ----               ----               -------
     Normal   Scheduled       48s                default-scheduler  Successfully assigned openshift-marketplace/example-catalog-bwt8z to ci-ln-jyryyf2-f76d1-fgdbq-worker-b-vsxjd
     Normal   AddedInterface  47s                multus             Add eth0 [10.131.0.40/23] from openshift-sdn
     Normal   BackOff         20s (x2 over 46s)  kubelet            Back-off pulling image "quay.io/example-org/example-catalog:v1"
     Warning  Failed          20s (x2 over 46s)  kubelet            Error: ImagePullBackOff
     Normal   Pulling         8s (x3 over 47s)   kubelet            Pulling image "quay.io/example-org/example-catalog:v1"
     Warning  Failed          8s (x3 over 47s)   kubelet            Failed to pull image "quay.io/example-org/example-catalog:v1": rpc error: code = Unknown desc = reading manifest v1 in quay.io/example-org/example-catalog: unauthorized: access to the requested resource is not authorized
     Warning  Failed          8s (x3 over 47s)   kubelet            Error: ErrImagePull

   在前面的示例输出中，错误消息表示目录源的索引镜像因为授权问题而无法成功拉取。例如，索引镜像可能存储在需要登录凭证的 registry 中。

流程

1. 列出集群中运行的 Operator。输出包括 Operator 版本、可用性和运行时间信息：

   $ oc get clusteroperators

2. 列出在 Operator 命名空间中运行的 Operator pod，以及 pod 状态、重启和年龄：

   $ oc get pod -n <operator_namespace>

3. 输出详细的 Operator pod 概述：

   $ oc describe pod <operator_pod_name> -n <operator_namespace>