5.6. 使用 Ceph 管理器 crash 模块

流程

1. 确定启用了 crash 模块：

   示例

   [ceph: root@host01 /]# ceph mgr module ls | more
   {
       "always_on_modules": [
           "balancer",
           "crash",
           "devicehealth",
           "orchestrator_cli",
           "progress",
           "rbd_support",
           "status",
           "volumes"
       ],
       "enabled_modules": [
           "dashboard",
           "pg_autoscaler",
           "prometheus"
       ]

   Copy to Clipboard Toggle word wrap

2. 保存崩溃转储：元数据文件是存储在 crash dir 中作为 meta 的 JSON blob。您可以调用 ceph 命令 -i - 选项，该选项会从 stdin 读取。

   示例

   [ceph: root@host01 /]# ceph crash post -i meta

   Copy to Clipboard Toggle word wrap

3. 列出所有新的以及归档的崩溃信息的时间戳或 UUID 崩溃 ID：

   示例

   [ceph: root@host01 /]# ceph crash ls

   Copy to Clipboard Toggle word wrap

4. 列出所有新崩溃信息的时间戳或 UUID 崩溃 ID：

   示例

   [ceph: root@host01 /]# ceph crash ls-new

   Copy to Clipboard Toggle word wrap

5. 列出所有新崩溃信息的时间戳或 UUID 崩溃 ID：

   示例

   [ceph: root@host01 /]# ceph crash ls-new

   Copy to Clipboard Toggle word wrap

6. 列出按年龄分组的保存崩溃信息的摘要：

   示例

   [ceph: root@host01 /]# ceph crash stat
   8 crashes recorded
   8 older than 1 days old:
   2022-05-20T08:30:14.533316Z_4ea88673-8db6-4959-a8c6-0eea22d305c2
   2022-05-20T08:30:14.590789Z_30a8bb92-2147-4e0f-a58b-a12c2c73d4f5
   2022-05-20T08:34:42.278648Z_6a91a778-bce6-4ef3-a3fb-84c4276c8297
   2022-05-20T08:34:42.801268Z_e5f25c74-c381-46b1-bee3-63d891f9fc2d
   2022-05-20T08:34:42.803141Z_96adfc59-be3a-4a38-9981-e71ad3d55e47
   2022-05-20T08:34:42.830416Z_e45ed474-550c-44b3-b9bb-283e3f4cc1fe
   2022-05-24T19:58:42.549073Z_b2382865-ea89-4be2-b46f-9a59af7b7a2d
   2022-05-24T19:58:44.315282Z_1847afbc-f8a9-45da-94e8-5aef0738954e

   Copy to Clipboard Toggle word wrap

7. 查看保存崩溃的详情：

   语法

   ceph crash info CRASH_ID

   Copy to Clipboard Toggle word wrap

   示例

   [ceph: root@host01 /]# ceph crash info 2022-05-24T19:58:42.549073Z_b2382865-ea89-4be2-b46f-9a59af7b7a2d
   {
       "assert_condition": "session_map.sessions.empty()",
       "assert_file": "/builddir/build/BUILD/ceph-16.1.0-486-g324d7073/src/mon/Monitor.cc",
       "assert_func": "virtual Monitor::~Monitor()",
       "assert_line": 287,
       "assert_msg": "/builddir/build/BUILD/ceph-16.1.0-486-g324d7073/src/mon/Monitor.cc: In function 'virtual Monitor::~Monitor()' thread 7f67a1aeb700 time 2022-05-24T19:58:42.545485+0000\n/builddir/build/BUILD/ceph-16.1.0-486-g324d7073/src/mon/Monitor.cc: 287: FAILED ceph_assert(session_map.sessions.empty())\n",
       "assert_thread_name": "ceph-mon",
       "backtrace": [
           "/lib64/libpthread.so.0(+0x12b30) [0x7f679678bb30]",
           "gsignal()",
           "abort()",
           "(ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f6798c8d37b]",
           "/usr/lib64/ceph/libceph-common.so.2(+0x276544) [0x7f6798c8d544]",
           "(Monitor::~Monitor()+0xe30) [0x561152ed3c80]",
           "(Monitor::~Monitor()+0xd) [0x561152ed3cdd]",
           "main()",
           "__libc_start_main()",
           "_start()"
       ],
       "ceph_version": "16.2.8-65.el8cp",
       "crash_id": "2022-07-06T19:58:42.549073Z_b2382865-ea89-4be2-b46f-9a59af7b7a2d",
       "entity_name": "mon.ceph-adm4",
       "os_id": "rhel",
       "os_name": "Red Hat Enterprise Linux",
       "os_version": "8.5 (Ootpa)",
       "os_version_id": "8.5",
       "process_name": "ceph-mon",
       "stack_sig": "957c21d558d0cba4cee9e8aaf9227b3b1b09738b8a4d2c9f4dc26d9233b0d511",
       "timestamp": "2022-07-06T19:58:42.549073Z",
       "utsname_hostname": "host02",
       "utsname_machine": "x86_64",
       "utsname_release": "4.18.0-240.15.1.el8_3.x86_64",
       "utsname_sysname": "Linux",
       "utsname_version": "#1 SMP Wed Jul 06 03:12:15 EDT 2022"
   }

   Copy to Clipboard Toggle word wrap

8. 删除比 KEEP days 旧的已保存的崩溃：其中 KEEP 必须是一个整数。

   语法

   ceph crash prune KEEP

   Copy to Clipboard Toggle word wrap

   示例

   [ceph: root@host01 /]# ceph crash prune 60

   Copy to Clipboard Toggle word wrap

9. 对崩溃报告进行归档，使其不再被视为 RECENT_CRASH 健康检查，且不会出现在 crash ls-new 输出中。它会出现在 crash ls 中。

   语法

   ceph crash archive CRASH_ID

   Copy to Clipboard Toggle word wrap

   示例

   [ceph: root@host01 /]# ceph crash archive 2022-05-24T19:58:42.549073Z_b2382865-ea89-4be2-b46f-9a59af7b7a2d

   Copy to Clipboard Toggle word wrap

10. 记录所有崩溃报告：

    示例

    [ceph: root@host01 /]# ceph crash archive-all

    Copy to Clipboard Toggle word wrap

11. 删除崩溃转储：

    语法

    ceph crash rm CRASH_ID

    Copy to Clipboard Toggle word wrap

    示例

    [ceph: root@host01 /]# ceph crash rm 2022-05-24T19:58:42.549073Z_b2382865-ea89-4be2-b46f-9a59af7b7a2d

    Copy to Clipboard Toggle word wrap