10.2. 컨테이너화된 환경의 Ceph 오브젝트 문제 해결

절차

1. 클러스터에 noout 플래그를 설정합니다.

   예제

   [root@mon ~]# ceph osd set noout

2. OSD 컨테이너를 호스팅하는 노드에 로그인합니다.

3. /etc/systemd/system/ceph-osd@.service 장치 파일을 /root 디렉토리로 백업합니다.

   예제

   [root@osd ~]# cp /etc/systemd/system/ceph-osd@.service /root/ceph-osd@.service.backup

4. /run/ceph-osd@OSD_ID.service-cid 파일을 /root 로 이동합니다.

   예제

   [root@osd ~]# mv /run/ceph-osd@0.service-cid /root

5. /etc/systemd/system/ceph-osd@.service 장치 파일을 편집하고 podman 명령에 -it --entrypoint /bin/bash 옵션을 추가합니다.

   예제

   # Please do not change this file directly since it is managed by Ansible and will be overwritten
   [Unit]
   Description=Ceph OSD
   After=network.target
   
   [Service]
   EnvironmentFile=-/etc/environment
   ExecStartPre=-/usr/bin/rm -f /%t/%n-pid /%t/%n-cid
   ExecStartPre=-/usr/bin/podman rm -f ceph-osd-%i
   ExecStart=/usr/bin/podman run -it --entrypoint /bin/bash \
     -d --conmon-pidfile /%t/%n-pid --cidfile /%t/%n-cid \
     --rm \
     --net=host \
     --privileged=true \
     --pid=host \
     --ipc=host \
     --cpus=2 \
     -v /dev:/dev \
     -v /etc/localtime:/etc/localtime:ro \
     -v /var/lib/ceph:/var/lib/ceph:z \
     -v /etc/ceph:/etc/ceph:z \
     -v /var/run/ceph:/var/run/ceph:z \
     -v /var/run/udev/:/var/run/udev/ \
     -v /var/log/ceph:/var/log/ceph:z \
     -e OSD_BLUESTORE=1 -e OSD_FILESTORE=0 -e OSD_DMCRYPT=0 \
     -e CLUSTER=ceph \
     -v /run/lvm/:/run/lvm/ \
     -e CEPH_DAEMON=OSD_CEPH_VOLUME_ACTIVATE \
     -e CONTAINER_IMAGE=registry.redhat.io/rhceph/rhceph-4-rhel8:latest \
     -e OSD_ID=%i \
     -e DEBUG=stayalive \
     --name=ceph-osd-%i \
      \
     registry.redhat.io/rhceph/rhceph-4-rhel8:latest
   ExecStop=-/usr/bin/sh -c "/usr/bin/podman rm -f `cat /%t/%n-cid`"
   KillMode=none
   Restart=always
   RestartSec=10s
   TimeoutStartSec=120
   TimeoutStopSec=15
   Type=forking
   PIDFile=/%t/%n-pid
   
   [Install]
   WantedBy=multi-user.target

6. systemd 관리자 구성을 다시 로드합니다.

   예제

   [root@osd ~]# systemctl daemon-reload

7. OSD_ID 와 연결된 OSD 서비스를 다시 시작합니다.

   구문

   systemctl restart ceph-osd@OSD_ID.service

   OSD_ID 를 OSD의 ID로 바꿉니다.

   예제

   [root@osd ~]# systemctl restart ceph-osd@0.service

8. OSD_ID 와 연결된 컨테이너에 로그인합니다.

   구문

   podman exec -it ceph-osd-OSD_ID /bin/bash

   예제

   [root@osd ~]# podman exec -it ceph-osd-0 /bin/bash

9. osd fsid 를 가져오고 OSD를 활성화하여 OSD의 논리 볼륨(LV)을 마운트합니다.

   구문

   ceph-volume lvm list |grep -A15 "osd\.OSD_ID"|grep "osd fsid"
   ceph-volume lvm activate --bluestore OSD_ID OSD_FSID

   예제

   [root@osd ~]# ceph-volume lvm list |grep -A15 "osd\.0"|grep "osd fsid"
                 osd fsid                  087eee15-6561-40a3-8fe4-9583ba64a4ff
   [root@osd ~]# ceph-volume lvm activate --bluestore 0 087eee15-6561-40a3-8fe4-9583ba64a4ff
   Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0
   Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0
   Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-41c69f8f-30e2-4685-9c5c-c605898c5537/osd-data-d073e8b3-0b89-4271-af5b-83045fd000dc --path /var/lib/ceph/osd/ceph-0 --no-mon-config
   Running command: /usr/bin/ln -snf /dev/ceph-41c69f8f-30e2-4685-9c5c-c605898c5537/osd-data-d073e8b3-0b89-4271-af5b-83045fd000dc /var/lib/ceph/osd/ceph-0/block
   Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block
   Running command: /usr/bin/chown -R ceph:ceph /dev/mapper/ceph--41c69f8f--30e2--4685--9c5c--c605898c5537-osd--data--d073e8b3--0b89--4271--af5b--83045fd000dc
   Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0
   Running command: /usr/bin/systemctl enable ceph-volume@lvm-0-087eee15-6561-40a3-8fe4-9583ba64a4ff
    stderr: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-volume@lvm-0-087eee15-6561-40a3-8fe4-9583ba64a4ff.service  /usr/lib/systemd/system/ceph-volume@.service.
   Running command: /usr/bin/systemctl enable --runtime ceph-osd@0
    stderr: Created symlink /run/systemd/system/ceph-osd.target.wants/ceph osd@0.service  /usr/lib/systemd/system/ceph-osd@.service.
   Running command: /usr/bin/systemctl start ceph-osd@0
    stderr: Running in chroot, ignoring request: start
   --> ceph-volume lvm activate successful for osd ID: 0

10. fsck 를 실행하고 복구 명령을 실행합니다.

    구문

    ceph-bluestore-tool fsck --path /var/lib/ceph/osd/ceph-OSD_ID
    ceph-bluestore-tool repair --path /var/lib/ceph/osd/ceph-OSD_ID

    예제

    [root@osd ~]# ceph-bluestore-tool fsck --path /var/lib/ceph/osd/ceph-0
    fsck success

    [root@osd ~]# ceph-bluestore-tool repair --path /var/lib/ceph/osd/ceph-0
    repair success

11. 컨테이너를 종료한 후 /root 디렉터리에서 /etc/systemd/system/ceph-osd@.service 장치 파일을 복사합니다.

    예제

    [root@osd ~]# cp /etc/systemd/system/ceph-osd@.service /root/ceph-osd@.service.modified
    [root@osd ~]# cp /root/ceph-osd@.service.backup /etc/systemd/system/ceph-osd@.service

12. systemd 관리자 구성을 다시 로드합니다.

    예제

    [root@osd ~]# systemctl daemon-reload

13. /run/ceph-osd@OSD_ID.service-cid 파일을 /tmp 로 이동합니다.

    예제

    [root@osd ~]# mv /run/ceph-osd@0.service-cid /tmp

14. OSD_ID 와 연결된 OSD 서비스를 다시 시작합니다.

    구문

    [root@osd ~]# systemctl restart ceph-osd@OSD_ID.service

    예제

    [root@osd ~]# systemctl restart ceph-osd@0.service