Red Hat Summit Red Hat SummitSupportConsoleDevelopersStart a trialContact

Chapter 4. Enabling and disabling encryption in-transit post deployment

You can enable encryption in-transit for the existing clusters after the deployment of clusters both in internal and external modes.

Prerequisites

  • OpenShift Data Foundation is deployed and a storage cluster is created.

Procedure

  1. Patch the storagecluster to add encryption enabled as true to the storage cluster spec: 

    $ oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": true}}} }]'
storagecluster.ocs.openshift.io/ocs-storagecluster patched

  2. Check the configurations. 

    $ oc get storagecluster ocs-storagecluster -n openshift-storage -o yaml | yq ‘.spec.network’
connections:
  encryption:
    enabled: true

  3. Wait for around 10 minutes for ceph daemons to restart and then check the pods. 

    $ oc get pods -n openshift-storage | grep rook-ceph
rook-ceph-crashcollector-ip-10-0-2-111.ec2.internal-796ffcm9kn9   1/1     Running     0          5m11s
rook-ceph-crashcollector-ip-10-0-27-61.ec2.internal-854b4d8sk5z   1/1     Running     0          5m9s
rook-ceph-crashcollector-ip-10-0-33-53.ec2.internal-589d9f4f8vx   1/1     Running     0          5m7s
rook-ceph-exporter-ip-10-0-2-111.ec2.internal-6d48cdc5fd-2tmsl    1/1     Running     0          5m9s
rook-ceph-exporter-ip-10-0-27-61.ec2.internal-546c66c7cc-9lnpz    1/1     Running     0          5m7s
rook-ceph-exporter-ip-10-0-33-53.ec2.internal-b5555994c-x8mzz     1/1     Running     0          5m5s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-7bd754f6vwps2   2/2     Running     0          4m56s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-6cc5cc647c78m   2/2     Running     0          4m30s
rook-ceph-mgr-a-6f8467578d-f8279                                  3/3     Running     0          3m40s
rook-ceph-mgr-b-66754d99cf-9q58g                                  3/3     Running     0          3m27s
rook-ceph-mon-a-75bc5dd655-tvdqf                                  2/2     Running     0          4m7s
rook-ceph-mon-b-6b6d4d9b4c-tjbpz                                  2/2     Running     0          4m55s
rook-ceph-mon-c-7456bb5f67-rtwpj                                  2/2     Running     0          4m32s
rook-ceph-operator-7b5b9cdb9b-tvmb6                               1/1     Running     0          45m
rook-ceph-osd-0-b78dd99f6-n4wbm                                   2/2     Running     0          3m3s
rook-ceph-osd-1-5887bf6d8d-2sncc                                  2/2     Running     0          2m39s
rook-ceph-osd-2-784b59c4c8-44phh                                  2/2     Running     0          2m14s
rook-ceph-osd-prepare-a075cf185c9b2e5d92ec3f7769565e38-ztrms      0/1     Completed   0          42m
rook-ceph-osd-prepare-b4b48dc5e3bef99ab377e2a255a9142a-mvgnd      0/1     Completed   0          42m
rook-ceph-osd-prepare-fae2ea2ad4aacbf62010ae5b60b87f57-6t9l5      0/1     Completed   0          42m
     
    $ oc get storagecluster -n openshift-storage
NAME                 AGE   PHASE   EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   27m   Ready              2024-11-06T16:15:26Z   4.18.0

  4. Remount existing volumes.

    Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume."

Prerequisites

  • OpenShift Data Foundation is deployed and a storage cluster is created.
  • Encryption in-transit is enabled.

Procedure

  1. Patch the storagecluster to update encryption enabled as false in the storage cluster spec: 

    ~ $ oc patch storagecluster ocs-storagecluster -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": false}}} }]'
storagecluster.ocs.openshift.io/ocs-storagecluster patched

  2. Check the configurations. 

    $ oc get storagecluster ocs-storagecluster -n openshift-storage -o yaml | yq ‘.spec.network’

connections:
  encryption:
    enabled: false

  3. Wait for around 10 minutes for ceph daemons to restart and then check the pods. 

    $ oc get pods -n openshift-storage | grep rook-ceph
rook-ceph-crashcollector-ip-10-0-2-111.ec2.internal-796ffcm9kn9   1/1     Running     0          5m11s
rook-ceph-crashcollector-ip-10-0-27-61.ec2.internal-854b4d8sk5z   1/1     Running     0          5m9s
rook-ceph-crashcollector-ip-10-0-33-53.ec2.internal-589d9f4f8vx   1/1     Running     0          5m7s
rook-ceph-exporter-ip-10-0-2-111.ec2.internal-6d48cdc5fd-2tmsl    1/1     Running     0          5m9s
rook-ceph-exporter-ip-10-0-27-61.ec2.internal-546c66c7cc-9lnpz    1/1     Running     0          5m7s
rook-ceph-exporter-ip-10-0-33-53.ec2.internal-b5555994c-x8mzz     1/1     Running     0          5m5s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-a-7bd754f6vwps2   2/2     Running     0          4m56s
rook-ceph-mds-ocs-storagecluster-cephfilesystem-b-6cc5cc647c78m   2/2     Running     0          4m30s
rook-ceph-mgr-a-6f8467578d-f8279                                  3/3     Running     0          3m40s
rook-ceph-mgr-b-66754d99cf-9q58g                                  3/3     Running     0          3m27s
rook-ceph-mon-a-75bc5dd655-tvdqf                                  2/2     Running     0          4m7s
rook-ceph-mon-b-6b6d4d9b4c-tjbpz                                  2/2     Running     0          4m55s
rook-ceph-mon-c-7456bb5f67-rtwpj                                  2/2     Running     0          4m32s
rook-ceph-operator-7b5b9cdb9b-tvmb6                               1/1     Running     0          45m
rook-ceph-osd-0-b78dd99f6-n4wbm                                   2/2     Running     0          3m3s
rook-ceph-osd-1-5887bf6d8d-2sncc                                  2/2     Running     0          2m39s
rook-ceph-osd-2-784b59c4c8-44phh                                  2/2     Running     0          2m14s
rook-ceph-osd-prepare-a075cf185c9b2e5d92ec3f7769565e38-ztrms      0/1     Completed   0          42m
rook-ceph-osd-prepare-b4b48dc5e3bef99ab377e2a255a9142a-mvgnd      0/1     Completed   0          42m
rook-ceph-osd-prepare-fae2ea2ad4aacbf62010ae5b60b87f57-6t9l5      0/1     Completed   0          42m
     
    $ oc get storagecluster -n openshift-storage
NAME                 AGE   PHASE   EXTERNAL   CREATED AT             VERSION
ocs-storagecluster   27m   Ready              2024-11-06T16:15:26Z   4.18.0

  4. Remount existing volumes.

    Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume."

Prerequisites

  • OpenShift Data Foundation is deployed and a storage cluster is created.

Procedure

  1. Patch the storagecluster to add encryption enabled as true the storage cluster spec: 

    $ oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": true}}} }]'
storagecluster.ocs.openshift.io/ocs-external-storagecluster patched

  2. Check the connection settings in the CR. 

    oc get storagecluster
NAME                          AGE   PHASE   EXTERNAL   CREATED AT             VERSION
ocs-external-storagecluster   9h    Ready   true       2024-11-06T20:48:03Z   4.18.0
     
    $ oc get storagecluster ocs-external-storagecluster -o yaml | yq '.spec.network.connections'
encryption:
  enabled: true

Procedure

  1. Apply Encryption in-transit settings. 

    root@ceph-client ~]# ceph config set global ms_client_mode secure
[root@ceph-client ~]# ceph config set global ms_cluster_mode secure
[root@ceph-client ~]# ceph config set global ms_service_mode secure
[root@ceph-client ~]# ceph config set global rbd_default_map_options ms_mode=secure

  2. Check the settings. 

    [root@ceph-client ~]# ceph config dump | grep ms_
ceph config dump | grep ms_
global   basic      ms_client_mode secure                    *
global   basic      ms_cluster_mode secure                   *
global   basic      ms_service_mode secure                   *
global   advanced   rbd_default_map_options ms_mode=secure   *

  3. Restart all Ceph daemons. 

    [root@ceph-client ~]# ceph orch ls --format plain | tail -n +2 | awk '{print $1}' | xargs -I {} ceph orch restart {}
Scheduled to restart alertmanager.osd-0 on host 'osd-0'
Scheduled to restart ceph-exporter.osd-0 on host 'osd-0'
Scheduled to restart ceph-exporter.osd-2 on host 'osd-2'
Scheduled to restart ceph-exporter.osd-3 on host 'osd-3'
Scheduled to restart ceph-exporter.osd-1 on host 'osd-1'
Scheduled to restart crash.osd-0 on host 'osd-0'
Scheduled to restart crash.osd-2 on host 'osd-2'
Scheduled to restart crash.osd-3 on host 'osd-3'
Scheduled to restart crash.osd-1 on host 'osd-1'
Scheduled to restart grafana.osd-0 on host 'osd-0'
Scheduled to restart mds.fsvol001.osd-0.lpciqk on host 'osd-0'
Scheduled to restart mds.fsvol001.osd-2.wocnxz on host 'osd-2'
Scheduled to restart mgr.osd-0.dtkyni on host 'osd-0'
Scheduled to restart mgr.osd-2.kqcxwu on host 'osd-2'
Scheduled to restart mon.osd-2 on host 'osd-2'
Scheduled to restart mon.osd-3 on host 'osd-3'
Scheduled to restart mon.osd-1 on host 'osd-1'
Scheduled to restart node-exporter.osd-0 on host 'osd-0'
Scheduled to restart node-exporter.osd-2 on host 'osd-2'
Scheduled to restart node-exporter.osd-3 on host 'osd-3'
Scheduled to restart node-exporter.osd-1 on host 'osd-1'
Scheduled to restart osd.1 on host 'osd-0'
Scheduled to restart osd.4 on host 'osd-0'
Scheduled to restart osd.0 on host 'osd-2'
Scheduled to restart osd.5 on host 'osd-2'
Scheduled to restart osd.2 on host 'osd-3'
Scheduled to restart osd.6 on host 'osd-3'
Scheduled to restart osd.3 on host 'osd-1'
Scheduled to restart osd.7 on host 'osd-1'
Scheduled to restart prometheus.osd-0 on host 'osd-0'
Scheduled to restart rgw.rgw.ssl.osd-1.smzpfj on host 'osd-1'

    Wait for the restarting of all the daemons.

4.3.2. Remount existing volumes.
Copy link

Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume.

Prerequisites

  • OpenShift Data Foundation is deployed and a storage cluster is created.
  • Encryption in-transit is enabled for the external mode cluster.

Procedure

Removing encryption in-transit settings from Red Hat Ceph Storage cluster

  1. Remove and check encryption in-transit configurations. 

    [root@ceph-client ~]# ceph config rm global ms_client_mode
[root@ceph-client ~]# ceph config rm global ms_cluster_mode
[root@ceph-client ~]# ceph config rm global ms_service_mode
[root@ceph-client ~]# ceph config rm global rbd_default_map_options

[root@ceph-client ~]# ceph config dump | grep ms_
[root@ceph-client ~]#

  2. Restart all Ceph daemons. 

    [root@ceph-client ~]# ceph orch ls --format plain | tail -n +2 | awk '{print $1}' | xargs -I {} ceph orch restart {}
Scheduled to restart alertmanager.osd-0 on host 'osd-0'
Scheduled to restart ceph-exporter.osd-0 on host 'osd-0'
Scheduled to restart ceph-exporter.osd-2 on host 'osd-2'
Scheduled to restart ceph-exporter.osd-3 on host 'osd-3'
Scheduled to restart ceph-exporter.osd-1 on host 'osd-1'
Scheduled to restart crash.osd-0 on host 'osd-0'
Scheduled to restart crash.osd-2 on host 'osd-2'
Scheduled to restart crash.osd-3 on host 'osd-3'
Scheduled to restart crash.osd-1 on host 'osd-1'
Scheduled to restart grafana.osd-0 on host 'osd-0'
Scheduled to restart mds.fsvol001.osd-0.lpciqk on host 'osd-0'
Scheduled to restart mds.fsvol001.osd-2.wocnxz on host 'osd-2'
Scheduled to restart mgr.osd-0.dtkyni on host 'osd-0'
Scheduled to restart mgr.osd-2.kqcxwu on host 'osd-2'
Scheduled to restart mon.osd-2 on host 'osd-2'
Scheduled to restart mon.osd-3 on host 'osd-3'
Scheduled to restart mon.osd-1 on host 'osd-1'
Scheduled to restart node-exporter.osd-0 on host 'osd-0'
Scheduled to restart node-exporter.osd-2 on host 'osd-2'
Scheduled to restart node-exporter.osd-3 on host 'osd-3'
Scheduled to restart node-exporter.osd-1 on host 'osd-1'
Scheduled to restart osd.1 on host 'osd-0'
Scheduled to restart osd.4 on host 'osd-0'
Scheduled to restart osd.0 on host 'osd-2'
Scheduled to restart osd.5 on host 'osd-2'
Scheduled to restart osd.2 on host 'osd-3'
Scheduled to restart osd.6 on host 'osd-3'
Scheduled to restart osd.3 on host 'osd-1'
Scheduled to restart osd.7 on host 'osd-1'
Scheduled to restart prometheus.osd-0 on host 'osd-0'
Scheduled to restart rgw.rgw.ssl.osd-1.smzpfj on host 'osd-1'
     
    [root@ceph-client ~]# ceph orch ps
NAME                       HOST   PORTS             STATUS          REFRESHED  AGE  MEM USE  MEM LIM  VERSION           IMAGE ID      CONTAINER ID
alertmanager.osd-0         osd-0  *:9093,9094       running (116s)     9s ago  10h    19.5M        -  0.26.0            7dbf12091920  4694a72d4bbd
ceph-exporter.osd-0        osd-0                    running (19s)      9s ago  10h    7310k        -  18.2.1-229.el9cp  3fd804e38f5b  49bdc7d99471
ceph-exporter.osd-1        osd-1                    running (97s)     26s ago  10h    7285k        -  18.2.1-229.el9cp  3fd804e38f5b  7000d59d23b4
ceph-exporter.osd-2        osd-2                    running (76s)     26s ago  10h    7306k        -  18.2.1-229.el9cp  3fd804e38f5b  3907515cc352
ceph-exporter.osd-3        osd-3                    running (49s)     26s ago  10h    6971k        -  18.2.1-229.el9cp  3fd804e38f5b  3f3952490780
crash.osd-0                osd-0                    running (17s)      9s ago  10h    6878k        -  18.2.1-229.el9cp  3fd804e38f5b  38e041fb86e3
crash.osd-1                osd-1                    running (96s)     26s ago  10h    6895k        -  18.2.1-229.el9cp  3fd804e38f5b  21ce3ef7d896
crash.osd-2                osd-2                    running (74s)     26s ago  10h    6899k        -  18.2.1-229.el9cp  3fd804e38f5b  210ca9c8d928
crash.osd-3                osd-3                    running (47s)     26s ago  10h    6899k        -  18.2.1-229.el9cp  3fd804e38f5b  710d42d9d138
grafana.osd-0              osd-0  *:3000            running (114s)     9s ago  10h    72.9M        -  10.4.0-pre        f142b583a1b1  3dc5e2248e95
mds.fsvol001.osd-0.qjntcu  osd-0                    running (99s)      9s ago  10h    17.5M        -  18.2.1-229.el9cp  3fd804e38f5b  50efa881c04b
mds.fsvol001.osd-2.qneujv  osd-2                    running (51s)     26s ago  10h    15.3M        -  18.2.1-229.el9cp  3fd804e38f5b  a306f2d2d676
mgr.osd-0.zukgyq           osd-0  *:9283,8765,8443  running (21s)      9s ago  10h     442M        -  18.2.1-229.el9cp  3fd804e38f5b  8ef9b728675e
mgr.osd-1.jqfyal           osd-1  *:8443,9283,8765  running (92s)     26s ago  10h     480M        -  18.2.1-229.el9cp  3fd804e38f5b  1ab52db89bfd
mon.osd-1                  osd-1                    running (90s)     26s ago  10h    41.7M    2048M  18.2.1-229.el9cp  3fd804e38f5b  88d1fe1e10ac
mon.osd-2                  osd-2                    running (72s)     26s ago  10h    31.1M    2048M  18.2.1-229.el9cp  3fd804e38f5b  02f57d3bb44f
mon.osd-3                  osd-3                    running (45s)     26s ago  10h    24.0M    2048M  18.2.1-229.el9cp  3fd804e38f5b  5e3783f2b4fa
node-exporter.osd-0        osd-0  *:9100            running (15s)      9s ago  10h    7843k        -  1.7.0             8c904aa522d0  2dae2127349b
node-exporter.osd-1        osd-1  *:9100            running (94s)     26s ago  10h    11.2M        -  1.7.0             8c904aa522d0  010c3fcd55cd
node-exporter.osd-2        osd-2  *:9100            running (69s)     26s ago  10h    17.2M        -  1.7.0             8c904aa522d0  436f2d513f31
node-exporter.osd-3        osd-3  *:9100            running (41s)     26s ago  10h    12.4M        -  1.7.0             8c904aa522d0  5579f0d494b8
osd.0                      osd-0                    running (109s)     9s ago  10h     126M    4096M  18.2.1-229.el9cp  3fd804e38f5b  997076cd39d4
osd.1                      osd-1                    running (85s)     26s ago  10h     139M    4096M  18.2.1-229.el9cp  3fd804e38f5b  08b720f0587d
osd.2                      osd-2                    running (65s)     26s ago  10h     143M    4096M  18.2.1-229.el9cp  3fd804e38f5b  104ad4227163
osd.3                      osd-3                    running (36s)     26s ago  10h    94.5M    1435M  18.2.1-229.el9cp  3fd804e38f5b  db8b265d9f43
osd.4                      osd-0                    running (104s)     9s ago  10h     164M    4096M  18.2.1-229.el9cp  3fd804e38f5b  50dcbbf7e012
osd.5                      osd-1                    running (80s)     26s ago  10h     131M    4096M  18.2.1-229.el9cp  3fd804e38f5b  63b21fe970b5
osd.6                      osd-3                    running (32s)     26s ago  10h     243M    1435M  18.2.1-229.el9cp  3fd804e38f5b  26c7ba208489
osd.7                      osd-2                    running (61s)     26s ago  10h     130M    4096M  18.2.1-229.el9cp  3fd804e38f5b  871a2b75e64f
prometheus.osd-0           osd-0  *:9095            running (12s)      9s ago  10h    44.6M        -  2.48.0            58069186198d  e49a064d2478
rgw.rgw.ssl.osd-1.bsmbgd   osd-1  *:80              running (78s)     26s ago  10h    75.4M        -  18.2.1-229.el9cp  3fd804e38f5b  d03c9f7ae4a4

    Patching the CR

  3. Patch the storagecluster to update encryption enabled as false in the storage cluster spec: 

    $ oc patch storagecluster ocs-external-storagecluster -n openshift-storage --type json --patch  '[{ "op": "replace", "path": "/spec/network", "value": {"connections": {"encryption": {"enabled": false}}} }]'
storagecluster.ocs.openshift.io/ocs-external-storagecluster patched

  4. Check the configurations. 

    $ oc get storagecluster
NAME                          AGE   PHASE   EXTERNAL   CREATED AT             VERSION
ocs-external-storagecluster   12h   Ready   true       2024-11-06T20:48:03Z   4.18.0
     
    $ oc get storagecluster ocs-external-storagecluster -o yaml | yq '.spec.network.connections'
encryption:
  enabled: false

    Remount existing volumes

    Depending on your best practices for application maintenance, you can choose the best approach for your environment to remount or remap volumes. One way to remount is to delete the existing application pod and bring up another application pod to use the volume. Another option is to drain the nodes where the applications are running..This ensures that the volume is unmounted from the current pod and then mounted to a new pod, allowing for remapping or remounting of the volume.

Red Hat logoGithubredditYoutubeTwitter

Learn

Try, buy, & sell

Communities

About Red Hat

We deliver hardened solutions that make it easier for enterprises to work across platforms and environments, from the core datacenter to the network edge.

Making open source more inclusive

Red Hat is committed to replacing problematic language in our code, documentation, and web properties. For more details, see the Red Hat Blog.

About Red Hat Documentation

We help Red Hat users innovate and achieve their goals with our products and services with content they can trust. Explore our recent updates.

Legal Notice

Theme

© 2026 Red HatBack to top