Chapter 8. Security

Procedure

1. Create a JSON file for the encryption configuration:

   Example

   [user@client ~]$ vi bucket-encryption.json

2. Add the encryption configuration rules to the file:

   Example

   {
   	"Rules": [
           {
   	    "ApplyServerSideEncryptionByDefault": {
                       "SSEAlgorithm": "AES256"
   	    }
          }
       ]
   }

3. Set the default encryption for the bucket:

   Syntax

   aws --endpoint-url=pass:q[_RADOSGW_ENDPOINT_URL_]:pass:q[_PORT_] s3api put-bucket-encryption --bucket pass:q[_BUCKET_NAME_] --server-side-encryption-configuration pass:q[_file://PATH_TO_BUCKET_ENCRYPTION_CONFIGURATION_FILE/BUCKET_ENCRYPTION_CONFIGURATION_FILE.json_]

   Example

   [user@client ~]$ aws --endpoint-url=http://host01:80 s3api put-bucket-encryption --bucket testbucket --server-side-encryption-configuration file://bucket-encryption.json

Procedure

1. Create a token policy:

   1. For the Key/Value secret engine:

      Example

      [root@vault ~]# vault policy write rgw-kv-policy -<<EOF
       path "secret/data/*" {
         capabilities = ["read"]
       }
      EOF

   2. For the Transit engine:

      Example

      [root@vault ~]# vault policy write rgw-transit-policy -<<EOF
        path "transit/keys/*" {
          capabilities = [ "create", "update" ]
          denied_parameters = {"exportable" = [], "allow_plaintext_backup" = [] }
        }
      
        path "transit/keys/*" {
          capabilities = ["read", "delete"]
        }
      
        path "transit/keys/" {
          capabilities = ["list"]
        }
      
        path "transit/keys/+/rotate" {
          capabilities = [ "update" ]
        }
      
        path "transit/*" {
          capabilities = [ "update" ]
        }
      EOF

      Note

      If you have used the Transit secret engine on an older version of Ceph, the token policy is:

      Example

      [root@vault ~]# vault policy write old-rgw-transit-policy -<<EOF
        path "transit/export/encryption-key/*" {
          capabilities = ["read"]
        }
      EOF

Procedure

1. Use the ceph config set client.rgw OPTION VALUE command to enable Vault as the encryption key store:

   Syntax

   ceph config set client.rgw rgw_crypt_s3_kms_backend vault

2. Add the following options and values:

   Syntax

   ceph config set client.rgw rgw_crypt_vault_auth agent
   ceph config set client.rgw rgw_crypt_vault_addr http://VAULT_SERVER:8100

3. Customize the policy as per the use case.

4. Get the role-id:

   Syntax

   vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .data.role_id > PATH_TO_FILE

5. Get the secret-id:

   Syntax

   vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .data.secret_id > PATH_TO_FILE

6. Create the configuration for the Vault agent:

   Example

   pid_file = "/run/kv-vault-agent-pid"
   auto_auth {
     method "AppRole" {
       mount_path = "auth/approle"
       config = {
         role_id_file_path ="/root/vault_configs/kv-agent-role-id"
         secret_id_file_path ="/root/vault_configs/kv-agent-secret-id"
         remove_secret_id_file_after_reading ="false"
       }
     }
   }
   cache {
     use_auto_auth_token = true
   }
   listener "tcp" {
     address = "127.0.0.1:8100"
     tls_disable = true
   }
   vault {
     address = "http://10.8.128.9:8200"
   }

7. Use systemctl to run the persistent daemon:

   Example

   [root@host03 ~]# /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl

8. A token file is populated with a valid token when the Vault agent runs.

9. Select a Vault secret engine, either Key/Value or Transit.

   1. If using Key/Value, then add the following line:

      Example

      [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine kv

   2. If using Transit, then add the following line:

      Example

      [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine transit

10. Use the ceph config set client.rgw OPTION VALUE command to set the Vault namespace to retrieve the encryption keys:

    Example

    [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_namespace testnamespace1

11. Restrict where the Ceph Object Gateway retrieves the encryption keys from the Vault by setting a path prefix:

    Example

    [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_prefix /v1/secret/data

    1. For exportable Transit keys, set the prefix path as follows:

       Example

       [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_prefix /v1/transit/export/encryption-key

       Assuming the domain name of the Vault server is vault-server, the Ceph Object Gateway will fetch encrypted transit keys from the following URL:

       Example

       http://vault-server:8200/v1/transit/export/encryption-key

12. Restart the Ceph Object Gateway daemons.

    1. To restart the Ceph Object Gateway on an individual node in the storage cluster:

       Syntax

       systemctl restart ceph-CLUSTER_ID@SERVICE_TYPE.ID.service

       Example

       [root@host03 ~]# systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service

    2. To restart the Ceph Object Gateways on all nodes in the storage cluster:

       Syntax

       ceph orch restart SERVICE_TYPE

       Example

       [ceph: root@host03 /]# ceph orch restart rgw

Procedure

1. Log into the Cephadm shell

   Example

   [root@host01 ~]# cephadm shell

2. Enable Vault as the secrets engine to retrieve SSE-S3 encryption keys:

   Syntax

   ceph config set client.rgw rgw_crypt_sse_s3_backend vault

3. To set the authentication method to use with SSE-S3 and Vault, configure the following settings:

   Syntax

   ceph config set client.rgw rgw_crypt_sse_s3_vault_auth agent
   ceph config set client.rgw rgw_crypt_sse_s3_vault_addr http://VAULT_AGENT:VAULT_AGENT_PORT

   Example

   [ceph: root@host01 ~]# ceph config set client.rgw rgw_crypt_sse_s3_vault_auth agent
   [ceph: root@host01 ~]# ceph config set client.rgw rgw_crypt_sse_s3_vault_addr http://vaultagent:8100

   1. Customize the policy as per your use case to set up a Vault agent.

   2. Get the role-id:

      Syntax

      vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .rgw-ap-role-id > PATH_TO_FILE

   3. Get the secret-id:

      Syntax

      vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .rgw-ap-secret-id > PATH_TO_FILE

   4. Create the configuration for the Vault agent:

      Example

      pid_file = "/run/rgw-vault-agent-pid"
      auto_auth {
        method "AppRole" {
          mount_path = "auth/approle"
          config = {
            role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id"
            secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id"
            remove_secret_id_file_after_reading ="false"
          }
        }
      }
      cache {
        use_auto_auth_token = true
      }
      listener "tcp" {
        address = "127.0.0.1:8100"
        tls_disable = true
      }
      vault {
        address = "https://vaultserver:8200"
      }

   5. Use systemctl to run the persistent daemon:

      Example

      [root@host01 ~]# /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl

   6. A token file is populated with a valid token when the Vault agent runs.

4. Set the Vault secret engine to use to retrieve encryption keys, either Key/Value or Transit.

   1. If using Key/Value, set the following:

      Example

      [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_secret_engine kv

   2. If using Transit, set the following:

      Example

      [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_secret_engine transit

5. Optional: Configure the Ceph Object Gateway to access Vault within a particular namespace to retrieve the encryption keys:

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_namespace company/testnamespace1

   Note

   Vault namespaces allow teams to operate within isolated environments known as tenants. The Vault namespaces feature is only available in the Vault Enterprise version.

6. Optional: Restrict access to a particular subset of the Vault secret space by setting a URL path prefix, where the Ceph Object Gateway retrieves the encryption keys from:

   1. If using Key/Value, set the following:

      Example

      [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_prefix /v1/secret/data

   2. If using Transit, set the following:

      Example

      [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_prefix /v1/transit

      Assuming the domain name of the Vault server is vaultserver, the Ceph Object Gateway will fetch encrypted transit keys from the following URL:

      Example

      http://vaultserver:8200/v1/transit

7. Optional: To use custom SSL certification to authenticate with Vault, configure the following settings:

   Syntax

   ceph config set client.rgw rgw_crypt_sse_s3_vault_verify_ssl true
   ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_cacert PATH_TO_CA_CERTIFICATE
   ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientcert PATH_TO_CLIENT_CERTIFICATE
   ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientkey PATH_TO_PRIVATE_KEY

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_verify_ssl true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_cacert /etc/ceph/vault.ca
   [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientcert /etc/ceph/vault.crt
   [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientkey /etc/ceph/vault.key

8. Restart the Ceph Object Gateway daemons.

   1. To restart the Ceph Object Gateway on an individual node in the storage cluster:

      Syntax

      systemctl restart ceph-CLUSTER_ID@SERVICE_TYPE.ID.service

      Example

      [root@host01 ~]# systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service

   2. To restart the Ceph Object Gateways on all nodes in the storage cluster:

      Syntax

      ceph orch restart SERVICE_TYPE

      Example

      [ceph: root@host01 /]# ceph orch restart rgw

Procedure

1. Enable the Key/Value version 2 secret engine:

   Example

   vault secrets enable -path secret kv-v2

2. Create a new key:

   Syntax

   vault kv put secret/PROJECT_NAME/BUCKET_NAME key=$(openssl rand -base64 32)

   Example

   [root@vault ~]# vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32)
   
   ====== Metadata ======
   Key              Value
   ---              -----
   created_time     2020-02-21T17:01:09.095824999Z
   deletion_time    n/a
   destroyed        false
   version          1

Procedure

1. Enable the Transit secret engine:

   [root@vault ~]# vault secrets enable transit

2. Create a new exportable key:

   Syntax

   vault write -f transit/keys/BUCKET_NAME exportable=true

   Example

   [root@vault ~]# vault write -f transit/keys/mybucketkey exportable=true

   Note

   By default the above command creates a aes256-gcm96 type key.

   1. Enable key rotation:

      Syntax

      vault write -f transit/keys/BUCKET_NAME/rotate exportable=true

      Example

      [root@vault ~]# vault write -f transit/keys/mybucketkey/rotate exportable=true

   2. Specify the duration for key rotation:

      Syntax

      vault write -f transit/keys/BUCKET_NAME/config auto_rotate_period=DURATION

      Example

      [root@vault ~]# vault write -f transit/keys/mybucketkey/config auto_rotate_period=30d

      In this example, 30d specifies that the key is rotated after 30 days. To specify the key rotation duration in hours, use auto_rotate_period=1h. 1h specifies that the key rotates every 1 hour.

   3. Verify key rotation is successful by ensuring that the latest_version value has incremented:

      Syntax

      vault read transit/export/encryption-key/BUCKET_NAME

      Example

      [root@vault ~]# vault read transit/export/encryption-key/mybucketkey

3. Verify the creation of the key:

   Syntax

   vault read transit/export/encryption-key/BUCKET_NAME/VERSION_NUMBER

   Example

   [root@vault ~]# vault read transit/export/encryption-key/mybucketkey/1
   
   Key     Value
   ---     -----
   keys    map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=]
   name    mybucketkey
   type    aes256-gcm96

   Note

   Providing the full key path, including the key version, is required.

Procedure

1. Upload an object using the AWS command line client and provide the Secure Side Encryption (SSE) key ID in the request:

   1. For the Key/Value secret engine:

      Example (with SSE-KMS)

      [user@client ~]$ aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey

      Example (with SSE-S3)

      [user@client ~]$ aws s3api --endpoint http://rgw_host:8080 put-object --bucket my-bucket --key obj1 --body test_file_to_upload --server-side-encryption AES256

      Note

      In the example, the Ceph Object Gateway would fetch the secret from http://vault-server:8200/v1/secret/data/myproject/mybucketkey

   2. For the Transit engine:

      Example (with SSE-KMS)

      [user@client ~]$ aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id mybucketkey

      Example (with SSE-S3)

      [user@client ~]$ aws s3api --endpoint http://rgw_host:8080 put-object --bucket my-bucket --key obj1 --body test_file_to_upload --server-side-encryption AES256

      Note

      In the example, the Ceph Object Gateway would fetch the secret from http://vaultserver:8200/v1/transit/mybucketkey

Procedure

1. Generate a 30 character seed from the urandom Linux device file and store it in the shell variable SEED:

   Example

   [user@host01 ~]$ SEED=$(head -10 /dev/urandom | sha512sum | cut -b 1-30)

2. Print the seed by running echo on the SEED variable:

   Example

   [user@host01 ~]$ echo $SEED
   492dedb20cf51d1405ef6a1316017e

   Configure the one-time password generator and the back-end MFA system to use the same seed.

Procedure

1. Resynchronize a multi-factor authentication TOTP token in case of time skew or failed checks.

   This requires passing in two consecutive pins: the previous pin, and the current pin.

   Syntax

   radosgw-admin mfa resync --uid=USERID --totp-serial=SERIAL --totp-pin=PREVIOUS_PIN --totp=pin=CURRENT_PIN

   Set USERID to the user name MFA is set up on, set SERIAL to the string that represents the ID for the TOTP token, set PREVIOUS_PIN to the user’s previous PIN, and set CURRENT_PIN to the user’s current PIN.

   Example

   [root@host01 ~]# radosgw-admin mfa resync --uid=johndoe --totp-serial=MFAtest --totp-pin=802021 --totp-pin=439996

2. Verify the token was successfully resynchronized by testing a new PIN:

   Syntax

   radosgw-admin mfa check --uid=USERID --totp-serial=SERIAL --totp-pin=PIN

   Set USERID to the user name MFA is set up on, set SERIAL to the string that represents the ID for the TOTP token, and set PIN to the user’s PIN.

   Example

   [root@host01 ~]# radosgw-admin mfa check  --uid=johndoe --totp-serial=MFAtest --totp-pin=870305
   ok

Procedure

1. Delete an MFA TOTP token:

   Syntax

   radosgw-admin mfa remove --uid=USERID --totp-serial=SERIAL

   Set USERID to the user name MFA is set up on and set SERIAL to the string that represents the ID for the TOTP token.

   Example

   [root@host01 ~]# radosgw-admin mfa remove --uid=johndoe --totp-serial=MFAtest

2. Verify the MFA TOTP token was deleted:

   Syntax

   radosgw-admin mfa get --uid=USERID --totp-serial=SERIAL

   Set USERID to the user name MFA is set up on and set SERIAL to the string that represents the ID for the TOTP token.

   Example

   [root@host01 ~]# radosgw-admin mfa get --uid=johndoe --totp-serial=MFAtest
   MFA serial id not found