Questo contenuto non è disponibile nella lingua selezionata.
Chapter 8. Security
As a storage administrator, securing the storage cluster environment is important. Red Hat Ceph Storage provides encryption and key management to secure the Ceph Object Gateway access point.
Prerequisites
- A healthy running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
8.1. Server-Side Encryption (SSE)
The Ceph Object Gateway supports server-side encryption of uploaded objects for the S3 application programming interface (API). Server-side encryption means that the S3 client sends data over HTTP in its unencrypted form, and the Ceph Object Gateway stores that data in the Red Hat Ceph Storage cluster in encrypted form.
- Red Hat does NOT support S3 object encryption of Static Large Object (SLO) or Dynamic Large Object (DLO).
-
Currently, none of the Server-Side Encryption (SSE) modes have implemented support for
CopyObject
. It is currently being developed [BZ#2149450].
Server-side encryption is not compatible with multi-site replication due to a known issue. This issue will be resolved in a future release. See Known issues- Mult-site Object Gateway for more details.
To use encryption, client requests MUST send requests over an SSL connection. Red Hat does not support S3 encryption from a client unless the Ceph Object Gateway uses SSL. However, for testing purposes, administrators can disable SSL during testing by setting the rgw_crypt_require_ssl
configuration setting to false
at runtime, using the ceph config set client.rgw
command, and then restarting the Ceph Object Gateway instance.
In a production environment, it might not be possible to send encrypted requests over SSL. In such a case, send requests using HTTP with server-side encryption.
For information about how to configure HTTP with server-side encryption, see the Additional Resources section below.
There are three options for the management of encryption keys:
Customer-provided Keys
When using customer-provided keys, the S3 client passes an encryption key along with each request to read or write encrypted data. It is the customer’s responsibility to manage those keys. Customers must remember which key the Ceph Object Gateway used to encrypt each object.
Ceph Object Gateway implements the customer-provided key behavior in the S3 API according to the Amazon SSE-C specification.
Since the customer handles the key management and the S3 client passes keys to the Ceph Object Gateway, the Ceph Object Gateway requires no special configuration to support this encryption mode.
Key Management Service
When using a key management service, the secure key management service stores the keys and the Ceph Object Gateway retrieves them on demand to serve requests to encrypt or decrypt data.
Ceph Object Gateway implements the key management service behavior in the S3 API according to the Amazon SSE-KMS specification.
Currently, the only tested key management implementations are HashiCorp Vault, and OpenStack Barbican. However, OpenStack Barbican is a Technology Preview and is not supported for use in production systems.
SSE-S3
When using SSE-S3, the keys are stored in vault, but they are automatically created and deleted by Ceph and retrieved as required to serve requests to encrypt or decrypt data.
Ceph Object Gateway implements the SSE-S3 behavior in the S3 API according to the Amazon SSE-S3 specification.
Additional Resources
8.1.1. Setting the default encryption for an existing S3 bucket
As a storage administrator, you can set the default encryption for an existing Amazon S3 bucket so that all objects are encrypted when they are stored in a bucket. You can use Bucket Encryption APIs to support server-side encryption with Amazon S3-managed keys (SSE-S3) or Amazon KMS customer master keys (SSE-KMS).
SSE-KMS is supported only from Red Hat Ceph Storage 5.x, not for Red Hat Ceph Storage 4.x.
You can manage default encryption for an existing Amazon S3 bucket using the PutBucketEncryption
API. All files uploaded to this bucket will have this encryption by defining the default encryption at the bucket level.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway.
- An S3 bucket created.
- An S3 user created with user access.
- Access to a Ceph Object Gateway client with the AWS CLI package installed.
Procedure
Create a JSON file for the encryption configuration:
Example
[user@client ~]$ vi bucket-encryption.json
Add the encryption configuration rules to the file:
Example
{ "Rules": [ { "ApplyServerSideEncryptionByDefault": { "SSEAlgorithm": "AES256" } } ] }
Set the default encryption for the bucket:
Syntax
aws --endpoint-url=pass:q[_RADOSGW_ENDPOINT_URL_]:pass:q[_PORT_] s3api put-bucket-encryption --bucket pass:q[_BUCKET_NAME_] --server-side-encryption-configuration pass:q[_file://PATH_TO_BUCKET_ENCRYPTION_CONFIGURATION_FILE/BUCKET_ENCRYPTION_CONFIGURATION_FILE.json_]
Example
[user@client ~]$ aws --endpoint-url=http://host01:80 s3api put-bucket-encryption --bucket testbucket --server-side-encryption-configuration file://bucket-encryption.json
Verification
Retrieve the bucket encryption configuration for the bucket:
Syntax
aws --endpoint-url=pass:q[_RADOSGW_ENDPOINT_URL_]:pass:q[_PORT_] s3api get-bucket-encryption --bucket BUCKET_NAME
Example
[user@client ~]$ aws --profile ceph --endpoint=http://host01:80 s3api get-bucket-encryption --bucket testbucket { "ServerSideEncryptionConfiguration": { "Rules": [ { "ApplyServerSideEncryptionByDefault": { "SSEAlgorithm": "AES256" } } ] } }
If the bucket does not have a default encryption configuration, the get-bucket-encryption
command returns ServerSideEncryptionConfigurationNotFoundError
.
8.1.2. Deleting the default bucket encryption
You can delete the default bucket encryption for a specified bucket using the s3api delete-bucket-encryption
command.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway.
- An S3 bucket created.
- An S3 user created with user access.
- Access to a Ceph Object Gateway client with the AWS CLI package installed.
Procedure
Delete a bucket encryption configuration:
Syntax
aws --endpoint-url=RADOSGW_ENDPOINT_URL:PORT s3api delete-bucket-encryption --bucket BUCKET_NAME
Example
[user@client ~]$ aws --endpoint-url=http://host01:80 s3api delete-bucket-encryption --bucket testbucket
Verification
Retrieve the bucket encryption configuration for the bucket:
Syntax
aws --endpoint-url=RADOSGW_ENDPOINT_URL:PORT s3api get-bucket-encryption --bucket BUCKET_NAME
Example
[user@client ~]$ aws --endpoint=http://host01:80 s3api get-bucket-encryption --bucket testbucket An error occurred (ServerSideEncryptionConfigurationNotFoundError) when calling the GetBucketEncryption operation: The server side encryption configuration was not found
8.2. Server-side encryption requests
In a production environment, clients often contact the Ceph Object Gateway through a proxy. This proxy is referred to as a load balancer because it connects to multiple Ceph Object Gateways. When the client sends requests to the Ceph Object Gateway, the load balancer routes those requests to the multiple Ceph Object Gateways, thus distributing the workload.
In this type of configuration, it is possible that SSL terminations occur both at a load balancer and between the load balancer and the multiple Ceph Object Gateways. Communication occurs using HTTP only. To set up the Ceph Object Gateways to accept the server-side encryption requests, see Configuring server-side encryption.
8.3. Configuring server-side encryption
You can set up server-side encryption to send requests to the Ceph Object Gateway using HTTP, in cases where it might not be possible to send encrypted requests over SSL.
This procedure uses HAProxy as proxy and load balancer.
Prerequisites
- Root-level access to all nodes in the storage cluster.
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
- Installation of the HAProxy software.
Procedure
Edit the
haproxy.cfg
file:Example
frontend http_web *:80 mode http default_backend rgw frontend rgw-https bind *:443 ssl crt /etc/ssl/private/example.com.pem default_backend rgw backend rgw balance roundrobin mode http server rgw1 10.0.0.71:8080 check server rgw2 10.0.0.80:8080 check
Comment out the lines that allow access to the
http
front end and add instructions to direct HAProxy to use thehttps
front end instead:Example
# frontend http_web *:80 # mode http # default_backend rgw frontend rgw-https bind *:443 ssl crt /etc/ssl/private/example.com.pem http-request set-header X-Forwarded-Proto https if { ssl_fc } http-request set-header X-Forwarded-Proto https # here we set the incoming HTTPS port on the load balancer (eg : 443) http-request set-header X-Forwarded-Port 443 default_backend rgw backend rgw balance roundrobin mode http server rgw1 10.0.0.71:8080 check server rgw2 10.0.0.80:8080 check
Set the
rgw_trust_forwarded_https
option totrue
:Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_trust_forwarded_https true
Enable and start HAProxy:
[root@host01 ~]# systemctl enable haproxy [root@host01 ~]# systemctl start haproxy
Additional Resources
- See the High availability service section in the Red Hat Ceph Storage Object Gateway Guide for additional details.
- See the Red Hat Ceph Storage installation chapter in the Red Hat Ceph Storage Installation Guide for additional details.
8.4. The HashiCorp Vault
As a storage administrator, you can securely store keys, passwords, and certificates in the HashiCorp Vault for use with the Ceph Object Gateway. The HashiCorp Vault provides a secure key management service for server-side encryption used by the Ceph Object Gateway.
The basic workflow:
- The client requests the creation of a secret key from the Vault based on an object’s key ID.
- The client uploads an object with the object’s key ID to the Ceph Object Gateway.
- The Ceph Object Gateway then requests the newly created secret key from the Vault.
- The Vault replies to the request by returning the secret key to the Ceph Object Gateway.
- Now the Ceph Object Gateway can encrypt the object using the new secret key.
- After encryption is done the object is then stored on the Ceph OSD.
Red Hat works with our technology partners to provide this documentation as a service to our customers. However, Red Hat does not provide support for this product. If you need technical assistance for this product, then contact Hashicorp for support.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
- Installation of the HashiCorp Vault software.
8.4.1. Secret engines for Vault
The HashiCorp Vault provides several secret engines to generate, store, or encrypt data. The application programming interface (API) sends data calls to the secret engine asking for action on that data, and the secret engine returns a result of that action request.
The Ceph Object Gateway supports two of the HashiCorp Vault secret engines:
- Key/Value version 2
- Transit
Key/Value version 2
The Key/Value secret engine stores random secrets within the Vault, on disk. With version 2 of the kv
engine, a key can have a configurable number of versions. The default number of versions is 10. Deleting a version does not delete the underlying data, but marks the data as deleted, allowing deleted versions to be undeleted. You can use the API endpoint or the destroy
command to permanently remove a version’s data. To delete all versions and metadata for a key, you can use the metadata
command or the API endpoint. The key names must be strings, and the engine will convert non-string values into strings when using the command line interface. To preserve non-string values, provide a JSON file or use the HTTP application programming interface (API).
For access control list (ACL) policies, the Key/Value secret engine recognizes the distinctions between the create
and update
capabilities.
Transit
The Transit secret engine performs cryptographic functions on in-transit data. The Transit secret engine can generate hashes, can be a source of random bytes, and can also sign and verify data. The Vault does not store data when using the Transit secret engine. The Transit secret engine supports key derivation, by allowing the same key to be used for multiple purposes. Also, the transit secret engine supports key versioning. The Transit secret engine supports these key types:
aes128-gcm96
- AES-GCM with a 128-bit AES key and a 96-bit nonce; supports encryption, decryption, key derivation, and convergent encryption
aes256-gcm96
- AES-GCM with a 256-bit AES key and a 96-bit nonce; supports encryption, decryption, key derivation, and convergent encryption (default)
chacha20-poly1305
- ChaCha20-Poly1305 with a 256-bit key; supports encryption, decryption, key derivation, and convergent encryption
ed25519
- Ed25519; supports signing, signature verification, and key derivation
ecdsa-p256
- ECDSA using curve P-256; supports signing and signature verification
ecdsa-p384
- ECDSA using curve P-384; supports signing and signature verification
ecdsa-p521
- ECDSA using curve P-521; supports signing and signature verification
rsa-2048
- 2048-bit RSA key; supports encryption, decryption, signing, and signature verification
rsa-3072
- 3072-bit RSA key; supports encryption, decryption, signing, and signature verification
rsa-4096
- 4096-bit RSA key; supports encryption, decryption, signing, and signature verification
Additional Resources
- See the KV Secrets Engine documentation on Vault’s project site for more information.
- See the Transit Secrets Engine documentation on Vault’s project site for more information.
8.4.2. Authentication for Vault
The HashiCorp Vault supports several types of authentication mechanisms. The Ceph Object Gateway currently supports the Vault agent method. The Ceph Object Gateway uses the rgw_crypt_vault_auth
, and rgw_crypt_vault_addr
options to configure the use of the HashiCorp Vault.
Red Hat supports the usage of Vault agent as the authentication method for containers and the usage of token authentication is not supported on containers.
Vault Agent
The Vault agent is a daemon that runs on a client node and provides client-side caching, along with token renewal. The Vault agent typically runs on the Ceph Object Gateway node. Run the Vault agent and refresh the token file. When the Vault agent is used in this mode, you can use file system permissions to restrict who has access to the usage of tokens. Also, the Vault agent can act as a proxy server, that is, Vault will add a token when required and add it to the requests passed to it before forwarding them to the actual server. The Vault agent can still handle token renewal just as it would when storing a token in the Filesystem. It is required to secure the network that Ceph Object Gateways uses to connect with the Vault agent, for example, the Vault agent listens to only the localhost.
Additional Resources
- See the Vault Agent documentation on Vault’s project site for more information.
8.4.3. Namespaces for Vault
Using HashiCorp Vault as an enterprise service provides centralized management for isolated namespaces that teams within an organization can use. These isolated namespace environments are known as tenants, and teams within an organization can utilize these tenants to isolate their policies, secrets, and identities from other teams. The namespace features of Vault help support secure multi-tenancy from within a single infrastructure.
Additional Resources
- See the Vault Enterprise Namespaces documentation on Vault’s project site for more information.
8.4.4. Transit engine compatibility support
There is compatibility support for the previous versions of Ceph which used the Transit engine as a simple key store. You can use the compat
option in the Transit engine to configure the compatibility support. You can disable previous support with the following command:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine transit compat=0
This is the default for future versions and you can use the current version for new installations.
The normal default with the current version is:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine transit compat=1
This enables the new engine for newly created objects and still allows the old engine to be used for the old objects. To access old and new objects, the Vault token must have both the old and new transit policies.
You can force use only the old engine with the following command:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine transit compat=2
This mode is selected by default if the Vault ends in export/encryption-key
.
After configuring the client.rgw
options, you need to restart the Ceph Object Gateway daemons for the new values to take effect.
Additional Resources
- See the Vault Agent documentation on Vault’s project site for more information.
8.4.5. Creating token policies for Vault
A token policy specifies the powers that all Vault tokens have. One token can have multiple policies. You should use the required policy for the configuration.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the HashiCorp Vault software.
- Root-level access to the HashiCorp Vault node.
Procedure
Create a token policy:
For the Key/Value secret engine:
Example
[root@vault ~]# vault policy write rgw-kv-policy -<<EOF path "secret/data/*" { capabilities = ["read"] } EOF
For the Transit engine:
Example
[root@vault ~]# vault policy write rgw-transit-policy -<<EOF path "transit/keys/*" { capabilities = [ "create", "update" ] denied_parameters = {"exportable" = [], "allow_plaintext_backup" = [] } } path "transit/keys/*" { capabilities = ["read", "delete"] } path "transit/keys/" { capabilities = ["list"] } path "transit/keys/+/rotate" { capabilities = [ "update" ] } path "transit/*" { capabilities = [ "update" ] } EOF
NoteIf you have used the Transit secret engine on an older version of Ceph, the token policy is:
Example
[root@vault ~]# vault policy write old-rgw-transit-policy -<<EOF path "transit/export/encryption-key/*" { capabilities = ["read"] } EOF
If you are using both SSE-KMS and SSE-S3, you should point each to separate containers. You could either use separate Vault instances or separately mount transit instances or different branches under a common transit point. If you are not using separate Vault instances, you can point SSE-KMS or SSE-S3 to serparate containers using rgw_crypt_vault_prefix
and rgw_crypt_sse_s3_vault_prefix
. When granting Vault permissions to SSE-KMS bucket owners, you should not give them permission to SSE-S3 keys; only Ceph should have access to the SSE-S3 keys.
8.4.6. Configuring the Ceph Object Gateway to use SSE-KMS with Vault
To configure the Ceph Object Gateway to use the HashiCorp Vault with SSE-KMS for key management, it must be set as the encryption key store. Currently, the Ceph Object Gateway supports two different secret engines, and two different authentication methods.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
- Root-level access to a Ceph Object Gateway node.
Procedure
Use the
ceph config set client.rgw OPTION VALUE
command to enable Vault as the encryption key store:Syntax
ceph config set client.rgw rgw_crypt_s3_kms_backend vault
Add the following options and values:
Syntax
ceph config set client.rgw rgw_crypt_vault_auth agent ceph config set client.rgw rgw_crypt_vault_addr http://VAULT_SERVER:8100
- Customize the policy as per the use case.
Get the role-id:
Syntax
vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .data.role_id > PATH_TO_FILE
Get the secret-id:
Syntax
vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .data.secret_id > PATH_TO_FILE
Create the configuration for the Vault agent:
Example
pid_file = "/run/kv-vault-agent-pid" auto_auth { method "AppRole" { mount_path = "auth/approle" config = { role_id_file_path ="/root/vault_configs/kv-agent-role-id" secret_id_file_path ="/root/vault_configs/kv-agent-secret-id" remove_secret_id_file_after_reading ="false" } } } cache { use_auto_auth_token = true } listener "tcp" { address = "127.0.0.1:8100" tls_disable = true } vault { address = "http://10.8.128.9:8200" }
Use systemctl to run the persistent daemon:
Example
[root@host03 ~]# /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl
- A token file is populated with a valid token when the Vault agent runs.
Select a Vault secret engine, either Key/Value or Transit.
If using Key/Value, then add the following line:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine kv
If using Transit, then add the following line:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_secret_engine transit
Use the
ceph config set client.rgw OPTION VALUE
command to set the Vault namespace to retrieve the encryption keys:Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_namespace testnamespace1
Restrict where the Ceph Object Gateway retrieves the encryption keys from the Vault by setting a path prefix:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_prefix /v1/secret/data
For exportable Transit keys, set the prefix path as follows:
Example
[ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_vault_prefix /v1/transit/export/encryption-key
Assuming the domain name of the Vault server is
vault-server
, the Ceph Object Gateway will fetch encrypted transit keys from the following URL:Example
http://vault-server:8200/v1/transit/export/encryption-key
Restart the Ceph Object Gateway daemons.
To restart the Ceph Object Gateway on an individual node in the storage cluster:
Syntax
systemctl restart ceph-CLUSTER_ID@SERVICE_TYPE.ID.service
Example
[root@host03 ~]# systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service
To restart the Ceph Object Gateways on all nodes in the storage cluster:
Syntax
ceph orch restart SERVICE_TYPE
Example
[ceph: root@host03 /]# ceph orch restart rgw
Additional Resources
- See the Secret engines for Vault section of the Red Hat Ceph Storage Object Gateway Guide for more details.
- See the Authentication for Vault section of the Red Hat Ceph Storage Object Gateway Guide for more details.
8.4.7. Configuring the Ceph Object Gateway to use SSE-S3 with Vault
To configure the Ceph Object Gateway to use the HashiCorp Vault with SSE-S3 for key management, it must be set as the encryption key store. Currently, the Ceph Object Gateway only uses agent authentication method.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
- Root-level access to a Ceph Object Gateway node.
Procedure
Log into the Cephadm shell
Example
[root@host01 ~]# cephadm shell
Enable Vault as the secrets engine to retrieve SSE-S3 encryption keys:
Syntax
ceph config set client.rgw rgw_crypt_sse_s3_backend vault
To set the authentication method to use with SSE-S3 and Vault, configure the following settings:
Syntax
ceph config set client.rgw rgw_crypt_sse_s3_vault_auth agent ceph config set client.rgw rgw_crypt_sse_s3_vault_addr http://VAULT_AGENT:VAULT_AGENT_PORT
Example
[ceph: root@host01 ~]# ceph config set client.rgw rgw_crypt_sse_s3_vault_auth agent [ceph: root@host01 ~]# ceph config set client.rgw rgw_crypt_sse_s3_vault_addr http://vaultagent:8100
- Customize the policy as per your use case to set up a Vault agent.
Get the role-id:
Syntax
vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .rgw-ap-role-id > PATH_TO_FILE
Get the secret-id:
Syntax
vault read auth/approle/role/rgw-ap/role-id -format=json | \ jq -r .rgw-ap-secret-id > PATH_TO_FILE
Create the configuration for the Vault agent:
Example
pid_file = "/run/rgw-vault-agent-pid" auto_auth { method "AppRole" { mount_path = "auth/approle" config = { role_id_file_path ="/usr/local/etc/vault/.rgw-ap-role-id" secret_id_file_path ="/usr/local/etc/vault/.rgw-ap-secret-id" remove_secret_id_file_after_reading ="false" } } } cache { use_auto_auth_token = true } listener "tcp" { address = "127.0.0.1:8100" tls_disable = true } vault { address = "https://vaultserver:8200" }
Use systemctl to run the persistent daemon:
Example
[root@host01 ~]# /usr/local/bin/vault agent -config=/usr/local/etc/vault/rgw-agent.hcl
- A token file is populated with a valid token when the Vault agent runs.
Set the Vault secret engine to use to retrieve encryption keys, either Key/Value or Transit.
If using Key/Value, set the following:
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_secret_engine kv
If using Transit, set the following:
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_secret_engine transit
Optional: Configure the Ceph Object Gateway to access Vault within a particular namespace to retrieve the encryption keys:
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_namespace company/testnamespace1
NoteVault namespaces allow teams to operate within isolated environments known as tenants. The Vault namespaces feature is only available in the Vault Enterprise version.
Optional: Restrict access to a particular subset of the Vault secret space by setting a URL path prefix, where the Ceph Object Gateway retrieves the encryption keys from:
If using Key/Value, set the following:
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_prefix /v1/secret/data
If using Transit, set the following:
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_prefix /v1/transit
Assuming the domain name of the Vault server is
vaultserver
, the Ceph Object Gateway will fetch encrypted transit keys from the following URL:Example
http://vaultserver:8200/v1/transit
Optional: To use custom SSL certification to authenticate with Vault, configure the following settings:
Syntax
ceph config set client.rgw rgw_crypt_sse_s3_vault_verify_ssl true ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_cacert PATH_TO_CA_CERTIFICATE ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientcert PATH_TO_CLIENT_CERTIFICATE ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientkey PATH_TO_PRIVATE_KEY
Example
[ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_verify_ssl true [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_cacert /etc/ceph/vault.ca [ceph: root@host01 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientcert /etc/ceph/vault.crt [ceph: root@host03 /]# ceph config set client.rgw rgw_crypt_sse_s3_vault_ssl_clientkey /etc/ceph/vault.key
Restart the Ceph Object Gateway daemons.
To restart the Ceph Object Gateway on an individual node in the storage cluster:
Syntax
systemctl restart ceph-CLUSTER_ID@SERVICE_TYPE.ID.service
Example
[root@host01 ~]# systemctl restart ceph-c4b34c6f-8365-11ba-dc31-529020a7702d@rgw.realm.zone.host01.gwasto.service
To restart the Ceph Object Gateways on all nodes in the storage cluster:
Syntax
ceph orch restart SERVICE_TYPE
Example
[ceph: root@host01 /]# ceph orch restart rgw
Additional Resources
- See the Secret engines for Vault section of the Red Hat Ceph Storage Object Gateway Guide for more details.
- See the Authentication for Vault section of the Red Hat Ceph Storage Object Gateway Guide for more details.
8.4.8. Creating a key using the kv
engine
Configure the HashiCorp Vault Key/Value secret engine (kv
) so you can create a key for use with the Ceph Object Gateway. Secrets are stored as key-value pairs in the kv
secret engine.
Keys for server-side encryption must be 256-bits long and encoded using base64
.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the HashiCorp Vault software.
- Root-level access to the HashiCorp Vault node.
Procedure
Enable the Key/Value version 2 secret engine:
Example
vault secrets enable -path secret kv-v2
Create a new key:
Syntax
vault kv put secret/PROJECT_NAME/BUCKET_NAME key=$(openssl rand -base64 32)
Example
[root@vault ~]# vault kv put secret/myproject/mybucketkey key=$(openssl rand -base64 32) ====== Metadata ====== Key Value --- ---- created_time 2020-02-21T17:01:09.095824999Z deletion_time n/a destroyed false version 1
8.4.9. Creating a key using the transit engine
Configure the HashiCorp Vault Transit secret engine (transit
) so you can create a key for use with the Ceph Object Gateway. Creating keys with the Transit secret engine must be exportable in order to be used for server-side encryption with the Ceph Object Gateway.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the HashiCorp Vault software.
- Root-level access to the HashiCorp Vault node.
Procedure
Enable the Transit secret engine:
[root@vault ~]# vault secrets enable transit
Create a new exportable key:
Syntax
vault write -f transit/keys/BUCKET_NAME exportable=true
Example
[root@vault ~]# vault write -f transit/keys/mybucketkey exportable=true
NoteBy default the above command creates a
aes256-gcm96
type key.Verify the creation of the key:
Syntax
vault read transit/export/encryption-key/BUCKET_NAME/VERSION_NUMBER
Example
[root@vault ~]# vault read transit/export/encryption-key/mybucketkey/1 Key Value --- ----- keys map[1:-gbTI9lNpqv/V/2lDcmH2Nq1xKn6FPDWarCmFM2aNsQ=] name mybucketkey type aes256-gcm96
NoteProviding the full key path, including the key version, is required.
8.4.10. Uploading an object using AWS and the Vault
When uploading an object to the Ceph Object Gateway, the Ceph Object Gateway will fetch the key from the Vault, and then encrypt and store the object in a bucket. When a request is made to download the object, the Ceph Object Gateway will automatically retrieve the corresponding key from the Vault and decrypt the object. To upload an object, the Ceph object Gateway fetches the key from the Vault and then encrypts the object and stores it in the bucket. The Ceph Object Gateway retrieves the corresponding key from the Vault and decrypts the object when there is a request to download the object.
The URL is constructed using the base address, set by the rgw_crypt_vault_addr
option, and the path prefix, set by the rgw_crypt_vault_prefix
option.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Installation of the Ceph Object Gateway software.
- Installation of the HashiCorp Vault software.
- Access to a Ceph Object Gateway client node.
- Access to Amazon Web Services (AWS).
Procedure
Upload an object using the AWS command line client and provide the Secure Side Encryption (SSE) key ID in the request:
For the Key/Value secret engine:
Example (with SSE-KMS)
[user@client ~]$ aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id myproject/mybucketkey
Example (with SSE-S3)
[user@client ~]$ aws s3api --endpoint http://rgw_host:8080 put-object --bucket my-bucket --key obj1 --body test_file_to_upload --server-side-encryption AES256
NoteIn the example, the Ceph Object Gateway would fetch the secret from
http://vault-server:8200/v1/secret/data/myproject/mybucketkey
For the Transit engine:
Example (with SSE-KMS)
[user@client ~]$ aws --endpoint=http://radosgw:8000 s3 cp plaintext.txt s3://mybucket/encrypted.txt --sse=aws:kms --sse-kms-key-id mybucketkey
Example (with SSE-S3)
[user@client ~]$ aws s3api --endpoint http://rgw_host:8080 put-object --bucket my-bucket --key obj1 --body test_file_to_upload --server-side-encryption AES256
NoteIn the example, the Ceph Object Gateway would fetch the secret from
http://vaultserver:8200/v1/transit/mybucketkey
Additional Resources
- See the Install Vault documentation on Vault’s project site for more information.
8.5. The Ceph Object Gateway and multi-factor authentication
As a storage administrator, you can manage time-based one time password (TOTP) tokens for Ceph Object Gateway users.
8.5.1. Multi-factor authentication
When a bucket is configured for object versioning, a developer can optionally configure the bucket to require multi-factor authentication (MFA) for delete requests. Using MFA, a time-based one time password (TOTP) token is passed as a key to the x-amz-mfa
header. The tokens are generated with virtual MFA devices like Google Authenticator, or a hardware MFA device like those provided by Gemalto.
Use radosgw-admin
to assign time-based one time password tokens to a user. You must set a secret seed and a serial ID. You can also use radosgw-admin
to list, remove, and resynchronize tokens.
In a multi-site environment it is advisable to use different tokens for different zones, because, while MFA IDs are set on the user’s metadata, the actual MFA one time password configuration resides on the local zone’s OSDs.
Term | Description |
---|---|
TOTP | Time-based One Time Password. |
Token serial | A string that represents the ID of a TOTP token. |
Token seed | The secret seed that is used to calculate the TOTP. It can be hexadecimal or base32. |
TOTP seconds | The time resolution used for TOTP generation. |
TOTP window | The number of TOTP tokens that are checked before and after the current token when validating tokens. |
TOTP pin | The valid value of a TOTP token at a certain time. |
8.5.2. Creating a seed for multi-factor authentication
To set up multi-factor authentication (MFA), you must create a secret seed for use by the one-time password generator and the back-end MFA system.
Prerequisites
- A Linux system.
- Access to the command line shell.
Procedure
Generate a 30 character seed from the
urandom
Linux device file and store it in the shell variableSEED
:Example
[user@host01 ~]$ SEED=$(head -10 /dev/urandom | sha512sum | cut -b 1-30)
Print the seed by running echo on the
SEED
variable:Example
[user@host01 ~]$ echo $SEED 492dedb20cf51d1405ef6a1316017e
Configure the one-time password generator and the back-end MFA system to use the same seed.
Additional Resources
- For more information, see the solution Unable to create RGW MFA token for bucket.
- For more information, see The Ceph Object Gateway and multi-factor authentication.
8.5.3. Creating a new multi-factor authentication TOTP token
Create a new multi-factor authentication (MFA) time-based one time password (TOTP) token.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
- A secret seed for the one-time password generator and Ceph Object Gateway MFA was generated.
Procedure
Create a new MFA TOTP token:
Syntax
radosgw-admin mfa create --uid=USERID --totp-serial=SERIAL --totp-seed=SEED --totp-seed-type=SEED_TYPE --totp-seconds=TOTP_SECONDS --totp-window=TOTP_WINDOW
Set USERID to the user name to set up MFA on, set SERIAL to a string that represents the ID for the TOTP token, and set SEED to a hexadecimal or base32 value that is used to calculate the TOTP. The following settings are optional: Set the SEED_TYPE to
hex
orbase32
, set TOTP_SECONDS to the timeout in seconds, or set TOTP_WINDOW to the number of TOTP tokens to check before and after the current token when validating tokens.Example
[root@host01 ~]# radosgw-admin mfa create --uid=johndoe --totp-serial=MFAtest --totp-seed=492dedb20cf51d1405ef6a1316017e
Additional Resources
- For more information, see Creating a seed for multi-factor authentication.
- For more information, See Resynchronizing a multi-factor authentication token.
8.5.4. Test a multi-factor authentication TOTP token
Test a multi-factor authentication (MFA) time-based one time password (TOTP) token.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
-
An MFA TOTP token was created using
radosgw-admin mfa create
.
Procedure
Test the TOTP token PIN to verify that TOTP functions correctly:
Syntax
radosgw-admin mfa check --uid=USERID --totp-serial=SERIAL --totp-pin=PIN
Set USERID to the user name MFA is set up on, set SERIAL to the string that represents the ID for the TOTP token, and set PIN to the latest PIN from the one-time password generator.
Example
[root@host01 ~]# radosgw-admin mfa check --uid=johndoe --totp-serial=MFAtest --totp-pin=870305 ok
If this is the first time you have tested the PIN, it may fail. If it fails, resynchronize the token. See Resynchronizing a multi-factor authentication token in the Red Hat Ceph Storage Object Gateway Configuration and Administration Guide.
Additional Resources
- For more information, see Creating a seed for multi-factor authentication.
- For more information, see Resynchronizing a multi-factor authentication token.
8.5.5. Resynchronizing a multi-factor authentication TOTP token
Resynchronize a multi-factor authentication (MFA) time-based one time password token.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
-
An MFA TOTP token was created using
radosgw-admin mfa create
.
Procedure
Resynchronize a multi-factor authentication TOTP token in case of time skew or failed checks.
This requires passing in two consecutive pins: the previous pin, and the current pin.
Syntax
radosgw-admin mfa resync --uid=USERID --totp-serial=SERIAL --totp-pin=PREVIOUS_PIN --totp=pin=CURRENT_PIN
Set USERID to the user name MFA is set up on, set SERIAL to the string that represents the ID for the TOTP token, set PREVIOUS_PIN to the user’s previous PIN, and set CURRENT_PIN to the user’s current PIN.
Example
[root@host01 ~]# radosgw-admin mfa resync --uid=johndoe --totp-serial=MFAtest --totp-pin=802021 --totp-pin=439996
Verify the token was successfully resynchronized by testing a new PIN:
Syntax
radosgw-admin mfa check --uid=USERID --totp-serial=SERIAL --totp-pin=PIN
Set USERID to the user name MFA is set up on, set SERIAL to the string that represents the ID for the TOTP token, and set PIN to the user’s PIN.
Example
[root@host01 ~]# radosgw-admin mfa check --uid=johndoe --totp-serial=MFAtest --totp-pin=870305 ok
Additional Resources
- For more information, see Creating a new multi-factor authentication TOTP token.
8.5.6. Listing multi-factor authentication TOTP tokens
List all multi-factor authentication (MFA) time-based one time password (TOTP) tokens that a particular user has.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
-
An MFA TOTP token was created using
radosgw-admin mfa create
.
Procedure
List MFA TOTP tokens:
Syntax
radosgw-admin mfa list --uid=USERID
Set USERID to the user name MFA is set up on.
Example
[root@host01 ~]# radosgw-admin mfa list --uid=johndoe { "entries": [ { "type": 2, "id": "MFAtest", "seed": "492dedb20cf51d1405ef6a1316017e", "seed_type": "hex", "time_ofs": 0, "step_size": 30, "window": 2 } ] }
Additional Resources
- For more information, see Creating a new multi-factor authentication TOTP token.
8.5.7. Display a multi-factor authentication TOTP token
Display a specific multi-factor authentication (MFA) time-based one time password (TOTP) token by specifying its serial.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
-
An MFA TOTP token was created using
radosgw-admin mfa create
.
Procedure
Show the MFA TOTP token:
Syntax
radosgw-admin mfa get --uid=USERID --totp-serial=SERIAL
Set USERID to the user name MFA is set up on and set SERIAL to the string that represents the ID for the TOTP token.
Additional Resources
- For more information, see Creating a new multi-factor authentication TOTP token.
8.5.8. Deleting a multi-factor authentication TOTP token
Delete a multi-factor authentication (MFA) time-based one time password (TOTP) token.
Prerequisites
- A running Red Hat Ceph Storage cluster.
- Ceph Object Gateway is installed.
- You have root access on a Ceph Monitor node.
-
An MFA TOTP token was created using
radosgw-admin mfa create
.
Procedure
Delete an MFA TOTP token:
Syntax
radosgw-admin mfa remove --uid=USERID --totp-serial=SERIAL
Set USERID to the user name MFA is set up on and set SERIAL to the string that represents the ID for the TOTP token.
Example
[root@host01 ~]# radosgw-admin mfa remove --uid=johndoe --totp-serial=MFAtest
Verify the MFA TOTP token was deleted:
Syntax
radosgw-admin mfa get --uid=USERID --totp-serial=SERIAL
Set USERID to the user name MFA is set up on and set SERIAL to the string that represents the ID for the TOTP token.
Example
[root@host01 ~]# radosgw-admin mfa get --uid=johndoe --totp-serial=MFAtest MFA serial id not found
Additional Resources
- For more information, see The Ceph Object Gateway and multi-factor authentication.