Chapter 6. Advanced configuration

1. Extract/Download a PEM-formatted certificate for the Certificate Authority (CA) that signed the LDAP server’s SSL certificate.
2. Confirm that /etc/openldap/ldap.conf does not have TLS_REQCERT set.
3. Confirm that /etc/openldap/ldap.conf contains a TLS_CACERTDIR /etc/openldap/certs setting.

4. Use the certutil command to add the AD CA to the store at /etc/openldap/certs. For example, if the CA is "msad-frog-MSAD-FROG-CA", and the PEM-formatted CA file is ldap.pem, use the following command:

   Example

   # certutil -d /etc/openldap/certs -A -t "TC,," -n "msad-frog-MSAD-FROG-CA" -i /path/to/ldap.pem

5. Update SELinux on all remote LDAP sites:

   Example

   # setsebool -P httpd_can_network_connect on

   Note

   This still has to be set even if SELinux is in permissive mode.

6. Make the certs database world-readable:

   Example

   # chmod 644 /etc/openldap/certs/*

7. Connect to the server using the "ldapwhoami" command as a non-root user.

   Example

   $ ldapwhoami -H ldaps://rh-directory-server.example.com -d 9

   The -d 9 option will provide debugging information in case something went wrong with the SSL negotiation.

Procedure

1. Create an LDAP user for the Ceph Object Gateway, and make a note of the binddn. Since the Ceph object gateway uses the ceph user, consider using ceph as the username. The user needs to have permissions to search the directory. The Ceph Object Gateway binds to this user as specified in rgw_ldap_binddn.

2. Test to ensure that the user creation worked. Where ceph is the user ID under People and example.com is the domain, you can perform a search for the user.

   # ldapsearch -x -D "uid=ceph,ou=People,dc=example,dc=com" -W -H ldaps://example.com -b "ou=People,dc=example,dc=com" -s sub 'uid=ceph'

3. On each gateway node, create a file for the user’s secret. For example, the secret might get stored in a file entitled /etc/bindpass. For security, change the owner of this file to the ceph user and group to ensure it is not globally readable.

4. Add the rgw_ldap_secret option:

   Syntax

   ceph config set client.rgw OPTION VALUE

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_ldap_secret /etc/bindpass

5. Patch the bind password file to the Ceph Object Gateway container and reapply the Ceph Object Gateway specification:

   Example

   service_type: rgw
   service_id: rgw.1
   service_name: rgw.rgw.1
   placement:
     label: rgw
     extra_container_args:
     - -v
     - /etc/bindpass:/etc/bindpass

   Note

   /etc/bindpass is not shipped automatically with Red Hat Ceph Storage and you need to ensure that the content is available on all the possible Ceph Object Gateway instance nodes.

1. Specifying a partial filter:

   Example

   "objectclass=inetorgperson"

   The Ceph Object Gateway generates the search filter with the user name from the token and the value of rgw_ldap_dnattr. The constructed filter is then combined with the partial filter from the rgw_ldap_searchfilter value. For example, the user name and the settings generate the final search filter:

   Example

   "(&(uid=joe)(objectclass=inetorgperson))"

   User joe is only granted access if he is found in the LDAP directory, has an object class of inetorgperson, and specifies a valid password.

2. Specifying a complete filter:

   A complete filter must contain a USERNAME token which is substituted with the user name during the authentication attempt. The rgw_ldap_dnattr setting is not used in this case. For example, to limit valid users to a specific group, use the following filter:

   Example

   "(&(uid=@USERNAME@)(memberOf=cn=ceph-users,ou=groups,dc=mycompany,dc=com))"

1. Export the access key:

   Syntax

   export RGW_ACCESS_KEY_ID="USERNAME"

2. Export the secret key:

   Syntax

   export RGW_SECRET_ACCESS_KEY="PASSWORD"

3. Export the token. For LDAP, use ldap as the token type (ttype).

   Example

   radosgw-token --encode --ttype=ldap

   For Active Directory, use ad as the token type.

   Example

   radosgw-token --encode --ttype=ad

   The result is a base-64 encoded string, which is the access token. Provide this access token to S3 clients in lieu of the access key. The secret key is no longer required.

4. Optional: For added convenience, export the base-64 encoded string to the RGW_ACCESS_KEY_ID environment variable if the S3 client uses the environment variable.

   Example

   export RGW_ACCESS_KEY_ID="ewogICAgIlJHV19UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAibGRhcCIsCiAgICAgICAgImlkIjogImNlcGgiLAogICAgICAgICJrZXkiOiAiODAwI0dvcmlsbGEiCiAgICB9Cn0K"

Procedure

1. Use the RGW_ACCESS_KEY_ID environment variable to configure the Ceph Object Gateway client. Alternatively, you can copy the base-64 encoded string and specify it as the access key. Following is an example of the configured S3 client:

   Example

   cat .aws/credentials
   
   [default]
   aws_access_key_id = ewogICaGbnjlwe9UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAiYWQiLAogICAgICAgICJpZCI6ICJjZXBoIiwKICAgICAgICAia2V5IjogInBhc3M0Q2VwaCIKICAgIH0KfQo=
   aws_secret_access_key =

   Note

   The secret key is no longer required.

2. Run the aws s3 ls command to verify the user:

   Example

   [root@host01 ~]# aws s3 ls --endpoint http://host03
   
   2023-12-11 17:08:50 mybucket
   2023-12-24 14:55:44 mybucket2

3. Optional: You can also run the radosgw-admin user command to verify the user in the directory:

   Example

   [root@host01 ~]# radosgw-admin user info --uid dir1
   {
       "user_id": "dir1",
       "display_name": "dir1",
       "email": "",
       "suspended": 0,
       "max_buckets": 1000,
       "subusers": [],
       "keys": [],
       "swift_keys": [],
       "caps": [],
       "op_mask": "read, write, delete",
       "default_placement": "",
       "default_storage_class": "",
       "placement_tags": [],
       "bucket_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "user_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "temp_url_keys": [],
       "type": "ldap",
       "mfa_ids": []
   }

Procedure

1. Create an LDAP user for the Ceph Object Gateway, and make a note of the binddn. Since the Ceph object gateway uses the ceph user, consider using ceph as the username. The user needs to have permissions to search the directory. The Ceph Object Gateway binds to this user as specified in rgw_ldap_binddn.

2. Test to ensure that the user creation worked. Where ceph is the user ID under People and example.com is the domain, you can perform a search for the user.

   # ldapsearch -x -D "uid=ceph,ou=People,dc=example,dc=com" -W -H ldaps://example.com -b "ou=People,dc=example,dc=com" -s sub 'uid=ceph'

3. On each gateway node, create a file for the user’s secret. For example, the secret might get stored in a file entitled /etc/bindpass. For security, change the owner of this file to the ceph user and group to ensure it is not globally readable.

4. Add the rgw_ldap_secret option:

   Syntax

   ceph config set client.rgw OPTION VALUE

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_ldap_secret /etc/bindpass

5. Patch the bind password file to the Ceph Object Gateway container and reapply the Ceph Object Gateway specification:

   Example

   service_type: rgw
   service_id: rgw.1
   service_name: rgw.rgw.1
   placement:
     label: rgw
     extra_container_args:
     - -v
     - /etc/bindpass:/etc/bindpass

   Note

   /etc/bindpass is not shipped automatically with Red Hat Ceph Storage and you need to ensure that the content is available on all the possible Ceph Object Gateway instance nodes.

1. Export the access key:

   Syntax

   export RGW_ACCESS_KEY_ID="USERNAME"

2. Export the secret key:

   Syntax

   export RGW_SECRET_ACCESS_KEY="PASSWORD"

3. Export the token. For LDAP, use ldap as the token type (ttype).

   Example

   radosgw-token --encode --ttype=ldap

   For Active Directory, use ad as the token type.

   Example

   radosgw-token --encode --ttype=ad

   The result is a base-64 encoded string, which is the access token. Provide this access token to S3 clients in lieu of the access key. The secret key is no longer required.

4. Optional: For added convenience, export the base-64 encoded string to the RGW_ACCESS_KEY_ID environment variable if the S3 client uses the environment variable.

   Example

   export RGW_ACCESS_KEY_ID="ewogICAgIlJHV19UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAibGRhcCIsCiAgICAgICAgImlkIjogImNlcGgiLAogICAgICAgICJrZXkiOiAiODAwI0dvcmlsbGEiCiAgICB9Cn0K"

Procedure

1. Use the RGW_ACCESS_KEY_ID environment variable to configure the Ceph Object Gateway client. Alternatively, you can copy the base-64 encoded string and specify it as the access key. Following is an example of the configured S3 client:

   Example

   cat .aws/credentials
   
   [default]
   aws_access_key_id = ewogICaGbnjlwe9UT0tFTiI6IHsKICAgICAgICAidmVyc2lvbiI6IDEsCiAgICAgICAgInR5cGUiOiAiYWQiLAogICAgICAgICJpZCI6ICJjZXBoIiwKICAgICAgICAia2V5IjogInBhc3M0Q2VwaCIKICAgIH0KfQo=
   aws_secret_access_key =

   Note

   The secret key is no longer required.

2. Run the aws s3 ls command to verify the user:

   Example

   [root@host01 ~]# aws s3 ls --endpoint http://host03
   
   2023-12-11 17:08:50 mybucket
   2023-12-24 14:55:44 mybucket2

3. Optional: You can also run the radosgw-admin user command to verify the user in the directory:

   Example

   [root@host01 ~]# radosgw-admin user info --uid dir1
   {
       "user_id": "dir1",
       "display_name": "dir1",
       "email": "",
       "suspended": 0,
       "max_buckets": 1000,
       "subusers": [],
       "keys": [],
       "swift_keys": [],
       "caps": [],
       "op_mask": "read, write, delete",
       "default_placement": "",
       "default_storage_class": "",
       "placement_tags": [],
       "bucket_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "user_quota": {
           "enabled": false,
           "check_on_raw": false,
           "max_size": -1,
           "max_size_kb": 0,
           "max_objects": -1
       },
       "temp_url_keys": [],
       "type": "ldap",
       "mfa_ids": []
   }

Procedure

1. List the endpoints under the Swift service:

   [root@swift~]# openstack endpoint list --service=swift

2. Verify settings for the endpoints listed in the previous command:

   Syntax

   [root@swift~]# openstack endpoint show ENDPOINT_ID

   Showing the endpoints will echo the endpoints settings, and the service settings.

   Table 6.2. Example

   Field	Value
   adminurl	http://radosgw.example.com:8080/swift/v1
   enabled	True
   id	e4249d2b60e44743a67b5e5b38c18dd3
   internalurl	http://radosgw.example.com:8080/swift/v1
   publicurl	http://radosgw.example.com:8080/swift/v1
   region	us-west
   service_id	37c4c0e79571404cb4644201a4a6e5ee
   service_name	swift
   service_type	object-store

Procedure

1. Convert the OpenSSL certificate to the nss db format:

   Example

   [root@osp ~]# mkdir /var/ceph/nss
   
   [root@osp ~]# openssl x509 -in /etc/keystone/ssl/certs/ca.pem -pubkey | \
       certutil -d /var/ceph/nss -A -n ca -t "TCu,Cu,Tuw"
   
   [root@osp ~]# openssl x509 -in /etc/keystone/ssl/certs/signing_cert.pem -pubkey | \
       certutil -A -d /var/ceph/nss -n signing_cert -t "P,P,P"

2. Install Keystone’s SSL certificate in the node running the Ceph Object Gateway. Alternatively set the value of the configurable rgw_keystone_verify_ssl setting to false.

   Setting rgw_keystone_verify_ssl to false means that the gateway will not attempt to verify the certificate.

Procedure

1. Do the following for each gateway instance.

   1. Set the nss_db_path setting to the path where the NSS database is stored:

      Example

      [ceph: root@host01 /]# ceph config set client.rgw nss_db_path "/var/lib/ceph/radosgw/ceph-rgw.rgw01/nss"

2. Provide authentication credentials:

   It is possible to configure a Keystone service tenant, user, and password for keystone for the OpenStack Identity API, similar to the way system administrators tend to configure OpenStack services. Providing a username and password avoids providing the shared secret to the rgw_keystone_admin_token setting.

   Important

   Red Hat recommends disabling authentication by admin token in production environments. The service tenant credentials should have admin privileges.

   The necessary configuration options are:

   Syntax

   ceph config set client.rgw rgw_keystone_verify_ssl TRUE/FALSE
   ceph config set client.rgw rgw_s3_auth_use_keystone TRUE/FALSE
   ceph config set client.rgw rgw_keystone_api_version API_VERSION
   ceph config set client.rgw rgw_keystone_url KEYSTONE_URL:ADMIN_PORT
   ceph config set client.rgw rgw_keystone_accepted_roles ACCEPTED_ROLES_
   ceph config set client.rgw rgw_keystone_accepted_admin_roles ACCEPTED_ADMIN_ROLES
   ceph config set client.rgw rgw_keystone_admin_domain default
   ceph config set client.rgw rgw_keystone_admin_project SERVICE_NAME
   ceph config set client.rgw rgw_keystone_admin_user KEYSTONE_TENANT_USER_NAME
   ceph config set client.rgw rgw_keystone_admin_password KEYSTONE_TENANT_USER_PASSWORD
   ceph config set client.rgw rgw_keystone_implicit_tenants KEYSTONE_IMPLICIT_TENANT_NAME
   ceph config set client.rgw rgw_swift_versioning_enabled TRUE/FALSE
   ceph config set client.rgw rgw_swift_enforce_content_length TRUE/FALSE
   ceph config set client.rgw rgw_swift_account_in_url TRUE/FALSE
   ceph config set client.rgw rgw_trust_forwarded_https TRUE/FALSE
   ceph config set client.rgw rgw_max_attr_name_len MAXIMUM_LENGTH_OF_METADATA_NAMES
   ceph config set client.rgw rgw_max_attrs_num_in_req MAXIMUM_NUMBER_OF_METADATA_ITEMS
   ceph config set client.rgw rgw_max_attr_size MAXIMUM_LENGTH_OF_METADATA_VALUE
   ceph config set client.rgw rgw_keystone_accepted_reader_roles SwiftSystemReader

   Example

   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_verify_ssl false
   [ceph: root@host01 /]# ceph config set client.rgw rgw_s3_auth_use_keystone true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_api_version 3
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_url http://<public Keystone endpoint>:5000/
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_accepted_roles 'member, Member, admin'
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_accepted_admin_roles 'ResellerAdmin, swiftoperator'
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_admin_domain default
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_admin_project service
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_admin_user swift
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_admin_password password
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_implicit_tenants true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_swift_versioning_enabled  true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_swift_enforce_content_length true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_swift_account_in_url true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_trust_forwarded_https true
   [ceph: root@host01 /]# ceph config set client.rgw rgw_max_attr_name_len 128
   [ceph: root@host01 /]# ceph config set client.rgw rgw_max_attrs_num_in_req 90
   [ceph: root@host01 /]# ceph config set client.rgw rgw_max_attr_size  1024
   [ceph: root@host01 /]# ceph config set client.rgw rgw_keystone_accepted_reader_roles SwiftSystemReader

   A Ceph Object Gateway user is mapped into a Keystone tenant. A Keystone user has different roles assigned to it on possibly more than a single tenant. When the Ceph Object Gateway gets the ticket, it looks at the tenant, and the user roles that are assigned to that ticket, and accepts or rejects the request according to the rgw_keystone_accepted_roles configurable.