Chapter 5. Storage classes and storage pools

• The OpenShift Data Foundation cluster is in Ready state.

• On the external key management system (KMS),

  • Ensure that a policy with a token exists and the key value backend path in Vault is enabled. For more information, see Enabling key value and policy in Vault.
  • Ensure that you are using signed certificates on your Vault servers.

• Create a secret in the tenant’s namespace as follows:

  • On the OpenShift Container Platform web console, navigate to Workloads Secrets.
  • Click Create Key/value secret.
  • Enter Secret Name as ceph-csi-kms-token.
  • Enter Key as token.
  • Enter Value. It is the token from Vault. You can either click Browse to select and upload the file containing the token or enter the token directly in the text box.
  • Click Create.

• The OpenShift Data Foundation cluster is in Ready state.

• On the external key management system (KMS),

  • Ensure that a policy exists and the key value backend path in Vault is enabled. For more information, see Enabling key value and policy in Vault.
  • Ensure that you are using signed certificates on your Vault servers.

• Create the following serviceaccount in the tenant namespace as shown below:

  $ cat <<EOF | oc create -f -
  apiVersion: v1
  kind: ServiceAccount
  metadata:
      name: ceph-csi-vault-sa
  EOF

• The Kubernetes authentication method must be configured before OpenShift Data Foundation can authenticate with and start using Vault. The instructions below create and configure serviceAccount, ClusterRole, and ClusterRoleBinding required to allow OpenShift Data Foundation to authenticate with Vault.

  1. Apply the following YAML to your Openshift cluster:

     apiVersion: v1
     kind: ServiceAccount
     metadata:
       name: rbd-csi-vault-token-review
     ---
     kind: ClusterRole
     apiVersion: rbac.authorization.k8s.io/v1
     metadata:
       name: rbd-csi-vault-token-review
     rules:
       - apiGroups: ["authentication.k8s.io"]
         resources: ["tokenreviews"]
         verbs: ["create", "get", "list"]
     
     ---
     kind: ClusterRoleBinding
     apiVersion: rbac.authorization.k8s.io/v1
     metadata:
       name: rbd-csi-vault-token-review
     subjects:
       - kind: ServiceAccount
         name: rbd-csi-vault-token-review
         namespace: openshift-storage
     roleRef:
       kind: ClusterRole
       name: rbd-csi-vault-token-review
       apiGroup: rbac.authorization.k8s.io

  2. Identify the secret name associated with the serviceaccount (SA) created above:

     $ oc -n openshift-storage get sa rbd-csi-vault-token-review -o jsonpath="{.secrets[*]['name']}"

  3. Get the token and the CA certificate from the secret:

     $ oc get secret <secret associated with SA> -o jsonpath="{.data['token']}" | base64 --decode; echo
     $ oc get secret <secret associated with SA> -o jsonpath="{.data['ca\.crt']}" | base64 --decode; echo

  4. Retrieve the OCP cluster endpoint:

     $ oc config view --minify --flatten -o jsonpath="{.clusters[0].cluster.server}"

  5. Use the information collected in the steps above to setup the kubernetes authentication method in Vault as shown below:

     $ vault auth enable kubernetes
     $ vault write auth/kubernetes/config token_reviewer_jwt=<SA token> kubernetes_host=<OCP cluster endpoint> kubernetes_ca_cert=<SA CA certificate>

  6. Create a role in Vault for the tenant namespace:

     $ vault write "auth/kubernetes/role/csi-kubernetes" bound_service_account_names="ceph-csi-vault-sa" bound_service_account_namespaces=<tenant_namespace> policies=<policy_name_in_vault>

     csi-kubernetes is the default role name that OpenShift Data Foundation looks for in Vault. The default service account name in the tenant namespace in the Openshift Data Foundation cluster is ceph-csi-vault-sa. These default values can be overridden by creating a ConfigMap in the tenant namespace.

     For more information about overriding the default names, see Overriding Vault connection details using tenant ConfigMap.

• In order to create a storageclass that uses the vaulttenantsa method for PV encrytpion, you must either edit the existing ConfigMap or create a ConfigMap named csi-kms-connection-details that will hold all the information needed to establish the connection with Vault.

  The sample yaml given below can be used to update or create the csi-kms-connection-detail ConfigMap:

  apiVersion: v1
  data:
    vault-tenant-sa: |-
      {
        "encryptionKMSType": "vaulttenantsa",
        "vaultAddress": "<https://hostname_or_ip_of_vault_server:port>",
        "vaultTLSServerName": "<vault TLS server name>",
        "vaultAuthPath": "/v1/auth/kubernetes/login",
        "vaultAuthNamespace": "<vault auth namespace name>"
        "vaultNamespace": "<vault namespace name>",
        "vaultBackendPath": "<vault backend path name>",
        "vaultCAFromSecret": "<secret containing CA cert>",
        "vaultClientCertFromSecret": "<secret containing client cert>",
        "vaultClientCertKeyFromSecret": "<secret containing client private key>",
        "tenantSAName": "<service account name in the tenant namespace>"
      }
  metadata:
    name: csi-kms-connection-details

  • encryptionKMSType: should be set to vaulttenantsa to use service accounts for authentication with vault.
  • vaultAddress: The hostname or IP address of the vault server with the port number.
  • vaultTLSServerName: (Optional) The vault TLS server name
  • vaultAuthPath: (Optional) The path where kubernetes auth method is enabled in Vault. The default path is kubernetes. If the auth method is enabled in a different path other than kubernetes, this variable needs to be set as "/v1/auth/<path>/login".
  • vaultAuthNamespace: (Optional) The Vault namespace where kubernetes auth method is enabled.
  • vaultNamespace: (Optional) The Vault namespace where the backend path being used to store the keys exists
  • vaultBackendPath: The backend path in Vault where the encryption keys will be stored
  • vaultCAFromSecret: The secret in the OpenShift Data Foundation cluster containing the CA certificate from Vault
  • vaultClientCertFromSecret: The secret in the OpenShift Data Foundation cluster containing the client certificate from Vault
  • vaultClientCertKeyFromSecret: The secret in the OpenShift Data Foundation cluster containing the client private key from Vault
  • tenantSAName: (Optional) The service account name in the tenant namespace. The default value is ceph-csi-vault-sa. If a different name is to be used, this variable has to be set accordingly.

Next steps

• The storage class can be used to create encrypted persistent volumes. For more information, see managing persistent volume claims.

  Important

  Red Hat works with the technology partners to provide this documentation as a service to the customers. However, Red Hat does not provide support for the HashiCorp product. For technical assistance with this product, contact HashiCorp.