Chapter 2. Preparing HashiCorp Vault for the Record Encryption filter

Deploying Vault using the example Helm deployment configuration

1. Download and extract the Streams for Apache Kafka Proxy installation artifacts.

   The proxy is available from Streams for Apache Kafka software downloads page.

   The files contain the deployment configuration required for deploying Vault.

2. Create a root token and make a note of it:

   cat /dev/urandom | LC_ALL=C tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1 > vault.root.token
   export VAULT_TOKEN=$(cat vault.root.token)

3. Install Vault using Helm:

   helm repo add hashicorp https://helm.releases.hashicorp.com
   helm install vault hashicorp/vault \
     --create-namespace --namespace=vault \
     --version <helm_version> \
     --values vault/helm-dev-values.yaml \
     --set server.dev.devRootToken=${VAULT_TOKEN} \
     --wait

   The root token is used for the Vault instance.

4. Check the status of the deployment:

   oc get pods -n vault

   Output shows the deployment name and readiness

   NAME                      READY  STATUS   RESTARTS
   vault-0                   1/1    Running  0

   A pod ID identifies the pod created.

   With the default deployment, you install a single proxy pod.

   READY shows the number of replicas that are ready/expected. The deployment is successful when the STATUS displays as Running.

5. Create a Vault address (VAULT_ADDR) environment variable to point to the new Vault instance:

   export VAULT_ADDR=$(oc get route -n vault vault --template='https://{{.spec.host}}')

6. Login to Vault as an administrator and enable the Vault Transit secrets engine:

   vault secrets enable transit

   If the secrets engine is already enabled, ignore the error.

7. Create an environment variable to point to the Vault Transit address:

   export VAULT_TRANSIT_URL=${VAULT_ADDR}/v1/transit

   The address is used in the proxy deployment configuration.

8. Create a Vault policy and token.

1. Create a Vault address environment variable (VAULT_ADDR and VAULT_NAMESPACE, if using Enterprise) to point to the Vault instance:

   export VAULT_ADDR=https://<vault server>:8200
   export VAULT_NAMESPACE=<namespaces>

2. Login to Vault as an administrator and enable the Vault Transit secrets engine:

   vault secrets enable transit

   If the secrets engine is already enabled, ignore the error.

3. Create an environment variable to point to the Vault Transit address:

   export VAULT_TRANSIT_URL=${VAULT_ADDR}/v1/${VAULT_NAMESPACE}/transit

   The address is used in the proxy deployment configuration.

4. Update the proxy deployment configuration to refer to your Vault instance:

   sed -i "s/\(vaultTransitEngineUrl:\).*$/\1 ${VAULT_TRANSIT_URL}/" */proxy/proxy-config.yaml

5. Create a Vault policy and token.

1. Create a Vault policy:

   vault policy write amqstreams_proxy_encryption_filter_policy vault/amqstreams_proxy_encryption_filter_policy.hcl

   Write the policy to Vault using the HashiCorp policy definition file (.hcl) provided with Streams for Apache Kafka Proxy. The policy is named amqstreams_proxy_encryption_filter_policy.

2. Create a Vault token:

   vault token create \
     -display-name "amqstreams-proxy encryption filter" \
     -policy=amqstreams_proxy_encryption_filter_policy \
     -no-default-policy \
     -orphan \
     -field=token > vault.encryption.token

   The command creates a token with the specified policy, and with no associated parent token or default policies.

3. Create a secret containing the token:

   oc create secret generic proxy-encryption-vault-token \
     -n proxy \
     --from-file=encryption-vault-token.txt=vault.encryption.token \
     --dry-run=client \
     -o yaml > base/proxy/proxy-encryption-vault-token-secret.yaml

   The command stores the Vault token in the secret and creates the secret as a YAML file in the proxy namespace.

   The proxy-encryption-vault-token-secret.yaml secret is applied to the OpenShift cluster when deploying Streams for Apache Kafka Proxy with the Record Encryption filter.