Chapter 17. Using a Vault to Obtain Secrets
Several fields in the administration support obtaining the value of a secret from an external vault.
To obtain a secret from a vault instead of entering it directly, enter the following specially crafted string into the appropriate field: ${vault.key}
where you replace the key
with the name of the secret as recognized by the vault.
In order to prevent secrets from leaking across realms, implementations may combine the realm name with the key
obtained from the vault expression. This means that the key
won’t directly map to an entry in the vault, but rather be used to create the final entry name according to the algorithm used to combine it with the realm name.
Currently, the secret can be obtained from the vault in the following fields:
- SMTP password
- In realm SMTP settings
- LDAP bind credential
- In LDAP settings of LDAP-based user federation.
- OIDC identity provider secret
- In Client Secret inside identity provider OpenID Connect Config
To use a vault, a vault provider must be registered within Red Hat Single Sign-On. It is possible to either use a built-in provider described below or implement your own provider. See the Server Developer Guide for more information.
There is at most one vault provider active per Red Hat Single Sign-On instance at any given time, and the vault provider in each instance within the cluster has to be configured consistently.
17.1. Kubernetes / OpenShift Files Plaintext Vault Provider
Red Hat Single Sign-On supports vault implementation for Kubernetes secrets. These secrets can be mounted as data volumes, and they appear as a directory with a flat file structure, where each secret is represented by a file whose name is the secret name, and contents of that file is the secret value.
The files within this directory have to be named as secret name prefixed by realm name and an underscore. All underscores within the secret name or the realm name have to be doubled in the file name. For example, for a field within a realm called sso_realm
, a reference to a secret with name secret-name
would be written as ${vault.secret-name}
, and the file name looked up would be sso__realm_secret-name
(note the underscore doubled in realm name).
To use this type of secret store, you have to declare the files-plaintext
vault provider in standalone.xml, and set its parameter for the directory that contains the mounted volume. The following example shows the files-plaintext
provider with the directory where vault files are searched for set to standalone/configuration/vault
relative to Red Hat Single Sign-On base directory:
<spi name="vault"> <default-provider>files-plaintext</default-provider> <provider name="files-plaintext" enabled="true"> <properties> <property name="dir" value="${jboss.home.dir}/standalone/configuration/vault/" /> </properties> </provider> </spi>
Here is the equivalent configuration using CLI commands:
/subsystem=keycloak-server/spi=vault/:add /subsystem=keycloak-server/spi=vault/provider=files-plaintext/:add(enabled=true,properties={dir => "${jboss.home.dir}/standalone/configuration/vault"})
17.2. Elytron Credential Store Vault Provider
Red Hat Single Sign-On also provides support for reading secrets stored in an Elytron credential store. The elytron-cs-keystore
vault provider is capable of retrieving secrets from the keystore-based implementation of the credential store, which is also the default implementation provided by Elytron.
This credential store is backed by a keystore (JCEKS
is the default format, but it is possible to use other formats such as PKCS12
) and users can create and manage the store contents using either the elytron
subsystem in WildFly/JBoss EAP, or using the elytron-tool.sh
script.
To use this provider, you have to declare the elytron-cs-keystore
in the keycloak-server
subsystem and set the location and master secret of the keystore that was created by Elytron. An example of the minimal configuration for the provider follows:
<spi name="vault"> <default-provider>elytron-cs-keystore</default-provider> <provider name="elytron-cs-keystore" enabled="true"> <properties> <property name="location" value="${jboss.home.dir}/standalone/configuration/vault/credential-store.jceks" /> <property name="secret" value="secretpw1!"/> </properties> </provider> </spi>
If the underlying keystore has a format other than JCEKS
, this format has to be informed using the keyStoreType
:
<spi name="vault"> <default-provider>elytron-cs-keystore</default-provider> <provider name="elytron-cs-keystore" enabled="true"> <properties> <property name="location" value="${jboss.home.dir}/standalone/configuration/vault/credential-store.p12" /> <property name="secret" value="secretpw1!"/> <property name="keyStoreType" value="PKCS12"/> </properties> </provider> </spi>
For the secret, the elytron-cs-keystore
provider supports both clear-text values (as shown above) and also values that were masked using the elytron-tool.sh
script:
<spi name="vault"> ... <property name="secret" value="MASK-3u2HNQaMogJJ8VP7J6gRIl;12345678;321"/> ... </spi>
For more detailed information on how to create/manage elytron credential stores, as well as how to mask keystore secrets, please refer to the Elytron documentation.
The elytron-cs-keystore
vault provider has been implemented as a WildFly extension and as such is only available if the Red Hat Single Sign-On server runs on WildFly/JBoss EAP.
17.3. Key Resolvers
All built-in providers support the configuration of one or more key resolvers. A key resolver essentially implements the algorithm or strategy for combining the realm name with the key (as obtained from the ${vault.key}
expression} into the final entry name that will be used to retrieve the secret from the vault. The keyResolvers
property is used to configure the resolvers that are to be used by the provider. The value is a comma-separated list of resolver names. An example of configuration for the files-plaintext
provider follows:
<spi name="vault"> <default-provider>files-plaintext</default-provider> <provider name="files-plaintext" enabled="true"> <properties> <property name="dir" value="${jboss.home.dir}/standalone/configuration/vault/" /> <property name="keyResolvers" value="REALM_UNDERSCORE_KEY, KEY_ONLY"/> </properties> </provider> </spi>
The resolvers are executed in the same order that they are declared in the configuration. For each resolver, the final entry name produced by the resolver that combines the realm with the vault key is used to search for the secret in the vault. If a secret is found, it is immediately returned. If not, the next resolver is used and this continues until a non-empty secret is found or all resolvers have been tried, in which case an empty secret is returned. In the example above, first the REALM_UNDERSCORE_KEY
resolver is used. If an entry is found in the vault with the name it produces, it is returned. If not, then the KEY_ONLY
resolver is used. Again, if an entry is found in the vault with the name it produces, it is returned. If not, an empty secret is returned since there are no more resolvers to be used.
A list of the currently available resolvers follows:
-
KEY_ONLY
: the realm name is ignored and the key from the vault expression is used as is. -
REALM_UNDERSCORE_KEY
: the realm and key are combined using an underscore_
character. Occurrences of underscore in either the realm or key are escaped by another underscore character. So if the realm is calledmaster_realm
and the key issmtp_key
, the combined key will bemaster__realm_smtp__key
. -
REALM_FILESEPARATOR_KEY
: the realm and key are combined using the platform file separator character. This is useful in situations where the keys are grouped by realm using a directory structure.
If no resolver is configured for the built-in providers, the REALM_UNDERSCORE_KEY
is selected by default.
17.4. Sample Configuration
The following is an example of configuring a vault and credential store. The procedure involves two parts:
- Creating the credential store and a vault, where the credential store and vault passwords are in plain text.
-
Updating the credential store and vault to have the password use a mask provided by
elytron-tool.sh
.
In this example, the test target used is an LDAP instance with BIND DN credential: secret12
. The target is mapped using user federation in the realm ldaptest
.
17.4.1. Configuring the credential store and vault without a mask
You create the credential store and a vault where the credential store and vault passwords are in plain text.
Prerequisites
-
A running LDAP instance has
BIND DN credential: secret12
. -
The alias uses the format <realm-name>_< key-value> when using the default key resolver. In this case, the instance is running in the realm
ldaptest
andldaptest_ldap_secret
is the alias that corresponds to the valueldap_secret
in that realm.
The resolver replaces underscore characters with double underscore characters in the realm and key names. For example, for the key ldaptest_ldap_secret
, the final key will be ldaptest_ldap__secret
.
Procedure
Create the Elytron credential store.
[standalone@localhost:9990 /] /subsystem=elytron/credential-store=test-store:add(create=true, location=/home/test/test-store.p12, credential-reference={clear-text=testpwd1!},implementation-properties={keyStoreType=PKCS12})
Add an alias to the credential store.
/subsystem=elytron/credential-store=test-store:add-alias(alias=ldaptest_ldap__secret,secret-value=secret12)
Notice how the resolver causes the key
ldaptest_ldap__secret
to use double underscores.List the aliases from the credential store to inspect the contents of the keystore that is produced by Elytron.
keytool -list -keystore /home/test/test-store.p12 -storetype PKCS12 -storepass testpwd1! Keystore type: PKCS12 Keystore provider: SUN Your keystore contains 1 entries ldaptest_ldap__secret/passwordcredential/clear/, Oct 12, 2020, SecretKeyEntry,
Configure the vault SPI in Red Hat Single Sign-On.
/subsystem=keycloak-server/spi=vault:add(default-provider=elytron-cs-keystore) /subsystem=keycloak-server/spi=vault/provider=elytron-cs-keystore:add(enabled=true, properties={location=>/home/test/test-store.p12, secret=>testpwd1!, keyStoreType=>PKCS12})
At this point, the vault and credentials store passwords are not masked.
<spi name="vault"> <default-provider>elytron-cs-keystore</default-provider> <provider name="elytron-cs-keystore" enabled="true"> <properties> <property name="location" value="/home/test/test-store.p12"/> <property name="secret" value="testpwd1!"/> <property name="keyStoreType" value="PKCS12"/> </properties> </provider> </spi> <credential-stores> <credential-store name="test-store" location="/home/test/test-store.p12" create="true"> <implementation-properties> <property name="keyStoreType" value="PKCS12"/> </implementation-properties> <credential-reference clear-text="testpwd1!"/> </credential-store> </credential-stores>
-
In the LDAP provider, replace
binDN credential
with${vault.ldap_secret}
. Test your LDAP connection.
LDAP Vault
17.4.2. Masking the password in the credential store and vault
You can now update the credential store and vault to have passwords that use a mask provided by elytron-tool.sh
.
Create a masked password using values for the
salt
and theiteration
parameters:$ EAP_HOME/bin/elytron-tool.sh mask --salt SALT --iteration ITERATION_COUNT --secret PASSWORD
For example:
elytron-tool.sh mask --salt 12345678 --iteration 123 --secret testpwd1! MASK-3BUbFEyWu0lRAu8.fCqyUk;12345678;123
Update the Elytron credential store configuration to use the masked password.
/subsystem=elytron/credential-store=cs-store:write-attribute(name=credential-reference.clear-text,value="MASK-3BUbFEyWu0lRAu8.fCqyUk;12345678;123")
Update the Red Hat Single Sign-On vault configuration to use the masked password.
/subsystem=keycloak-server/spi=vault/provider=elytron-cs-keystore:remove() /subsystem=keycloak-server/spi=vault/provider=elytron-cs-keystore:add(enabled=true, properties={location=>/home/test/test-store.p12, secret=>”MASK-3BUbFEyWu0lRAu8.fCqyUk;12345678;123”, keyStoreType=>PKCS12})
The vault and credential store are now masked:
<spi name="vault"> <default-provider>elytron-cs-keystore</default-provider> <provider name="elytron-cs-keystore" enabled="true"> <properties> <property name="location" value="/home/test/test-store.p12"/> <property name="secret" value="MASK-3BUbFEyWu0lRAu8.fCqyUk;12345678;123"/> <property name="keyStoreType" value="PKCS12"/> </properties> </provider> </spi> .... ..... <credential-stores> <credential-store name="test-store" location="/home/test/test-store.p12" create="true"> <implementation-properties> <property name="keyStoreType" value="PKCS12"/> </implementation-properties> <credential-reference clear-text="MASK-3BUbFEyWu0lRAu8.fCqyUk;12345678;123"/> </credential-store> </credential-stores>
-
You can now test the connection to the LDAP using
${vault.ldap_secret}
.
Additional resources
For more information about the Elytron tool, see Using Credential Stores with Elytron Client.