Chapter 5. Securing Red Hat OpenStack deployments with TLS and PKI

Red Hat OpenStack Platform consists of many networks and endpoints that handle sensitive or confidential data that you can secure. When you use Transport Layer Security (TLS), you secure traffic with symmetric key encryption. The key and cipher are negotiated in the TLS handshake, which requires validation of the server’s identity through a shared trust in an intermediary called a Certificate Authority (CA).

Public Key Infrastructure (PKI) is a framework for validating an entity through a certificate authority.

5.1. Components of Public Key Infrastructure (PKI)

The core components of PKI are shown in the folling table:

Table 5.1. Key Terms
Term	Definition
End entity	The user, process, or system that validates itself through the use of a digital certificate.
Certificate Authority (CA)	The CA is an entity that is trusted by both the end entity, and the relying party that validates the end entity.
Relying party	The relying party receives the digital certificate as validation of the end entity, and has the capability of verifying the digital certificate.
Digital certificates	Signed public key certificates have a verifiable entity and a public key, and are issued by a CA. When a CA signs a certificate, it creates a message digest from the certificate encrypted with its private key. You can verify the signature using the public key associated with CA. The X.509 standard is used to define the certificates.
Registration Authority (RA)	An RA is an optional dedicated authority that can perform management functions such as authenticating end entities before they are issued a certificate by a CA. The CA authenticates end entities if there is no RA.
Certificate Revocation List (CRL)	A CRL is a list of certificate serial numbers that have been revoked. End entities presenting certificates with revoked serial numbers are not trusted in a PKI model.
CRL issuer	An optional system to which a CA delegates the publication of certificate revocation lists.
Certificate Repository	The location where the end entity certificates and certificate revocation lists are stored and queried.

5.2. Certificate Authority requirements and recommendations

You must get certificates signed by a widely recognized certificate authority (CA) for publicly available Red Hat OpenStack Platform Dashboards or publicly accessible APIs.

You must give a DNS domain or subdomain to each endpoint that you secure with TLS. The domains you provide are used to create the certificates issued by a CA. Customers access the dashboard or the API using the DNS name so that the CA can validate the endpoint.

Red Hat recommends using a separate and internally managed CA to secure internal traffic. This allows the cloud deployer to maintain control of their Private Key Infrastructure (PKI) implementation and makes requesting, signing and deploying certificates for internal systems easier.

You can enable SSL/TLS on your overcloud endpoints. Due to the number of certificates required to configure TLS everywhere (TLS-e), director integrates with a Red Hat Identity Management (IdM) server to act as a certificate authority and manage the overcloud certificates. For more information on configuring TLS-e, see Implementing TLS-e with Ansible.

To check the status of TLS support across the OpenStack components, refer to the TLS Enablement status matrix.

If want to you use a SSL certificate with your own certificate authority, see Enabling SSL/TLS on overcloud public endpoints.

Note

This will configure Red Hat OpenStack Platform with SSL/TLS on publicly accessible endpoints only.

5.3. Identifying TLS versions in your environment

Important

TLS version 1.0 is is deprecated for Red Hat OpenStack platform. Additionally, you must at minimum use TLS 1.2 for NIST-approval. For more information, see Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations.

You can use cipherscan to determine the versions of TLS being presented by your deployment. Cipherscan can be cloned from https://github.com/mozilla/cipherscan. This example output demonstrates results received from horizon:

Note

Run cipherscan from a non-production system, as it might install additional dependencies when you first run it.

Procedure

Run cipher scan against the accessible URL of the Dashboard service:

$ ./cipherscan https://openstack.lab.local
..............................
Target: openstack.lab.local:443

prio  ciphersuite                  protocols  pfs                 curves
1     ECDHE-RSA-AES128-GCM-SHA256  TLSv1.2    ECDH,P-256,256bits  prime256v1
2     ECDHE-RSA-AES256-GCM-SHA384  TLSv1.2    ECDH,P-256,256bits  prime256v1
3     DHE-RSA-AES128-GCM-SHA256    TLSv1.2    DH,1024bits         None
4     DHE-RSA-AES256-GCM-SHA384    TLSv1.2    DH,1024bits         None
5     ECDHE-RSA-AES128-SHA256      TLSv1.2    ECDH,P-256,256bits  prime256v1
6     ECDHE-RSA-AES256-SHA384      TLSv1.2    ECDH,P-256,256bits  prime256v1
7     ECDHE-RSA-AES128-SHA         TLSv1.2    ECDH,P-256,256bits  prime256v1
8     ECDHE-RSA-AES256-SHA         TLSv1.2    ECDH,P-256,256bits  prime256v1
9     DHE-RSA-AES128-SHA256        TLSv1.2    DH,1024bits         None
10    DHE-RSA-AES128-SHA           TLSv1.2    DH,1024bits         None
11    DHE-RSA-AES256-SHA256        TLSv1.2    DH,1024bits         None
12    DHE-RSA-AES256-SHA           TLSv1.2    DH,1024bits         None
13    ECDHE-RSA-DES-CBC3-SHA       TLSv1.2    ECDH,P-256,256bits  prime256v1
14    EDH-RSA-DES-CBC3-SHA         TLSv1.2    DH,1024bits         None
15    AES128-GCM-SHA256            TLSv1.2    None                None
16    AES256-GCM-SHA384            TLSv1.2    None                None
17    AES128-SHA256                TLSv1.2    None                None
18    AES256-SHA256                TLSv1.2    None                None
19    AES128-SHA                   TLSv1.2    None                None
20    AES256-SHA                   TLSv1.2    None                None
21    DES-CBC3-SHA                 TLSv1.2    None                None

Certificate: trusted, 2048 bits, sha256WithRSAEncryption signature
TLS ticket lifetime hint: None
NPN protocols: None
OCSP stapling: not supported
Cipher ordering: server
Curves ordering: server - fallback: no
Server supports secure renegotiation
Server supported compression methods: NONE
TLS Tolerance: yes

Intolerance to:
 SSL 3.254           : absent
 TLS 1.0             : PRESENT
 TLS 1.1             : PRESENT
 TLS 1.2             : absent
 TLS 1.3             : absent
 TLS 1.4             : absent

When scanning a server, Cipherscan advertises support for a specific TLS version, which is the highest TLS version it is willing to negotiate. If the target server correctly follows TLS protocol, it will respond with the highest version that is mutually supported, which may be lower than what Cipherscan initially advertised. If the server does proceed to establish a connection with the client using that specific version, it is not considered to be intolerant to that protocol version. If it does not establish the connection (with the specified version, or any lower version), then intolerance for that version of protocol is considered to be present. For example:

Intolerance to:
 SSL 3.254           : absent
 TLS 1.0             : PRESENT
 TLS 1.1             : PRESENT
 TLS 1.2             : absent
 TLS 1.3             : absent
 TLS 1.4             : absent

In this output, intolerance of TLS 1.0 and TLS 1.1 is reported as PRESENT, meaning that the connection could not be established, and that Cipherscan was unable to connect while advertising support for those TLS versions. As a result, it is reasonable to conclude that those (and any lower) versions of the protocol are not enabled on the scanned server.

5.4. Identity Management (IdM) server recommendations for OpenStack

Red Hat provides the following information to help you integrate your IdM server and OpenStack environment.

For information on preparing Red Hat Enterprise Linux for an IdM installation, see Installing Identity Management.

Run the ipa-server-install command to install and configure IdM. You can use command parameters to skip interactive prompts. Use the following recommendations so that your IdM server can integrate with your Red Hat OpenStack Platform environment:

Table 5.2. Parameter recommendations
Option	Recommendation
`--admin-password`	Note the value you provide. You will need this password when configuring Red Hat OpenStack Platform to work with IdM.
`--ip-address`	Note the value you provide. The undercloud and overcloud nodes require network access to this ip address.
`--setup-dns`	Use this option to install an integrated DNS service on the IdM server. The undercloud and overcloud nodes use the IdM server for domain name resolution.
`--auto-forwarders`	Use this option to use the addresses in `/etc/resolv.conf` as DNS forwarders.
`--auto-reverse`	Use this option to resolve reverse records and zones for the IdM server IP addresses. If neither reverse records or zones are resolvable, IdM creates the reverse zones. This simplifies the IdM deployment.
`--ntp-server`, `--ntp-pool`	You can use both or either of these options to configure your NTP source. Both the IdM server and your OpenStack environment must have correct and synchronized time.

You must open the firewall ports required by IdM to enable communication with Red Hat OpenStack Platform nodes. For more information, see Opening the ports required by IdM.

Additional resources

5.5. Implementing TLS-e with Ansible

You can use the new tripleo-ipa method to enable SSL/TLS on overcloud endpoints, called TLS everywhere (TLS-e). Due to the number of certificates required, Red Hat OpenStack Platform integrates with Red Hat Identity management (IdM). When you use tripleo-ipa to configure TLS-e, IdM is the certificate authority.

Prerequisites

Ensure that all configuration steps for the undercloud, such as the creation of the stack user, are complete. For more details, see Installing and managing Red Hat OpenStack Platform with director for more details
The IP address of your DNS server is configured on the undercloud to the IP address of the IdM server. One of the following parameters must be configured in your undercloud.conf file:
- DEFAULT/undercloud_nameservers
- %SUBNET_SECTION%/dns_nameservers

Procedure

Use the following procedure to implement TLS-e on a new installation of Red Hat OpenStack Platform, or an existing deployment that you want to configure with TLS-e. You must use this method if you deploy Red Hat OpenStack Platform with TLS-e on pre-provisioned nodes.

Note

If you are implementing TLS-e for an existing environment, you are required to run commands such as openstack undercloud install, and openstack overcloud deploy. These procedures are idempotent and only adjust your existing deployment configuration to match updated templates and configuration files.

Configure the /etc/resolv.conf file:
Set the appropriate search domains and the nameserver on the undercloud in /etc/resolv.conf. For example, if the deployment domain is example.com, and the domain of the FreeIPA server is bigcorp.com, then add the following lines to /etc/resolv.conf:
```
search example.com bigcorp.com
nameserver $IDM_SERVER_IP_ADDR
```

Install required software:

sudo dnf install -y python3-ipalib python3-ipaclient krb5-devel

Export environmental variables with values specific to your environment.:

export IPA_DOMAIN=bigcorp.com
export IPA_REALM=BIGCORP.COM
export IPA_ADMIN_USER=$IPA_USER 1
export IPA_ADMIN_PASSWORD=$IPA_PASSWORD 2
export IPA_SERVER_HOSTNAME=ipa.bigcorp.com
export UNDERCLOUD_FQDN=undercloud.example.com 3
export USER=stack
export CLOUD_DOMAIN=example.com

1 2: The IdM user credentials are an administrative user that can add new hosts and services.
3: The value of the UNDERCLOUD_FQDN parameter matches the first hostname-to-IP address mapping in /etc/hosts.

Run the undercloud-ipa-install.yaml ansible playbook on the undercloud:

ansible-playbook \
--ssh-extra-args "-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" \
/usr/share/ansible/tripleo-playbooks/undercloud-ipa-install.yaml

Add the following parameters to undercloud.conf

undercloud_nameservers = $IDM_SERVER_IP_ADDR
overcloud_domain_name = example.com

[Optional] If your IPA realm does not match your IPA domain, set the value of the certmonger_krb_realm parameter:
1. Set the value of the certmonger_krb_realm in /home/stack/hiera_override.yaml:
```
parameter_defaults:
  certmonger_krb_realm: EXAMPLE.COMPANY.COM
```
2. Set the value of the custom_env_files parameter in undercloud.conf to /home/stack/hiera_override.yaml:
```
custom_env_files = /home/stack/hiera_override.yaml
```
Deploy the undercloud:
```
openstack undercloud install
```

Verification

Verify that the undercloud was enrolled correctly by completing the following steps:

List the hosts in IdM:
```
$ kinit admin
$ ipa host-find
```
Confirm that /etc/novajoin/krb5.keytab exists on the undercloud.
```
ls /etc/novajoin/krb5.keytab
```

Note

The novajoin directory name is for legacy naming purposes only.

Configuring TLS-e on the overcloud

When you deploy the overcloud with TLS everywhere (TLS-e), IP addresses from the Undercloud and Overcloud will automatically be registered with IdM.

Before deploying the overcloud, create a YAML file tls-parameters.yaml with contents similar to the following. The values you select will be specific for your environment:

parameter_defaults:
    DnsSearchDomains: ["example.com"]
    CloudDomain: example.com
    CloudName: overcloud.example.com
    CloudNameInternal: overcloud.internalapi.example.com
    CloudNameStorage: overcloud.storage.example.com
    CloudNameStorageManagement: overcloud.storagemgmt.example.com
    CloudNameCtlplane: overcloud.ctlplane.example.com
    IdMServer: freeipa-0.redhat.local
    IdMDomain: redhat.local
    IdMInstallClientPackages: False

resource_registry:
      OS::TripleO::Services::IpaClient: /usr/share/openstack-tripleo-heat-templates/deployment/ipa/ipaservices-baremetal-ansible.yaml

The shown value of the OS::TripleO::Services::IpaClient parameter overrides the default setting in the enable-internal-tls.yaml file. You must ensure the tls-parameters.yaml file follows enable-internal-tls.yaml in the openstack overcloud deploy command.
For more information about the parameters that you use to implement TLS-e, see Parameters for tripleo-ipa.

[Optional] If your IPA realm does not match your IPA domain, you must also include value of the CertmongerKerberosRealm parameter in the tls-parameters.yaml file:
```
    CertmongerKerberosRealm: EXAMPLE.COMPANY.COM
```

Deploy the overcloud. You will need to include the tls-parameters.yaml in the deployment command:

DEFAULT_TEMPLATES=/usr/share/openstack-tripleo-heat-templates/
CUSTOM_TEMPLATES=/home/stack/templates

openstack overcloud deploy \
-e ${DEFAULT_TEMPLATES}/environments/ssl/tls-everywhere-endpoints-dns.yaml \
-e ${DEFAULT_TEMPLATES}/environments/services/haproxy-public-tls-certmonger.yaml \
-e ${DEFAULT_TEMPLATES}/environments/ssl/enable-internal-tls.yaml \
-e ${CUSTOM_TEMPLATES}/tls-parameters.yaml \
...

Confirm each endpoint is using HTTPS by querying keystone for a list of endpoints:
```
openstack endpoint list
```

5.6. Parameters for tripleo-ipa

Use the fully qualified domain name (FQDN) of your cloud to define the cloud name and cloud domain parameters required for tripleo-ipa. For example, with an FQDN of overcloud.example.com, use the following values:

CloudDomain: example.com
CloudName: overcloud.example.com
CloudNameCtlplane: overcloud.ctlplane.example.com
CloudNameInternal: overcloud.internalapi.example.com
CloudNameStorage: overcloud.storage.example.com
CloudNameStorageManagement: overcloud.storagemgmt.example.com

Set the following additional parameters based on the requirements of your environment:

CertmongerKerberosRealm: Set CertmongerKerberosRealm parameter to the value of the IPA realm. This is required if the IPA realm does not match the IPA domain.
DnsSearchDomains: The DnsSearchDomains parameter is a comma-separated list. If the domain of the IdM server is different than the cloud domain, include the domain of the IdM server in the DnsSearchDomains parameter.
EnableEtcdInternalTLS: If you deploy TLSe on a distributed compute node (DCN) architecture, you must add the EnableEtcdInternalTLS parameter with the value of True.
IDMInstallClientPackages: If you have preprovisioned your compute nodes, set the IDMInstallClientPackages parameter to a value of True. Otherwise, set the value to False.
IDMModifyDNS: Set the IDMModifyDNS parameter to false to disable automatic IP registration of the overcloud nodes on Red Hat Identity Server.
IdmDomain: Set the IdmDomain parameter to the domain portion of the FQDN of your Red Hat Identity server. The value that you specify is also used as the value of the IdM realm. If the IdM domain and IdM realm differ, set the realm explicitly using the CertmongerKerberosRealm parameter.
IdmServer: Set the IdmServer parameter to the FQDN of your Red Hat Identity server. If you use a replicated IdM environment, then set multiple values using a comma delimited list. For more information on IdM replicas, see Installing an IdM replica.

5.7. Encrypting memcached traffic under TLS everywhere (TLS-e)

You can now encrypt memcached traffic with TLS-e. This feature works with both novajoin and tripleo-ipa:

Create an environment file called memcached.yaml with the following contents to add TLS support for memcached:
```
parameter_defaults:
    MemcachedTLS: true
    MemcachedPort: 11212
```

Include the memcached.yaml environment file in the overcloud deployment process:

openstack overcloud deploy --templates \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-public-tls-certmonger.yaml \
-e /home/stack/memcached.yaml
...

Additional Resources

For more information about deploying TLSe with tripleo-ipa, see Implementing TLS-e with Ansible.

5.8. Increasing the size of private keys

You can improve security by increasing the size of private keys used to create certificates for encrypted service traffic. The default RHOSP private key size of 2048 bits matches the National Institute of Standards and Technology (NIST) recommended minimum.

Use the CertificateKeySize parameter to change the size of private keys globally.
Use a service specific parameter such as RedisCertificateKeySize, to modify a specific private key, or to override the global CertificateKeySize parameter.

Use these parameters in an environment heat template, and include the template in your overcloud deployment command. If you have already deployed the overcloud, you must rerun the same openstack overcloud deploy command with the same templates you originally used, and include the new parameters for the changes to take effect.

In the following example, the global value for private keys is 4096. The private key for redis is 2048 because the RedisCertificateKeySize overrides the global parameter:

Example

parameter_defaults:
    CertificateKeySize: '4096'
    RedisCertificateKeySize: '2048'

5.9. Replacing the IdM server for Red Hat OpenStack Platform with its replica

When you replace an existing IPA server with its replica, you must update the necessary parameters. Failure to do so results in failed overcloud deployments when you update your cluster’s configuration.

Procedure

On each overcloud node, edit the /etc/ipa/default.conf configuration file to ensure that the server and xmlrpc_uri parameters include the fully qualified domain name (FQDN) of the IdM server:

#File modified by ipa-client-install

[global]
basedn = dc=redhat,dc=local
realm = REDHAT.LOCAL
domain = redhat.local
server = freeipa-0.redhat.local
host = undercloud-0.redhat.local
xmlrpc_uri = https://freeipa-0.redhat.local/ipa/xml
enable_ra = True

On the undercloud, edit the /home/stack/templates/tls-parameters.yaml environment file and ensure that the IPA_SERVER_HOSTNAME parameter matches the FQDN that is shown in the xmlrpc_uri and server parameters in /etc/ipa/default.conf. Ensure that all parameters match your environment:
```
export IPA_DOMAIN=bigcorp.com
export IPA_REALM=BIGCORP.COM
export IPA_ADMIN_USER=$IPA_USER
export IPA_ADMIN_PASSWORD=$IPA_PASSWORD
export IPA_SERVER_HOSTNAME=ipa.bigcorp.com
export UNDERCLOUD_FQDN=undercloud.example.com
export USER=stack
export CLOUD_DOMAIN=example.com
```

Chapter 5. Securing Red Hat OpenStack deployments with TLS and PKI

5.1. Components of Public Key Infrastructure (PKI)

5.2. Certificate Authority requirements and recommendations

5.3. Identifying TLS versions in your environment

5.4. Identity Management (IdM) server recommendations for OpenStack

5.5. Implementing TLS-e with Ansible

5.6. Parameters for tripleo-ipa

5.7. Encrypting memcached traffic under TLS everywhere (TLS-e)

5.8. Increasing the size of private keys

5.9. Replacing the IdM server for Red Hat OpenStack Platform with its replica

Learn

Try, buy, & sell

Communities

About Red Hat Documentation

Making open source more inclusive

About Red Hat

Red Hat legal and privacy links

Red Hat legal and privacy links