Chapter 2. Adding custom TLS certificates for Red Hat OpenStack Services on OpenShift
When you deploy Red Hat OpenStack Services on OpenShift (RHOSO), TLS-e (TLS everywhere) is enabled by default. TLS is handled by cert-manager, which applies both ingress (public) encryption, as well as reencryption to each pod. Currently, disabling TLS on RHOSO is not supported.
2.1. TLS in Red Hat OpenStack Services on OpenShift Copy linkLink copied to clipboard!
When you deploy Red Hat OpenStack Services on OpenShift (RHOSO), most API connections are protected by TLS.
TLS is not currently available for the internal Alert Manager Web UI service endpoint.
You might be required to protect public APIs using your own internal certificate authority. In order to replace the automatically generated certificates you must create a secret that contains your additional ca certs, including all certificates in needed chains of trust.
You can apply trusted certificates from your own internal certificate authority (CA) to public interfaces on RHOSO. The public interface is where ingress traffic meets the service’s route. Do not attempt to manage encryption on internal (pod level) interfaces.
If you decide to apply trusted certificates from your own internal certificate authority (CA), you will need the following information.
- DNS names
For each service you apply your own custom certificate to, you will need its DNS hostname for the process of generating the certificate. You can get a list of public hostnames using the following command:
oc get -n openstack routesNoteTo use a single certificate for two or more services, use a wildcard in the DNS name field, or list multiple DNS names in the subject alt names field. If you do not use a wildcard, then you must update the certificate in the event of a route hostname change.
- Duration
- To update a service’s certificate in OpenShift, the service must be restarted. The duration for the certificate is the longest amount of time a service can stay live without being restarted, subject to your internal security policies.
- Usages
-
You must include -
key encipherment,digital signature, andserver authwithin the list of usages in your certificate.
Updating TLS to use custom certificates requires edits to both the control plane and the data plane.
The following is the default TLS settings that are used if not annotated and changed:
apiVersion: core.openstack.org/v1beta1
kind: OpenStackControlPlane
metadata:
name: myctlplane
spec:
tls:
default:
ingress:
ca:
duration: 87600h
cert:
duration: 43800h
enabled: true
podLevel:
enabled: true
internal:
ca:
duration: 87600h
cert:
duration: 43800h
libvirt:
ca:
duration: 87600h
cert:
duration: 43800h
ovn:
ca:
duration: 87600h
cert:
duration: 43800h
2.2. Adding custom CA certificates to the control plane Copy linkLink copied to clipboard!
When you deploy Red Hat OpenStack Services on OpenShift (RHOSO), default CA certificates are also deployed on the control plane. When you add a custom CA certificate from a Red Hat Satellite Server (RHSS) or another 3rd party certificate authority to your RHOSO control plane, RHOSO services can validate certificates issued by those 3rd party certificate authorities.
To accomplish this, you must add your custom CA certificate into a bundle that includes all certificates that OpenStack services can verify against.
If TLS is not enabled on each node set, then it must be enabled, which requires deploying the data plane.
Procedure
-
Create a PEM-formatted bundle, for example,
mybundle.pem. Include all the CA certificates that you want OpenStack to trust. Create a manifest file called
cacerts.yamlthat includes themybundle.pemcreated in the previous step. Include all the certificates in chains of trust if applicable:apiVersion: v1 kind: Secret metadata: name: cacerts namespace: openstack type: Opaque data: myBundleExample: <cat mybundle.pem | base64 -w0> CACertExample: <cat cacert.pem | base64 -w0>where:
mybundle.pem-
Specifies the name of your certificate or certificate bundle. The results are pasted as the value of the
myBundleExamplefield. cacert.pem- Specifies the name of your CA certificate.
Create the secret from the manifest file:
$ oc apply -f cacerts.yamlEdit the
openstack_control_plane.yamlcustom resource (CR) file and add your bundle as the parameter forcaBundleSecretName:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane spec: tls: podLevel: enabled: true caBundleSecretName: cacertsApply the control plane changes:
$ oc apply -f openstack_control_plane.yamlDetermine if TLS is enabled on each node set, by running the following command, which returns
trueif TLS is enabled on the specified node set:$ oc get openstackdataplanenodeset <node_set_name> -n <namespace> -o json | jq .items[0].spec.tlsEnabledwhere:
<node_set_name>- Specifies the name of the OpenStackDataPlaneNodeSet CR that the node belongs to.
<namespace>-
Specifies the namespace of the required Red Hat OpenStack Services on OpenShift (RHOSO) environment, for example,
openstack.
If TLS is not enabled, you must enable it:
Open the OpenStackDataPlaneNodeSet CR file for each node on the data plane, and enable TLS in each:
apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneNodeSet metadata: name: <node_set_name> namespace: openstack spec: tlsEnabled: trueSave the updated OpenStackDataPlaneNodeSet CR files and apply the updates:
$ oc apply -f openstack_data_plane.yaml -n <namespace>
Create a file on your workstation to define the OpenStackDataPlaneDeployment CR:
apiVersion: dataplane.openstack.org/v1beta1 kind: OpenStackDataPlaneDeployment metadata: name: <node_set_deployment_name>where:
<node_set_deployment_name>- Specifies the name of the OpenStackDataPlaneDeployment CR. This name must be unique, must consist of lower case alphanumeric characters, - (hyphen) or . (period), and must start and end with an alphanumeric character.
Add the OpenStackDataPlaneNodeSet CRs to the OpenStackDataPlaneDeployment CR file:
spec: ... nodeSets: - <node_set_name>- Save the OpenStackDataPlaneDeployment CR deployment file.
Deploy the modified
OpenStackDataPlaneNodeSetCRs:$ oc create -f openstack_data_plane_deploy.yaml -n <namespace>You can view the Ansible logs while the deployment executes:
$ oc get pod -l app=openstackansibleee -w $ oc logs -l app=openstackansibleee -f --max-log-requests 10If the
oc logscommand returns an error similar to the following error, increase the--max-log-requestsvalue:error: you are attempting to follow 19 log streams, but maximum allowed concurrency is 10, use --max-log-requests to increase the limitVerify that the modified OpenStackDataPlaneNodeSet CRs are deployed:
$ oc get openstackdataplanedeployment -n <namespace> NAME STATUS MESSAGE openstack-data-plane True Setup Complete $ oc get openstackdataplanenodeset -n <namespace> NAME STATUS MESSAGE openstack-data-plane True NodeSet ReadyFor information about the meaning of the returned status, see Data plane conditions and states in the Deploying Red Hat OpenStack Services on OpenShift guide.
If the status indicates that the data plane has not been deployed, then troubleshoot the deployment. For information, see Troubleshooting the data plane creation and deployment in the Deploying Red Hat OpenStack Services on OpenShift guide.
2.3. Updating the control plane with custom certificates for public services Copy linkLink copied to clipboard!
You might be required to protect public APIs by using your own internal certificate authority (CA). To replace the automatically generated route certificates with common signed certificates from your CA, you must create a secret that contains your additional CA certificate, and all certificates in the chain of trust.
Prerequisites
-
You have a list of each of the public services for which to apply your custom service certificates. You can get this list using the
oc route list -n openstackcommand. Use this information for the number of certificates you must create, the DNS names for those certificates, as well as finding the relevant services to edit in theopenstack_control_plane.yamlcustom resource (CR). - You have a service certificate for the public services
Procedure
Create a manifest file called
cacerts.yamlthat includes all CA certificates. Include all certificates in chains of trust if applicable:apiVersion: v1 kind: Secret metadata: name: cacerts namespace: openstack type: Opaque data: myBundleExample: <cat mybundle.pem | base64 -w0> CACertExample: <cat cacert.pem | base64 -w0>where:
mybundle.pem-
Specifies the name of your certificate or certificate bundle. The results are pasted as the value of the
myBundleExamplefield. cacert.pem- Specifies the name of your CA certificate.
Create the secret from the manifest file:
$ oc apply -f cacerts.yamlCreate a manifest file for each secret named
api_certificate_<service>_secret.yaml:apiVersion: v1 kind: Secret metadata: name: api_certificate_<service>_secret namespace: openstack type: kubernetes.io/tls data: tls.crt: <cat tlscrt.pem | base64 -w0> tls.key: <cat tlskey.pem | base64 -w0> ca.crt: <cat cacrt.pem | base64 -w0>where:
<service>- Specifies the name of the service that this secret is for.
tlscrt.pem- Specifies the name of your signed certificate.
tlskey.pem- Specifies the name of your private key.
cacrt.pem- Specifies the name of your CA certificate.
Create the secret
$ oc apply -f api_certificate_<service>_secret.yamlEdit the
openstack_control_plane.yamlcustom resource and add your bundle as the parameter forcaBundleSecretName:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane spec: tls: podLevel: enabled: true caBundleSecretName: cacertsApply the secret service certificates to each of the public services under the apiOverride field. For example enter the following for the Identity service (keystone):
apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane namespace: openstack spec: ... keystone: apiOverride: tls: secretName: api_certificate_keystone_secretThe edits for the Compute service (nova) and noVNCProxy appear as the following:
apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane namespace: openstack spec: ... nova: apiOverride: tls: secretName: api_certificate_nova_secret route: {} cellOverride: cell1: noVNCProxy: tls: secretName: api_certificate_novavncproxy_secretApply the control plane changes
$ oc apply -f openstack_control_plane.yaml
2.4. Updating the control plane with a single custom certificate for public services Copy linkLink copied to clipboard!
You might be required to protect public APIs by using your own internal certificate authority (CA). To replace the automatically generated route certificates with a common signed certificate from your CA, you must create a secret that contains your CA certificate, and all certificates in the chain of trust.
Prerequisites
-
You have a list of each of the public services for which to apply your custom service certificate. You can get this list by using the
oc route list -n openstackcommand. Use this information for the DNS names for the certificate, as well as for finding the relevant services to edit in theopenstack_control_plane.yamlcustom resource (CR).
Procedure
Create a signed certificate that includes the hostname for every service in the
alt_namessection:[alt_names] DNS.1 = barbican-public-openstack.apps.ocp.openstack.lab DNS.2 = cinder-public-openstack.apps.ocp.openstack.lab DNS.3 = glance-default-public-openstack.apps.ocp.openstack.lab DNS.4 = horizon-openstack.apps.ocp.openstack.lab DNS.5 = keystone-public-openstack.apps.ocp.openstack.lab DNS.6 = manila-public-openstack.apps.ocp.openstack.lab DNS.7 = neutron-public-openstack.apps.ocp.openstack.lab DNS.8 = nova-novncproxy-cell1-public-openstack.apps.ocp.openstack.lab DNS.9 = nova-public-openstack.apps.ocp.openstack.lab DNS.10 = placement-public-openstack.apps.ocp.openstack.labCreate a manifest file called
cacerts.yamlthat includes all CA certificates. Include all certificates in chains of trust if applicable:apiVersion: v1 kind: Secret metadata: name: cacerts namespace: openstack type: Opaque data: myBundleExample: <cat mybundle.pem | base64 -w0> CACertExample: <cat cacert.pem | base64 -w0>where:
mybundle.pem-
Specifies the name of your certificate or certificate bundle. The results are pasted as the value of the
myBundleExamplefield. cacert.pem- Specifies the name of your CA certificate.
Create the secret from the manifest file:
$ oc apply -f cacerts.yamlCreate a manifest file for a secret named
certificate-secret.yaml:apiVersion: v1 kind: Secret metadata: name: certificate-secret namespace: openstack type: kubernetes.io/tls data: tls.crt: <cat tlscrt.pem | base64 -w0> tls.key: <cat tlskey.pem | base64 -w0> ca.crt: <cat cacrt.pem | base64 -w0>where:
tlscrt.pem- Specifies the name of your signed certificate.
tlskey.pem- Specifies the name of your private key.
cacrt.pem- Specifies the name of your CA certificate.
Create the secret
$ oc apply -f certificate-secret.yamlEdit the
openstack_control_plane.yamlcustom resource and add your bundle as the parameter forcaBundleSecretName:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane spec: tls: podLevel: enabled: true caBundleSecretName: cacertsApply the secret service certificates to each of the public services under the
apiOverridefield. For example, enter the following for the Identity service (keystone):apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane namespace: openstack spec: ... keystone: apiOverride: tls: secretName: certificate-secretThe edits for the Compute service (nova) and
NoVNCProxyappear as the following:apiVersion: core.openstack.org/v1beta1 kind: OpenStackControlPlane metadata: name: myctlplane namespace: openstack spec: ... nova: apiOverride: tls: secretName: certificate-secret route: {} cellOverride: cell1: NoVNCProxy: tls: secretName: certificate-secretApply the control plane changes
$ oc apply -f openstack_control_plane.yaml
2.5. Using your CA certs on remote clients Copy linkLink copied to clipboard!
If you do not use a trusted CA from a public entity, openstack client commands fail with an SSL verification error, requiring the --insecure command option to succeed. You can securely communicate with OpenStack API using a private certificate authority using the following steps.
Prerequisites
- You have deployed RHOSO with default certificates, or have used custom certificates that are not signed by a public certificate authority.
Procedure
- Log onto OpenShift with global administrative permissions.
Extract the ca cert for the public endpoints from the
rootca-publicsecret.$ oc get secret rootca-public -o json | jq -r '.data."ca.crt"' | base64 -d > ca.crt-
Transfer the
ca.crtfile to the client that accesses the OpenStack API. Update your authentication file with the path to
ca.crt.If you use a
clouds.ymlauthentication file, add thecacertparameter:clouds: secure: cacert: </path/to/ca.crt>where:
</path/to/ca.crt>- Specifies the absolute path and name of the CA cert on your system.
If you use a resource credentials file, update the file with the exported
CACERTvariable:$ export OC_CACERT=</path/to/ca.crt>