Chapter 3. Release Information

This update adds hostname and network name to the output of the 'openstack port list' command.
The additional information makes it easier to associate Neutron port and IP addresses with a particular host.

This update adds hostname and network name to the output of the 'openstack port list' command.
The additional information makes it easier to associate Neutron port and IP addresses with a particular host.

With this update, the libvirt compute driver now allows users to create instances with trusted SR-IOV virtual functions. When trusted, a VF can perform certain operations, such as modifying the VF’s MAC address in the guest.

Interface bonding requires that all slaves use the same MAC address, which in turn requires MAC address modifications on one of the VFs during a failover. Because MAC address altering is a privileged operation, participating VFs must be trusted in order to successfully configure bonding in the guest.

Administrators can now configure trusted mode for VFs. This requires two steps. First, the 'trusted' value of the '[pci] passthrough_whitelist' JSON configuration option in nova.conf must be set to 'true'. For example:

    [pci]
    passthrough_whitelist = {"devname": "eth0", "trusted": "true",
                             "physical_network":"sriovnet1"}

Then, when creating the port, 'trusted=true' must be set for the binding profile. For example:

    $ neutron port-create <net-id> \
        --name sriov_port \
        --vnic-type direct \
        --binding:profile type=dict trusted=true

Because trusted mode only applies to SR-IOV VFs, the 'vnic-type' must be one of 'hw_veb' or 'direct'.

With this update, the libvirt compute driver now allows users to create instances with trusted SR-IOV virtual functions. When trusted, a VF can perform certain operations, such as modifying the VF’s MAC address in the guest.

Interface bonding requires that all slaves use the same MAC address, which in turn requires MAC address modifications on one of the VFs during a failover. Because MAC address altering is a privileged operation, participating VFs must be trusted in order to successfully configure bonding in the guest.

Administrators can now configure trusted mode for VFs. This requires two steps. First, the 'trusted' value of the '[pci] passthrough_whitelist' JSON configuration option in nova.conf must be set to 'true'. For example:

    [pci]
    passthrough_whitelist = {"devname": "eth0", "trusted": "true",
                             "physical_network":"sriovnet1"}

Then, when creating the port, 'trusted=true' must be set for the binding profile. For example:

    $ neutron port-create <net-id> \
        --name sriov_port \
        --vnic-type direct \
        --binding:profile type=dict trusted=true

Because trusted mode only applies to SR-IOV VFs, the 'vnic-type' must be one of 'hw_veb' or 'direct'.

Heat templates now include the `CephClusterName` parameter. This parameter enables you to customize the Ceph cluster name, which you might need to do if you use an external Ceph cluster or a Ceph RBDMirror.

Heat templates now include the `CephClusterName` parameter. This parameter enables you to customize the Ceph cluster name, which you might need to do if you use an external Ceph cluster or a Ceph RBDMirror.

With this update, Users can create application credentials to allow their applications to authenticate to keystone.

See https://docs.openstack.org/keystone/latest/user/application_credentials.html

With this update, Users can create application credentials to allow their applications to authenticate to keystone.

See https://docs.openstack.org/keystone/latest/user/application_credentials.html

Feature:
Add the CPUWeigher weigher for nova-scheduler.

Reason:
The CPUWeigher allows operators to configure a stacking or spreading policy for vCPUs.

Result:
Operators can enable the CPUWeigher and configure a stacking (use all vCPUs on one node first) or spreading (attempt to use a roughly equal amount of vCPUs from all hosts) policy.

Feature:
Add the CPUWeigher weigher for nova-scheduler.

Reason:
The CPUWeigher allows operators to configure a stacking or spreading policy for vCPUs.

Result:
Operators can enable the CPUWeigher and configure a stacking (use all vCPUs on one node first) or spreading (attempt to use a roughly equal amount of vCPUs from all hosts) policy.

This update supports two new tunable options that can be used to reduce packet drop.

Virtual CPUs (vCPUs) can be preempted by the hypervisor kernel thread even with strong partitioning in place (isolcpus, tuned). Preemptions are not frequent, a few per second, but with 256 descriptors per virtio queue, just one preemption of the vCPU can lead to packet drop, because the 256 slots are filled during the preemption. This is the case for network functions virtualization (NFV) VMs in which the per queue packet rate is above 1 Mpps (1 million packets per second).

This update supports two new tunable options: 'rx_queue_size' and 'tx_queue_size'. Use these options to configure the RX queue size and TX queue size of virtio NICs, respectively, to reduce packet drop.

This update supports two new tunable options that can be used to reduce packet drop.

Virtual CPUs (vCPUs) can be preempted by the hypervisor kernel thread even with strong partitioning in place (isolcpus, tuned). Preemptions are not frequent, a few per second, but with 256 descriptors per virtio queue, just one preemption of the vCPU can lead to packet drop, because the 256 slots are filled during the preemption. This is the case for network functions virtualization (NFV) VMs in which the per queue packet rate is above 1 Mpps (1 million packets per second).

This update supports two new tunable options: 'rx_queue_size' and 'tx_queue_size'. Use these options to configure the RX queue size and TX queue size of virtio NICs, respectively, to reduce packet drop.

Instance resources have three new attributes: launched_at, deleted_at, created_at to track the exact time that Nova creates/launches/deletes instances.

Instance resources have three new attributes: launched_at, deleted_at, created_at to track the exact time that Nova creates/launches/deletes instances.

OpenStack director now uses Ansible for software configuration of the overcloud nodes. Ansible provides a more familiar and debuggable operator experience during overcloud deployment. Ansible is used to replace the communication and transport of the software configuration deployment data between heat and the heat agent (os-collect-config) on the overcloud nodes.

Instead of os-collect-config running on each overcloud node and polling for deployment data from heat, the Ansible control node applies the configuration by running an ansible-playbook with an Ansible inventory file and a set of playbooks and tasks. The Ansible control node (the node running ansible-playbook) is the undercloud by default.

OpenStack director now uses Ansible for software configuration of the overcloud nodes. Ansible provides a more familiar and debuggable operator experience during overcloud deployment. Ansible is used to replace the communication and transport of the software configuration deployment data between heat and the heat agent (os-collect-config) on the overcloud nodes.

Instead of os-collect-config running on each overcloud node and polling for deployment data from heat, the Ansible control node applies the configuration by running an ansible-playbook with an Ansible inventory file and a set of playbooks and tasks. The Ansible control node (the node running ansible-playbook) is the undercloud by default.

OpenStack Sahara now supports Cloudera Distribution Hadoop (CDH) plugin 5.13.

OpenStack Sahara now supports Cloudera Distribution Hadoop (CDH) plugin 5.13.

This update adds support of s3-compatible object stores for OpenStack Sahara.

This update adds support of s3-compatible object stores for OpenStack Sahara.

With this release, Nova's libvirt driver now allows the specification of granular CPU feature flags when configuring CPU models.

One benefit of this change is the alleviation of a performance degradation experienced on guests running with certain Intel-based virtual CPU models after application of the "Meltdown" CVE fixes. This guest performance impact is reduced by exposing the CPU feature flag 'PCID' ("Process-Context ID") to the *guest* CPU, assuming that the PCID flag is available in the physical hardware itself.

For more details, refer to the  documentation of ``[libvirt]/cpu_model_extra_flags`` in ``nova.conf`` for usage details.

With this release, Nova's libvirt driver now allows the specification of granular CPU feature flags when configuring CPU models.

One benefit of this change is the alleviation of a performance degradation experienced on guests running with certain Intel-based virtual CPU models after application of the "Meltdown" CVE fixes. This guest performance impact is reduced by exposing the CPU feature flag 'PCID' ("Process-Context ID") to the *guest* CPU, assuming that the PCID flag is available in the physical hardware itself.

For more details, refer to the  documentation of ``[libvirt]/cpu_model_extra_flags`` in ``nova.conf`` for usage details.

This update introduces multi-tenant bare metal networking with the "neutron" network interface.

By configuring the bare metal nodes with the "neutron" network interface, an operator can enable the users to use isolated VLAN networks for provisioning and tenant traffic on bare metal nodes.

This update introduces multi-tenant bare metal networking with the "neutron" network interface.

By configuring the bare metal nodes with the "neutron" network interface, an operator can enable the users to use isolated VLAN networks for provisioning and tenant traffic on bare metal nodes.

A new tripleo heat template has been added to support QDR.
Enabling the metrics_qdr service deploys QDR service on all overcloud nodes, which will be used for routing metrics data from collectd service running on each overcloud node.

The relevant heat template parameters are shown below.

MetricsQdrPort:
   default: '5666'
   description: Service name or port number on which the qdrouterd will accept
                connections. This argument must be string, even if the numeric
                form is used.
   type: string
 MetricsQdrUsername:
   default: 'guest'
   description: Username which should be used to authenticate to the deployed
                qdrouterd.
   type: string
 MetricsQdrPassword:
   default: 'guest'
   description: Password which should be used to authenticate to the deployed
                qdrouterd.
   type: string
   hidden: true
 MetricsQdrConnectors:
   default: []
   description: Connectors configuration (array of hashes).
   type: json
 MetricsQdrAddresses:
   default:
     - prefix: 'collectd/notify'
       distribution: multicast
     - prefix: 'collectd/telemetry'
       distribution: multicast
   description: Addresses configuration (array of hashes).
   type: json
 MetricsQdrUseSSL:
   default: false
   description: Set to true if it is required to use SSL or TLS on
                the connection for listener.
   type: boolean
 MetricsQdrUseEncryption:
   default: false
   description: Set to true if it is required to encrypt connection to the peer
                for listener.
   type: boolean
 MetricsQdrSaslMechanisms:
   default: 'ANONYMOUS'
   description: List of accepted SASL auth mechanisms for listener in format
                of comma separated list.
   type: string
 MetricsQdrSslCertDb:
   default: ''
   description: Path to SSL certificate db for listener.
   type: string
 MetricsQdrSslCertFile:
   default: ''
   description: Path to SSL certificate file for listener.
   type: string
 MetricsQdrSslKeyFile:
   default: ''
   description: Path to SSL private key file for listener.
   type: string
 MetricsQdrSslPwFile:
   default: ''
   description: Path to SSL password file for certificate key for listener.
   type: string
 MetricsQdrSslPassword:
   default: ''
   description: SSL password to be supplied for listener.
   type: string
 MetricsQdrTrustedCerts:
   default: ''
   description: Path to file containing trusted certificates for listener.
   type: string

A new tripleo heat template has been added to support QDR.
Enabling the metrics_qdr service deploys QDR service on all overcloud nodes, which will be used for routing metrics data from collectd service running on each overcloud node.

The relevant heat template parameters are shown below.

MetricsQdrPort:
   default: '5666'
   description: Service name or port number on which the qdrouterd will accept
                connections. This argument must be string, even if the numeric
                form is used.
   type: string
 MetricsQdrUsername:
   default: 'guest'
   description: Username which should be used to authenticate to the deployed
                qdrouterd.
   type: string
 MetricsQdrPassword:
   default: 'guest'
   description: Password which should be used to authenticate to the deployed
                qdrouterd.
   type: string
   hidden: true
 MetricsQdrConnectors:
   default: []
   description: Connectors configuration (array of hashes).
   type: json
 MetricsQdrAddresses:
   default:
     - prefix: 'collectd/notify'
       distribution: multicast
     - prefix: 'collectd/telemetry'
       distribution: multicast
   description: Addresses configuration (array of hashes).
   type: json
 MetricsQdrUseSSL:
   default: false
   description: Set to true if it is required to use SSL or TLS on
                the connection for listener.
   type: boolean
 MetricsQdrUseEncryption:
   default: false
   description: Set to true if it is required to encrypt connection to the peer
                for listener.
   type: boolean
 MetricsQdrSaslMechanisms:
   default: 'ANONYMOUS'
   description: List of accepted SASL auth mechanisms for listener in format
                of comma separated list.
   type: string
 MetricsQdrSslCertDb:
   default: ''
   description: Path to SSL certificate db for listener.
   type: string
 MetricsQdrSslCertFile:
   default: ''
   description: Path to SSL certificate file for listener.
   type: string
 MetricsQdrSslKeyFile:
   default: ''
   description: Path to SSL private key file for listener.
   type: string
 MetricsQdrSslPwFile:
   default: ''
   description: Path to SSL password file for certificate key for listener.
   type: string
 MetricsQdrSslPassword:
   default: ''
   description: SSL password to be supplied for listener.
   type: string
 MetricsQdrTrustedCerts:
   default: ''
   description: Path to file containing trusted certificates for listener.
   type: string

Red Hat OpenStack Platform 14 is now supported on IBM POWER9 CPUs. This support is provided with the `rhosp-director-images-ppc64lep9` and `rhosp-director-images-ipa-ppc64lep9` packages.

Red Hat OpenStack Platform 14 is now supported on IBM POWER9 CPUs. This support is provided with the `rhosp-director-images-ppc64lep9` and `rhosp-director-images-ipa-ppc64lep9` packages.

This release adds a Technology Preview of the ability to attach a volume to multiple hosts or servers simultaneously in both cinder and nova with read/write (RW) mode when this feature is supported by the back end driver. This feature addresses the clustered application workloads use case that typically requires active/active or active/standby scenarios.

This release adds a Technology Preview of the ability to attach a volume to multiple hosts or servers simultaneously in both cinder and nova with read/write (RW) mode when this feature is supported by the back end driver. This feature addresses the clustered application workloads use case that typically requires active/active or active/standby scenarios.

This feature enables forwarding tenant traffic based on DSCP marking from tenants encapsulated in the VXLAN IP header. This feature is a technology preview for OSP14.

This feature enables forwarding tenant traffic based on DSCP marking from tenants encapsulated in the VXLAN IP header. This feature is a technology preview for OSP14.

You can now configure automatic restart of instances on a Compute node if the compute node reboots without first migrating the instances. Nova and the libvirt-guests agent can be configured to gracefully shut down the instances and start them when the Compute node reboots.

New parameters:
NovaResumeGuestsStateOnHostBoot (True/False)
NovaResumeGuestsShutdownTimeout (default 300s)

You can now configure automatic restart of instances on a Compute node if the compute node reboots without first migrating the instances. Nova and the libvirt-guests agent can be configured to gracefully shut down the instances and start them when the Compute node reboots.

New parameters:
NovaResumeGuestsStateOnHostBoot (True/False)
NovaResumeGuestsShutdownTimeout (default 300s)

The default value of `--http-boot` changed from `/httpboot` to
`/var/lib/ironic/httpboot` as containerized Ironic services
expect.

The default value of `--http-boot` changed from `/httpboot` to
`/var/lib/ironic/httpboot` as containerized Ironic services
expect.

With this update, logrotate's copytruncate is used by default for containerized services logs rotation. The default period to keep old logs remains unchanged (14 days).

With this update, logrotate's copytruncate is used by default for containerized services logs rotation. The default period to keep old logs remains unchanged (14 days).

OpenStack Rally, previously included as a technical preview, is removed from this release.

OpenStack Rally, previously included as a technical preview, is removed from this release.

When you use the web-download feature, the staging area - defined in the configuration using the `node_staging_uri` option - is not cleaned up properly. Ensure that `file` is part of the `stores` configuration option in the `glance_store` section of the glance-api.conf file.

When you use the web-download feature, the staging area - defined in the configuration using the `node_staging_uri` option - is not cleaned up properly. Ensure that `file` is part of the `stores` configuration option in the `glance_store` section of the glance-api.conf file.

When you use the image conversion feature, ensure that `file` is part of the `stores` configuration option in the `glance_store` section of the glance-api.conf file.

When you use the image conversion feature, ensure that `file` is part of the `stores` configuration option in the `glance_store` section of the glance-api.conf file.

For glance image conversion, the glance-direct method is not enabled by default. To enable this feature, set `enabled_import_methods` to `[glance-direct,web-download]` or `[glance-direct]` in the DEFAULT section of glance-api.conf.

For glance image conversion, the glance-direct method is not enabled by default. To enable this feature, set `enabled_import_methods` to `[glance-direct,web-download]` or `[glance-direct]` in the DEFAULT section of glance-api.conf.

Glance image conversion is not enabled by default on a new install of Red Hat OpenStack Platform 14. To use this feature, edit the glance-image-import.conf file.
In the image_import_opts section, insert the following line:
image_import_plugins = ['image_conversion']

Glance image conversion is not enabled by default on a new install of Red Hat OpenStack Platform 14. To use this feature, edit the glance-image-import.conf file.
In the image_import_opts section, insert the following line:
image_import_plugins = ['image_conversion']

OpenDaylight does not support IPv6 for tenant or provider networks. Therefore, use only IPv4 networks. You may experience issues related to floating IPs if IPv6 networks are used along with IPv4 networks.

OpenDaylight does not support IPv6 for tenant or provider networks. Therefore, use only IPv4 networks. You may experience issues related to floating IPs if IPv6 networks are used along with IPv4 networks.

The OvsDpdkMemoryChannels parameter cannot be derived through the DPDK derive parameters workflow. The value is set to 4 by default. You can change that value in your custom environments file to match your hardware.

The OvsDpdkMemoryChannels parameter cannot be derived through the DPDK derive parameters workflow. The value is set to 4 by default. You can change that value in your custom environments file to match your hardware.

When Octavia is configured to use a small Nova flavor, Amphorae (Nova instances) are created successfully but load balancers can get stuck in PENDING state for about 25 minutes. Instead, the load balancer should go to error state and the Amphorae should be deleted.

As a workaround for small Nova flavors, tune Octavia configurations "connection_max_retries", "connection_retry_interval", "build_active_retries" and "build_retry_interval" in section [haproxy_amphora] to a more reasonable production values. This will cause load balancers will transition from PENDING to ERROR state faster with a small Nova flavor.

When Octavia is configured to use a small Nova flavor, Amphorae (Nova instances) are created successfully but load balancers can get stuck in PENDING state for about 25 minutes. Instead, the load balancer should go to error state and the Amphorae should be deleted.

As a workaround for small Nova flavors, tune Octavia configurations "connection_max_retries", "connection_retry_interval", "build_active_retries" and "build_retry_interval" in section [haproxy_amphora] to a more reasonable production values. This will cause load balancers will transition from PENDING to ERROR state faster with a small Nova flavor.

Workflow triggers for generating Openstack rc files are hardcoded in python-tripleoclient.
As a result, OpenStack-specific workflows are triggered after director deploys OpenShift. Users can see OpenStack-specific URLs in stdout and OpenStack rc files created.

Workflow triggers for generating Openstack rc files are hardcoded in python-tripleoclient.
As a result, OpenStack-specific workflows are triggered after director deploys OpenShift. Users can see OpenStack-specific URLs in stdout and OpenStack rc files created.

There is currently a known issue with fernet token rotation where the keys are not automatically deployed onto the overcloud. The workflow task `tripleo.fernet_keys.v1.rotate_fernet_keys ` generates the keys but they are not successfully pushed to the overcloud. This issue is expected to be addressed in a future release. If you plan to perform rotation before this update, you can choose to follow one of these workarounds:
* Start os-collect-config on the overcloud nodes before running the rotation. You can then stop it afterwards if you do not need it for anything else.
* Enable os-collect-config on all overcloud nodes. You can choose to disable it once the update with the fix is released.
NOTE: If you do not need to rotate keys before the update comes out, then you do not need to do anything.

There is currently a known issue with fernet token rotation where the keys are not automatically deployed onto the overcloud. The workflow task `tripleo.fernet_keys.v1.rotate_fernet_keys ` generates the keys but they are not successfully pushed to the overcloud. This issue is expected to be addressed in a future release. If you plan to perform rotation before this update, you can choose to follow one of these workarounds:
* Start os-collect-config on the overcloud nodes before running the rotation. You can then stop it afterwards if you do not need it for anything else.
* Enable os-collect-config on all overcloud nodes. You can choose to disable it once the update with the fix is released.
NOTE: If you do not need to rotate keys before the update comes out, then you do not need to do anything.

Deploying the overcloud with a Gnocchi file backend might fail due to access permissions to the /var/lib/gnocchi/ directory.

Workaround: Before you deploy the overcloud, set the permissions to the directory in the openstack/tripleo-heat-templates/docker/services/gnocchi-api.yaml file as follows:

            - path:
                list_join:
                  - "/"
                  - - {get_param: GnocchiFileBasePath}
                    - "tmp"
              owner: gnocchi:gnocchi
              perm: '0600'
              recurse: true

Deploying the overcloud with a Gnocchi file backend might fail due to access permissions to the /var/lib/gnocchi/ directory.

Workaround: Before you deploy the overcloud, set the permissions to the directory in the openstack/tripleo-heat-templates/docker/services/gnocchi-api.yaml file as follows:

            - path:
                list_join:
                  - "/"
                  - - {get_param: GnocchiFileBasePath}
                    - "tmp"
              owner: gnocchi:gnocchi
              perm: '0600'
              recurse: true

On a director-deployed OpenShift environment, the GlusterFS playbooks auto-generate a new heketi secret key for each run.
As a result of this, operations such as scale out or configuration changes on CNS deployments fail.

As a workaround, complete the following steps:
1. Post-deployment, retrieve the heketi secret key. Use this command on one of the master nodes:
sudo oc get secret heketi-storage-admin-secret --namespace glusterfs -o json | jq -r .data.key | base64 -d
2. In an environment file, set the following parameters to that value:
  openshift_storage_glusterfs_heketi_admin_key
  openshift_storage_glusterfs_registry_heketi_admin_key

As a result of this workaround, operations such as scale out or configuration changes on CNS deployments work as long as the parameters were manually extracted.

On a director-deployed OpenShift environment, the GlusterFS playbooks auto-generate a new heketi secret key for each run.
As a result of this, operations such as scale out or configuration changes on CNS deployments fail.

As a workaround, complete the following steps:
1. Post-deployment, retrieve the heketi secret key. Use this command on one of the master nodes:
sudo oc get secret heketi-storage-admin-secret --namespace glusterfs -o json | jq -r .data.key | base64 -d
2. In an environment file, set the following parameters to that value:
  openshift_storage_glusterfs_heketi_admin_key
  openshift_storage_glusterfs_registry_heketi_admin_key

As a result of this workaround, operations such as scale out or configuration changes on CNS deployments work as long as the parameters were manually extracted.

When you restart all three controller nodes, it might not be possible to launch tenant instances in the overcloud. A "DuplicateMessageError" message  is logged in the overcloud logs.
As a workaround,  on one of the overcloud controllers, run this command:
pcs resource restart rabbitmq-bundle

When you restart all three controller nodes, it might not be possible to launch tenant instances in the overcloud. A "DuplicateMessageError" message  is logged in the overcloud logs.
As a workaround,  on one of the overcloud controllers, run this command:
pcs resource restart rabbitmq-bundle

For proxying requests to the routers on Infra nodes, director sets up port 443 on the HAProxy instance running on master nodes. Port 443 cannot be used on OpenShift master nodes for binding the OpenShift API. OpenShift API cannot be configured on port 443 on a director deployed OpenShift environment.

For proxying requests to the routers on Infra nodes, director sets up port 443 on the HAProxy instance running on master nodes. Port 443 cannot be used on OpenShift master nodes for binding the OpenShift API. OpenShift API cannot be configured on port 443 on a director deployed OpenShift environment.

The overcloud-full image provided by director causes an RPM conflict with the python-setuptools provided by the OpenShift repos. Any post-deployment yum update on the OpenShift nodes fails with a broken dependency issue.

To fix this, run the following on the undercloud:
source ~/stackrc
tripleo-ansible-inventory --stack openshift --static-yaml-inventory
/home/stack/openshift_inventory.yaml
export ANSIBLE_HOST_KEY_CHECKING=False
ansible -i openshift_inventory.yaml -m shell -b -a 'rpm -e --nodeps python-setuptools-0.9.8-7.el7.noarch; yum -y install python-setuptools' overcloud

This update replaces the python-setuptools provided by the overcloud-full image with the version provided by the OpenShift repos. Any subsequent yum updates are successful.

The overcloud-full image provided by director causes an RPM conflict with the python-setuptools provided by the OpenShift repos. Any post-deployment yum update on the OpenShift nodes fails with a broken dependency issue.

To fix this, run the following on the undercloud:
source ~/stackrc
tripleo-ansible-inventory --stack openshift --static-yaml-inventory
/home/stack/openshift_inventory.yaml
export ANSIBLE_HOST_KEY_CHECKING=False
ansible -i openshift_inventory.yaml -m shell -b -a 'rpm -e --nodeps python-setuptools-0.9.8-7.el7.noarch; yum -y install python-setuptools' overcloud

This update replaces the python-setuptools provided by the overcloud-full image with the version provided by the OpenShift repos. Any subsequent yum updates are successful.

In some OVS versions, `updelay` and `downdelay` bond settings are ignored, and the default settings are always used.

In some OVS versions, `updelay` and `downdelay` bond settings are ignored, and the default settings are always used.

Nova-compute ironic driver tries to update BM node while the node is being cleaned up. The cleaning takes approximately five minutes but nova-compute attempts to update the node for approximately two minutes. After timeout, nova-compute stops and puts nova instance into ERROR state.

As a workaround, set the following configuration option for nova-compute service:
[ironic]
api_max_retries = 180

As a result, nova-compute continues to attempt to update BM node longer and eventually succeeds.

Nova-compute ironic driver tries to update BM node while the node is being cleaned up. The cleaning takes approximately five minutes but nova-compute attempts to update the node for approximately two minutes. After timeout, nova-compute stops and puts nova instance into ERROR state.

As a workaround, set the following configuration option for nova-compute service:
[ironic]
api_max_retries = 180

As a result, nova-compute continues to attempt to update BM node longer and eventually succeeds.

The `neutron_driver` parameter has the value `null` in the containers-prepare-parameter.yaml file. This might cause minor updates to the overcloud in OpenDaylight deployments.

Workaround: Before you update the overcloud, set the value of the `neutron_driver` parameter to `odl`.

The `neutron_driver` parameter has the value `null` in the containers-prepare-parameter.yaml file. This might cause minor updates to the overcloud in OpenDaylight deployments.

Workaround: Before you update the overcloud, set the value of the `neutron_driver` parameter to `odl`.

Scaling out with an additional OpenShift master node of a director deployed OpenShift environment fails with a message similar to: "The field 'vars' has an invalid value, which includes an undefined variable. The error was: 'openshift_master_etcd_urls' is undefined…”

Scaling out with an additional OpenShift master node of a director deployed OpenShift environment fails with a message similar to: "The field 'vars' has an invalid value, which includes an undefined variable. The error was: 'openshift_master_etcd_urls' is undefined…”

Scaling out with an additional Infra node on a director deployed OpenShift environment with CNS enabled fails with a message similar to the following: “fatal: [openshift-master-2]: FAILED! => {"changed": false, "msg": "Error mounting /tmp/openshift-glusterfs-registry-c8qImT: Mount failed.”

Scaling out with an additional Infra node on a director deployed OpenShift environment with CNS enabled fails with a message similar to the following: “fatal: [openshift-master-2]: FAILED! => {"changed": false, "msg": "Error mounting /tmp/openshift-glusterfs-registry-c8qImT: Mount failed.”

Director and openshift-ansible have different expectations regarding image tags. For example, when importing the remote container images locally, director converts the generic tag into one that uniquely identifies the image based on the `version` and `release` labels from the image metadata. Openshift-ansible, however, relies on a unique `openshift_image_tag` variable for all the openshift images tags making it impossible to specify tags of images individually. Deployment of OCP via director fails when the floating v3.11 tag in the remote container image registry points to images with non-consistent `release` or `version` labels in their metadata.

From the undercloud, import the odd images prior to deploying OpenShift and set the tag to be consistent across all openshift images:

  skopeo --tls-verify=false copy docker://registry.access.redhat.com/openshift3/prometheus:v3.11.51-1 docker://192.168.24.1:8787/openshift3/prometheus:v3.11.51-2

Deployment of OpenShift from director completes without missing image.

Director and openshift-ansible have different expectations regarding image tags. For example, when importing the remote container images locally, director converts the generic tag into one that uniquely identifies the image based on the `version` and `release` labels from the image metadata. Openshift-ansible, however, relies on a unique `openshift_image_tag` variable for all the openshift images tags making it impossible to specify tags of images individually. Deployment of OCP via director fails when the floating v3.11 tag in the remote container image registry points to images with non-consistent `release` or `version` labels in their metadata.

From the undercloud, import the odd images prior to deploying OpenShift and set the tag to be consistent across all openshift images:

  skopeo --tls-verify=false copy docker://registry.access.redhat.com/openshift3/prometheus:v3.11.51-1 docker://192.168.24.1:8787/openshift3/prometheus:v3.11.51-2

Deployment of OpenShift from director completes without missing image.

Director does not support triggering Red Hat Enterprise Linux OS and OpenShift Container Platform updates on director deployed OpenShift environments. Director deployed OpenShift environments cannot be minor updated.

Director does not support triggering Red Hat Enterprise Linux OS and OpenShift Container Platform updates on director deployed OpenShift environments. Director deployed OpenShift environments cannot be minor updated.

After config-download has generated the playbooks for the Overcloud, if you execute ansible-playbook with --check parameter, it does not work. Expect an error about undefined stdout for ftype. This will be fixed in the next version.

After config-download has generated the playbooks for the Overcloud, if you execute ansible-playbook with --check parameter, it does not work. Expect an error about undefined stdout for ftype. This will be fixed in the next version.

Due to a known Ansible issue (https://github.com/ansible/ansible/issues/24449) changes to /etc/ssh/ssh_known_hosts are not propagated to existing nova_compute and nova_libvirt containers within an environment after adding compute hosts.

As a result, live migration, cold migration and instance resize operations using the newly introduced compute hosts will fail as a result as the original hosts will not have the required public SSH keys.

Workaround:
Restarting all nova_compute and nova_libvirt containers will ensure the updates to /etc/ssh/ssh_known_hosts are correctly written.

Detailed steps can be found in the following KCS article:

[OSP14] After successful scale out, live/cold migration and resize fails to the new added compute with 'Host key verification failed'
https://access.redhat.com/solutions/3792021

Due to a known Ansible issue (https://github.com/ansible/ansible/issues/24449) changes to /etc/ssh/ssh_known_hosts are not propagated to existing nova_compute and nova_libvirt containers within an environment after adding compute hosts.

As a result, live migration, cold migration and instance resize operations using the newly introduced compute hosts will fail as a result as the original hosts will not have the required public SSH keys.

Workaround:
Restarting all nova_compute and nova_libvirt containers will ensure the updates to /etc/ssh/ssh_known_hosts are correctly written.

Detailed steps can be found in the following KCS article:

[OSP14] After successful scale out, live/cold migration and resize fails to the new added compute with 'Host key verification failed'
https://access.redhat.com/solutions/3792021

A recent change made memory allocation for instances with NUMA topologies pagesize aware. With this change, memory for instances with NUMA topologies can no longer be oversubscribed.

Memory oversubscription is currently disabled for all instances with a NUMA topology, whereas previously only instances with hugepages were not allowed to use oversubscription. This affects instances with an explicit NUMA topology and those with an implicit topology. An instance can have an implicit NUMA topology due to the use of hugepages or CPU pinning.

If possible, avoid the use of explicit NUMA topologies. If CPU pinning is required, resulting in an implicit NUMA topology, there is no workaround.

A recent change made memory allocation for instances with NUMA topologies pagesize aware. With this change, memory for instances with NUMA topologies can no longer be oversubscribed.

Memory oversubscription is currently disabled for all instances with a NUMA topology, whereas previously only instances with hugepages were not allowed to use oversubscription. This affects instances with an explicit NUMA topology and those with an implicit topology. An instance can have an implicit NUMA topology due to the use of hugepages or CPU pinning.

If possible, avoid the use of explicit NUMA topologies. If CPU pinning is required, resulting in an implicit NUMA topology, there is no workaround.

OpenDaylight was first made available in OSP 13 and is being deprecated in OSP 14.

Our combined OpenDaylight in OpenStack solution will no longer accept new feature enhancements and we would like to inform those who were looking for an OpenDaylight integrated solution from Red Hat to seek alternatives.

OpenDaylight will continue to be supported and receive bug fixes for the duration of the OSP 14 deprecation cycle, with support planned to be completely dropped by the end of the OSP 13 lifecycle (June 27, 2021).

OpenDaylight was first made available in OSP 13 and is being deprecated in OSP 14.

Our combined OpenDaylight in OpenStack solution will no longer accept new feature enhancements and we would like to inform those who were looking for an OpenDaylight integrated solution from Red Hat to seek alternatives.

OpenDaylight will continue to be supported and receive bug fixes for the duration of the OSP 14 deprecation cycle, with support planned to be completely dropped by the end of the OSP 13 lifecycle (June 27, 2021).

This enhancement adds the boolean parameter `NovaLibvirtVolumeUseMultipath`, which provides a value for the multipath configuration parameter `libvirt/volume_use_multipath` in the `nova.conf` file for Compute nodes. You can set this parameter for each Compute role. Default value is `False`.

This enhancement adds the boolean parameter `NovaLibvirtVolumeUseMultipath`, which provides a value for the multipath configuration parameter `libvirt/volume_use_multipath` in the `nova.conf` file for Compute nodes. You can set this parameter for each Compute role. Default value is `False`.

This enhancement sets the number of RPC workers to `1` by default in OVN tripleo deployments. The goal of this setting is to reduce the number of workers to save memory resources and the number of connections to OVSDB, in cases where the Neutron DHCP agent is not deployed alongside OVN services.

This enhancement sets the number of RPC workers to `1` by default in OVN tripleo deployments. The goal of this setting is to reduce the number of workers to save memory resources and the number of connections to OVSDB, in cases where the Neutron DHCP agent is not deployed alongside OVN services.

This enhancement adds the networking-ansible heat parameter `IronicDefaultNetworkInterface`, which determines the value of the `default_network_interface` parameter in the `ironic.conf` configuration file. This value is set to the  `neutron` interface by default, which enables virtual networking through Neutron on bare metal nodes.

Note: The switches attached to the bare metal nodes must be programmable by the networking service if the `default_network_interface` is set to `neutron`.

This enhancement adds the networking-ansible heat parameter `IronicDefaultNetworkInterface`, which determines the value of the `default_network_interface` parameter in the `ironic.conf` configuration file. This value is set to the  `neutron` interface by default, which enables virtual networking through Neutron on bare metal nodes.

Note: The switches attached to the bare metal nodes must be programmable by the networking service if the `default_network_interface` is set to `neutron`.

There is currently a known issue where director can hang while deploying OCP. This occurs because the fix described in https://bugzilla.redhat.com/show_bug.cgi?id=1671861 is not a part of the `overcloud-full` image for the Red Hat OpenStack Platform 14 z1 release.
As a workaround, prior to deploying the overcloud, follow the steps below to update the docker package in the `overcloud-full` image. For more information on this procedure, see https://access.redhat.com/articles/1556833.
After completing these steps, you can expect the director to successfully deploy OCP:

$ sudo yum install -y libguestfs-tools
$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --install docker
$ source stackrc
$ openstack overcloud image upload --update-existing

There is currently a known issue where director can hang while deploying OCP. This occurs because the fix described in https://bugzilla.redhat.com/show_bug.cgi?id=1671861 is not a part of the `overcloud-full` image for the Red Hat OpenStack Platform 14 z1 release.
As a workaround, prior to deploying the overcloud, follow the steps below to update the docker package in the `overcloud-full` image. For more information on this procedure, see https://access.redhat.com/articles/1556833.
After completing these steps, you can expect the director to successfully deploy OCP:

$ sudo yum install -y libguestfs-tools
$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --install docker
$ source stackrc
$ openstack overcloud image upload --update-existing

This feature adds the capability to configure the Cinder Dell EMC StorageCenter driver to use a multipath for volume-to-image and image-to-volume transfers. The feature includes a new parameter `CinderDellScMultipathXfer` with a default value of `True`. Enabling multipath transfers can reduce the total time of data transfers between volumes and images.

This feature adds the capability to configure the Cinder Dell EMC StorageCenter driver to use a multipath for volume-to-image and image-to-volume transfers. The feature includes a new parameter `CinderDellScMultipathXfer` with a default value of `True`. Enabling multipath transfers can reduce the total time of data transfers between volumes and images.

Previously, when using TLS Everywhere, your controller node was required to access IdM through the `ctlplane` network. As a result, if traffic was routed through a different network, then the overcloud deployment process would fail due to `getcert` errors. To address this, IdM enrolment has been moved into a composable service that runs within `host_prep_tasks`; this runs at the start of the deployment phase. Note that the script will simply exit if the instance has already been enrolled in IdM.

Previously, when using TLS Everywhere, your controller node was required to access IdM through the `ctlplane` network. As a result, if traffic was routed through a different network, then the overcloud deployment process would fail due to `getcert` errors. To address this, IdM enrolment has been moved into a composable service that runs within `host_prep_tasks`; this runs at the start of the deployment phase. Note that the script will simply exit if the instance has already been enrolled in IdM.

Scaling out with an additional OpenShift master node of a director deployed OpenShift environment fails with a message similar to: "The field 'vars' has an invalid value, which includes an undefined variable. The error was: 'openshift_master_etcd_urls' is undefined…”

Scaling out with an additional OpenShift master node of a director deployed OpenShift environment fails with a message similar to: "The field 'vars' has an invalid value, which includes an undefined variable. The error was: 'openshift_master_etcd_urls' is undefined…”

Director and openshift-ansible have different expectations regarding image tags. For example, when importing the remote container images locally, director converts the generic tag into one that uniquely identifies the image based on the `version` and `release` labels from the image metadata. Openshift-ansible, however, relies on a unique `openshift_image_tag` variable for all the openshift images tags making it impossible to specify tags of images individually. Deployment of OCP via director fails when the floating v3.11 tag in the remote container image registry points to images with non-consistent `release` or `version` labels in their metadata.

From the undercloud, import the odd images prior to deploying OpenShift and set the tag to be consistent across all openshift images:

  skopeo --tls-verify=false copy docker://registry.access.redhat.com/openshift3/prometheus:v3.11.51-1 docker://192.168.24.1:8787/openshift3/prometheus:v3.11.51-2

Deployment of OpenShift from director completes without missing image.

Director and openshift-ansible have different expectations regarding image tags. For example, when importing the remote container images locally, director converts the generic tag into one that uniquely identifies the image based on the `version` and `release` labels from the image metadata. Openshift-ansible, however, relies on a unique `openshift_image_tag` variable for all the openshift images tags making it impossible to specify tags of images individually. Deployment of OCP via director fails when the floating v3.11 tag in the remote container image registry points to images with non-consistent `release` or `version` labels in their metadata.

From the undercloud, import the odd images prior to deploying OpenShift and set the tag to be consistent across all openshift images:

  skopeo --tls-verify=false copy docker://registry.access.redhat.com/openshift3/prometheus:v3.11.51-1 docker://192.168.24.1:8787/openshift3/prometheus:v3.11.51-2

Deployment of OpenShift from director completes without missing image.

A recent change made memory allocation for instances with NUMA topologies pagesize aware. With this change, memory for instances with NUMA topologies can no longer be oversubscribed.

Memory oversubscription is currently disabled for all instances with a NUMA topology, whereas previously only instances with hugepages were not allowed to use oversubscription. This affects instances with an explicit NUMA topology and those with an implicit topology. An instance can have an implicit NUMA topology due to the use of hugepages or CPU pinning.

If possible, avoid the use of explicit NUMA topologies. If CPU pinning is required, resulting in an implicit NUMA topology, there is no workaround.

A recent change made memory allocation for instances with NUMA topologies pagesize aware. With this change, memory for instances with NUMA topologies can no longer be oversubscribed.

Memory oversubscription is currently disabled for all instances with a NUMA topology, whereas previously only instances with hugepages were not allowed to use oversubscription. This affects instances with an explicit NUMA topology and those with an implicit topology. An instance can have an implicit NUMA topology due to the use of hugepages or CPU pinning.

If possible, avoid the use of explicit NUMA topologies. If CPU pinning is required, resulting in an implicit NUMA topology, there is no workaround.

There is currently a known issue where director can hang while deploying OCP. This occurs because the fix described in https://bugzilla.redhat.com/show_bug.cgi?id=1671861 is not a part of the `overcloud-full` image for the Red Hat OpenStack Platform 14 z1 release.
As a workaround, prior to deploying the overcloud, follow the steps below to update the docker package in the `overcloud-full` image. For more information on this procedure, see https://access.redhat.com/articles/1556833.
After completing these steps, you can expect the director to successfully deploy OCP:

$ sudo yum install -y libguestfs-tools
$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --install docker
$ source stackrc
$ openstack overcloud image upload --update-existing

There is currently a known issue where director can hang while deploying OCP. This occurs because the fix described in https://bugzilla.redhat.com/show_bug.cgi?id=1671861 is not a part of the `overcloud-full` image for the Red Hat OpenStack Platform 14 z1 release.
As a workaround, prior to deploying the overcloud, follow the steps below to update the docker package in the `overcloud-full` image. For more information on this procedure, see https://access.redhat.com/articles/1556833.
After completing these steps, you can expect the director to successfully deploy OCP:

$ sudo yum install -y libguestfs-tools
$ virt-customize --selinux-relabel -a overcloud-full.qcow2 --install docker
$ source stackrc
$ openstack overcloud image upload --update-existing

As of this release the director graphical user interface is deprecated. Bug fixes and support will be provided through the end of the OSP 13 lifecycle but no new feature enhancements will be made.

As of this release the director graphical user interface is deprecated. Bug fixes and support will be provided through the end of the OSP 13 lifecycle but no new feature enhancements will be made.

Red Hat OpenStack Platform director now has the ability to control Block Storage service (Cinder) snapshots on NFS back ends. A new director parameter, CinderNfsSnapshotSupport, has a default value of True.

Red Hat OpenStack Platform director now has the ability to control Block Storage service (Cinder) snapshots on NFS back ends. A new director parameter, CinderNfsSnapshotSupport, has a default value of True.

Prior to this release, the communication between hapoxy and the Shared File Systems service (Manila) API was not secured when deployed with TLS everywhere. Support has been added for the Manila API to configured with SSL certificates, allowing TLS on the internal API network. This feature is now automatically configured when TLS everywhere is enabled.

Prior to this release, the communication between hapoxy and the Shared File Systems service (Manila) API was not secured when deployed with TLS everywhere. Support has been added for the Manila API to configured with SSL certificates, allowing TLS on the internal API network. This feature is now automatically configured when TLS everywhere is enabled.

The API for the OpenStack Shared File Systems service (Manila) now runs behind httpd. The Apache error and access logs for this service are available in `/var/log/containers/httpd/manila-api` on all nodes that run the Manila API container. The logs for the main API remain in `/var/log/containers/manila`.

The API for the OpenStack Shared File Systems service (Manila) now runs behind httpd. The Apache error and access logs for this service are available in `/var/log/containers/httpd/manila-api` on all nodes that run the Manila API container. The logs for the main API remain in `/var/log/containers/manila`.

Previously, when the `PING` type health monitor was configured, HAProxy would silently use TCP connect instead. This is because Red Hat OpenStack Platform uses an older version of HAProxy that does not support external monitors. The setting `allow_ping_health_monitors` is now set to `False` by default.

Previously, when the `PING` type health monitor was configured, HAProxy would silently use TCP connect instead. This is because Red Hat OpenStack Platform uses an older version of HAProxy that does not support external monitors. The setting `allow_ping_health_monitors` is now set to `False` by default.

Director does not support triggering Red Hat Enterprise Linux OS and OpenShift Container Platform updates on director deployed OpenShift environments. Director deployed OpenShift environments cannot be minor updated.

Director does not support triggering Red Hat Enterprise Linux OS and OpenShift Container Platform updates on director deployed OpenShift environments. Director deployed OpenShift environments cannot be minor updated.