Chapter 3. Release information RHOSO 18.0

Fix for instances created before OpenStack Victoria

In OpenStack Victoria, the instance_numa_topology object was extended to enable mix cpus (pinned and unpinned cpus) in the same instance. Object conversion code was added to handle upgrades but did not account for flavors that have either hw:mem_page_size or hw:numa_nodes set with hw:cpu_policy not set to dedicated

Setting hw-architecture or architecture on Image service (glance) image does not work as expected

In RHOSO 18.0, the image metadata prefilter is enabled by default. RHOSO does not support emulation of non-native architectures. As part of the introduction of emulation support upstream, the image metadata prefilter was enhanced to support the scheduling of instances based on the declared VM architecture, for example, hw_architecture=x86_64.

Compute service power management feature disabled by default

The Compute service (nova) power management feature is disabled by default. You can enable it with the following nova-compute configuration:

QEMU process failure

A paused instance that uses local storage cannot be live migrated more than once. The second migration causes the QEMU process to crash and nova puts the instance to ERROR state.

The value for edpm_kernel_hugepages is reliably set on the kernel command line.

Before this update, the value for edpm_kernel_hugepages could be missing from the kernel commandline due to an an error in an ansible role that configures it. With this update, this problem is resolved, and no work arounds are required.

Metadata rate-limiting feature

This update fixes a bug that prevented successful use of metadata rate-limiting. Metadata rate limiting is now available.

Router deletion problem and workaround

After an update to RHOSO 18.0.2, you cannot delete a pre-existing router as expected.

Do not use virtual functions (VF) for the RHOSO control plane interface

This RHOSO release does not support use of VFs for the RHOSO control plane interface.

OpenStack command output does not account for storage pool changes in the Shared File Systems service (manila)

The openstack share pool list command output does not account for storage pool changes, for example, changes to pool characteristics on back end storage systems, or removal of existing pools from the deployment. Provisioning operations are not affected by this issue. Workaround: Restart the scheduler service to reflect the latest statistics. Perform the restart during scheduled downtime because it causes a minor disruption.

Setting hw-architecture or architecture on Image service (glance) image does not work as expected

In RHOSO 18.0, the image metadata prefilter is enabled by default. RHOSO does not support emulation of non-native architectures. As part of the introduction of emulation support upstream, the image metadata prefilter was enhanced to support the scheduling of instances based on the declared VM architecture, for example, hw_architecture=x86_64.

Compute service power management feature disabled by default

The Compute service (nova) power management feature is disabled by default. You can enable it with the following nova-compute configuration:

QEMU process failure

A paused instance that uses local storage cannot be live migrated more than once. The second migration causes the QEMU process to crash and nova puts the instance to ERROR state.

Using the download-cache service no longer prevents Podman from pulling images for data plane deployment

Before this bug fix, if you included download-cache service in spec.services of the OpenStackDataPlaneNodeSet, a bug prevented Podman from pulling container images that are required by the data plane deployment.

Set edpm_kernel_args variable if you configure the Ansible variable edpm_kernel_hugepages

To configure the Ansible variable edpm_kernel_hugepages in the ansibleVars section of an OpenStackDataPlaneNodeSet CR, you must also set the edpm_kernel_args variable. If you do not need to configure edpm_kernel_args with a particular value, then set it to an empty string:

Support for security group logging on Compute nodes

With this update, when security group logging is enabled, RHOSO writes logs to the data plane node that hosts the project instance. In the /var/log/messages file, each log entry contains the string, acl_log.

Fixed delay between the oc patch command and update of OVN databases

Before this update, custom configuration settings applied with the oc patch command did not affect the Networking service (neutron) OVN databases until 10 minutes passed.

MAC_Binding aging functionality added back in RHOSO 18.0.1

The MAC_Binding aging functionality that was added in OSP 17.1.2 was missing from 18.0 GA. This update to RHOSO 18.0.1 adds it back.

Delayed OVN database update after oc patch command

Any custom configuration settings applied with the oc patch command do not affect the Networking service OVN databases until 10 minutes have passed.

Metadata rate-limiting feature

Metadata rate-limiting is not available in RHOSO 18.0.1. A fix is in progress.

DPDK bonds are now validated in os-net-config

Previously, when OVS or DPDK bonds were configured with a single port, no error was reported despite the ovs bridge not being in the right state. With this update os-net-config reports an error if the bond has a single interface.

Do not use virtual functions (VF) for the RHOSO control plane interface

This RHOSO release does not support use of VFs for the RHOSO control plane interface.

Image import no longer remains in importing state after conversion with ISO image format

Before this update, when you used image conversion with the ISO image format, the image import operation remained in an "importing" state.

Deploy metric storage with Telemetry Operator

The Telemetry Operator now supports deploying and operating Prometheus by using the cluster-observability-operator through a MonitoringStack resource.

Expanded interaction with metrics and alarms

You can now use the openstack metric and openstack alarm commands in the OpenStack CLI to interact with metrics and alarms. These commands are useful for troubleshooting.

Ceilometer uses TCP publisher to expose data for Prometheus

Ceilometer can now use the TCP publisher to publish metric data to sg-core, which exposes them for scraping by Prometheus.

Prometheus replaces Gnocchi for metrics storage and metrics-based autoscaling

In RHOSO 18.0, Prometheus replaces Gnocchi for metrics and metrics-based autoscaling.

Compute node log collection

RHOSO uses the Cluster Logging Operator (cluster-logging-operator) to collect and centrally store logs from OpenStack Compute nodes.

Graphing dashboards for OpenStack metrics

The Red Hat OpenShift Container Platform (RHOCP) console UI now provides graphing dashboards for OpenStack Metrics.

The compute service now supports native Secure RBAC

In osp 17.1 secure role-based access control was implemented using custom policy. In RHOSO-18.0.0 this is implemented using nova native support for SRBAC. As a result all OpenStack deployments support the ADMIN, MEMBER and READER roles by default.

Setting the hostname of the Compute service (nova) instance by using the Compute service API microversions 2.90 and 2.94

This enhancement enables you to set the hostname of the Compute service (nova) instance by using the Compute service API microversions 2.90 and 2.94 that are now included in the 18.0 release of RHOSO.

Manage dedicated CPU power state

You can now configure the nova-compute service to manage dedicated CPU power state by setting [libvirt]cpu_power_management to True.

Evacuate to STOPPED with v2.95

Starting with the v2.95 micro version, any evacuated instance will be stopped at the destination. Operators can still continue using the previous behaviour by selecting a microversion below v2.95. Prior to v2.95, if the VM was active prior to the evacuation, it was restored to the active state following a failed evacuation. If the workload encountered I/O corruption as a result of the hypervisor outage, this could potentially make recovery effort harder or cause further issues if the workload was a clustered application that tolerated the failure of a single VM. For this reason, it is considered safer to always evacuate to Stopped and allow the tenant to decide how to recover the VM.

Compute service hostname change

If you start the Compute service (nova) and your Compute host detects a name change, you must know the reason for the change of the host names. When you resolve the issue, you must restart the Compute service.

Create a neutron port without an IP address if the port requires only L2 network connectivity

You can now create an instance with a non-deferred port that has no fixed IP address if the network back end has L2 connectivity.

New enlightenments to the libvirt XML for Windows guests in RHOSO 18.0.0

This update adds the following enlightenments to the libvirt XML for Windows guests:

New default for managing instances on NUMA nodes

In RHOSP 17.1.4, the default was to pack instances on NUMA nodes.

Rebuild a volume-backed instance with a different image

This update adds the ability to rebuild a volume-backed instance from a different image.

Archive 'task_log' database records

This enhancement adds the --task-log option to the nova-manage db archive_deleted_rows CLI. When you use the  --task-log option, the task_log table records get archived while archiving the database. This option is the default in the nova-operator database purge cron job. Previously, there was no method to delete the task_log table without manual database modification.

Support for virtual IOMMU device

The Libvirt driver can add a virtual IOMMU device to guests. This capability applies to x86 hosts that use the Q35 machine type. To enable the capability, provide the hw:viommu_model extra spec or equivalent image metadata property hw_viommu_model.The following values are supported: intel, smmuv3, virtio, auto. The default value is auto, which automatically selects virtio.

More options for the server unshelve command

With this update, new options are added to the server unshelve command in RHOSO 18.0.0.

Support for the bochs libvirt video model

This release adds the ability to use the bochs libvirt video model. The bochs libvirt video model is a legacy-free video model that is best suited for UEFI guests. In some cases, it can be usable for BIOS guests, such as when the guest does not depend on direct VGA hardware access.

Schedule archival and purge of deleted rows from Compute service (nova) cells

The nova-operator now schedules a periodic job for each Compute service (nova) cell to archive and purge the deleted rows from the cell database. The frequency of the job and the age of the database rows to archive and purge can be fine tuned in the {{OpenStackControlPlane.spec.nova.template.cellTemplates[].dbPurge}} structure for each cell in the cellTemplates.

Migrating paused instance no longer generates error messages

Before this update, live migration of a paused instance with live_migration_permit_post_copy=True in nova.conf caused the libvirt driver to erroneously generate error messages similar to [1].

No network block device (NBD) live migration with TLS enabled

In RHOSO 18.0 Beta, a bug prevents you from using network block device (NBD) to live migrate storage between Compute nodes with TLS enabled. See https://issues.redhat.com/browse/OSPRH-6931.

Cannot delete instance when cpu_power_managment is set to true

in the rhos-18.0.0 beta release, a known issue was discovered preventing the deletion of an instance shortly after it was created if power management was enabled.

Technology preview of PCI device tracking in Placement service

RHOSO 18.0.0 introduces a technology preview of the ability to track PCI devices in the OpenStack Placement service.

Use of Identity service (Keystone) unified limits in the Compute service (nova)

This RHOSO release supports Identity service unified limits in the Compute service. Unified limits centralize management of resource quota limits in the Identity service (Keystone) and enable flexibility for users to manage quota limits for any Compute service resource being tracked in the Placement service.

Keypair generation removed from RHOSO 18

Keypair generation was deprecated in RHOSP 17 and has been removed from RHOSO 18. Now you need to precreate the keypair by the SSH command line tool ssh-keygen and then pass the public key to the nova API.

i440fx PC machine type no longer tested or supported

In RHOSP 17, the i440fx PC machine type, pc-i440fx, was deprecated and Q35 became the default machine type for x86_64.

Unsupported: vDPA and Hardware offload OVS are unsupported

Hardware offload OVS consists of processing network traffic in hardware with the kernel swtichdev and tcflower protocols.

Setting hw-architecture or architecture on Image service (glance) image does not work as expected

In RHOSO 18.0, the image metadata prefilter is enabled by default. RHOSO does not support emulation of non-native architectures. As part of the introduction of emulation support upstream, the image metadata prefilter was enhanced to support the scheduling of instances based on the declared VM architecture, for example hw_architecture=x86_64.

QEMU process failure

A paused instance that uses local storage cannot be live migrated more than once. The second migration causes the QEMU process to crash and nova puts the instance to ERROR state.

Compute service power management feature disabled by default

The Compute service (nova) power management feature is disabled by default. You can enable it with the following nova-compute configuration.

Using the download-cache service prevents Podman from pulling images for data plane deployment

Do not list the download-cache service in spec.services of the OpenStackDataPlaneNodeSet. If you list download-cache in OpenStackDataPlaneNodeSet, Podman can not pull the container images required by the data plane deployment.

Increased EFI partition size

Before RHOSP 17.1.4, the EFI partition size of an overcloud node was 16MB. With this update, the image used for provisioned EDPM nodes now has an EFI partition size of 200MB to align with RHEL and to accommodate firmware upgrades.

Octavia Operator availability zones

The Octavia Management network created and managed by the Octavia operator requires that the OpenStack routers and networks are scheduled on the OVN controller on the OpenShift worker nodes.

OVN pod no longer goes into loop due to NIC Mapping

When using a large number of NIC mappings, OVN could go into a creation loop. This is now fixed

QoS minimum bandwidth policy (technology preview)

In RHOSO 18.0.0, a technology preview is available for the Networking service (neutron) for QoS minimum bandwidth for placement reporting and scheduling.

Load-balancing service (Octavia) support of multiple VIP addresses

This update adds a technology preview of support for multiple VIP addresses allocated from the same Neutron network for the Load-balancing service.

Delayed OVN database update after oc patch command

Any custom configuration settings applied with 'oc patch …​' command do not affect neutron ovn databases until 10 minutes have passed.

MAC_Binding aging functionality missing in RHOSO 18.0.0

The MAC_Binding aging functionality that was added in OSP 17.1.2 is missing from 18.0 GA. A fix is in progress.

10-minute delay between 'oc patch`command and update of OVN databases

Custom configuration settings applied with the 'oc patch' command do not affect the Networking service (neutron) OVN databases until 10 minutes have passed.

Metadata rate-limiting feature

Metadata rate-limiting is not available in RHOSO 18.0.0. A fix is in progress.

AMD CPU powersave profiles

A power save profile, cpu-partitioning-powersave, was introduced in Red Hat Enterprise Linux 9 (RHEL 9), and made available in Red Hat OpenStack Platform (RHOSP) 17.1.3.

Physical function (PF) MAC address now matches between VM instances and SR-IOV physical functions (PFs)

This update fixes a bug that caused a PF MAC address mismatch between VM instances and SR-IOV PFs (Networking service ports with vnic-type set to direct-physical).

In RHOSO 18.0, a technology preview is available for the nmstate provider back-end in os-net-config.

This technology preview of nmstate and NIC hardware offload has known issues that make it unsuitable for production use. For production, use the openstack-network-scripts package rather than nmstate and NetworkManager.

Data Center Bridge (DCB)-based QoS settings technology preview

Specific to port/interface, DCB-based QoS settings are now available as a technology preview as part of the os-net-config tool’s network configuration template. For more information, see this knowledge base article: https://access.redhat.com/articles/7062865

TimeMaster service is deprecated in RHOSO 18.0

In RHOSO 18.0, support for the TimeMaster service is deprecated. Bug fixes and support are provided through the end of the RHOSO 18.0 lifecycle, but no new feature enhancements will be made.

Do not use virtual functions (VF) for the RHOSO control plane interface

This RHOSO release does not support use of VFs for the RHOSO control plane interface.

Bonds require minimum of two interfaces

If you configure an OVS or DPDK bond, always configure at least two interfaces. Bonds with only a single interface do not function as expected.

Password rotation

This update introduces the ability to generate and rotate OpenStack database passwords.

Shared File Systems support for scaleable CephFS-NFS

The Shared File Systems service (manila) now supports a scaleable CephFS-NFS service. In earlier releases of Red Hat OpenStack Platform, only active/passive high-availability that was orchestrated with Director, using Pacemaker/Corosync, was supported. With this release, deployers can create active/active clusters of CephFS-NFS and integrate these clusters with the Shared File Systems service for improved scalability and high availability for NFS workloads.

Block Storage service (cinder) volume deletion

With this release, the Block Storage service RBD driver takes advantage of recent Ceph developments to allow RBD volumes to meet normal volume deletion expectations.

project_id in API URLs now optional

You are no longer required to include project_id in Block Storage service (cinder) API URLs.

Dell PowerStore storage systems driver

A new share driver has been added to support Dell PowerStore storage systems with the Shared File Systems service (Manila) service.

Dell PowerFlex storage systems driver

A new share driver has been added to support Dell PowerFlex storage systems with the Shared File Systems service (Manila) service.

openstack-must-gather SOS report support

You can now collect diagnostic information about your RHOSO deployment using the openstack-must-gather.

Key Manager service configuration fix enables Image service image signing and verification

With this fix, the Image service (glance) is automatically configured to interact with the Key Manager service (barbican), and you can now perform encrypted image signing and verification.

Fixed faulty share creation in the NetApp ONTAP driver when using SVM scoped accounts

Due to a faulty kerberos enablement check upon shares creation, the NetApp ONTAP driver failed to create shares when configured with SVM scoped accounts. A fix has been committed to openstack-manila and shares creation should work smoothly.

Deployment and scale of Object Storage service

This feature allows for the deployment and scale of Object Storage service (swift) data on data plane nodes. This release of the feature is a technology preview.

RGW does not pass certain Tempest object storage metadata tests

Red Hat OpenStack Services on OpenShift 18.0 supports Red Hat Ceph Storage 7. Red Hat Ceph Storage 7 RGW does not pass certain Tempest object storage metadata tests as tracked by the following Jiras:

Image import remains in importing state after conversion with ISO image format

When you use image conversion with the ISO image format, the image import operation remains in an "importing" state.

Example:

glance image-create \
--name <iso_image> \
--disk-format iso \
--container-format bare \
--file <my_file.iso>

Hypervisor status now includes vCPU and pCPU information

Before this update, pCPU usage was excluded from the hypervisor status in the Dashboard service (horizon) even if the cpu_dedicated_set configuration option was set in the nova.conf file. This enhancement uses the Placement API to display information about vCPUs and pCPUs. You can view vCPU and pCPU usage diagrams under the Resource Providers Summary and find more information on vCPUs and pCPUs on the new Resource provider tab at the Hypervisors panel.

With this update, you can now customize the OpenStack Dashboard (horizon) container.

The customization can be performed by using the extra mounts feature to add or change files inside of the Dashboard container.

TLS everywhere in RHOSO Dashboard Operator

With this update, the RHOSO Dashboard (horizon) Operator automatically configures TLS-related configuration settings.

Host spoofing protective measure

Before this update, the hosts configuration option was not populated with the minimum hosts necessary to protect against host spoofing.

Dashboard service operators now include HSTS header

Before this update, HSTS was only enabled in Django through the Dashboard service (horizon) application. However, user HTTPS sessions were going through the OpenShift route, where HSTS was disabled. With this update, HSTS is enabled on the OpenShift route.

You can schedule archival and purge of deleted rows from Compute service (nova) cells

The nova-operator now schedules a periodic job for each Compute service (nova) cell to archive and purge the deleted rows from the cell database. The frequency of the job and the age of the database rows to archive and purge can be fine tuned in the {{OpenStackControlPlane.spec.nova.template.cellTemplates[].dbPurge}} structure for each cell in the cellTemplates.

i440fx PC machine type no longer tested or supported

In RHOSP 17, the i440fx PC machine type, pc-i440fx, was deprecated and Q35 became the default machine type for x86_64.

No network block device (NBD) live migration with TLS enabled

In RHOSO 18.0 Beta, a bug prevents you from using network block device (NBD) to live migrate storage between Compute nodes with TLS enabled. See https://issues.redhat.com/browse/OSPRH-6931.

Do not mix NUMA and non-NUMA instances on same Compute host

Instances without a NUMA topology should not coexist with NUMA instances on the same host.

Cannot delete instance when cpu_power_managment is set to true

When an instance is first started and the host core state is changed there is a short time period where it cannot be updated again. during this period instance deletion can fail. if this happens a second delete attempt should succeed after a short delay of a few seconds.

OVN pod goes into loop due to NIC Mapping

When using a large number of NIC mappings, OVN might go into a creation loop.

Listing physical function (PF) ports using neutron might show the wrong MAC

Lists of PF ports might show the wrong MAC.

Image uploads might fail if a multipathing path for Block Storage service (cinder) volumes is offline

If you use multipath for Block storage service volumes, and you have configured the Block Storage service as the back end for the Image service (glance), image uploads might fail if one of the paths goes offline.

RGW does not pass certain Tempest object storage metadata tests

Red Hat OpenStack Services on OpenShift 18.0 supports Red Hat Ceph Storage 7. Red Hat Ceph Storage 7 RGW does not pass certain Tempest object storage metadata tests as tracked by the following Jiras:

Missing Barbican configuration in the Image service (glance)

The Image service is not automatically configured to interact with Key Manager (barbican), and encrypted image signing and verification fails due to the missing configuration.

Removal of snmp and snmpd

The snmp service and snmpd daemon are removed in RHOSO 18.0.

Tempest test-operator does not work with LVMS storage class

When the test-operator is used to run Tempest, it requests a “ReadWriteMany” PersistentVolumeClaim (PVC) which the LVMS storage class does not support. This causes the tempest-test pod to become stuck in the pending state.