Chapter 4. Technical notes
This chapter supplements the information contained in the text of Red Hat OpenStack Platform "Wallaby" errata advisories released through the Content Delivery Network.
4.1. RHEA-2022:6543 — Release of components for OSP 17.0
Changes to the ceph component:
There is currently a known issue where the Swift API does not work and returns a 401 error when multiple Controller nodes are deployed and Ceph is enabled.
A workaround is available at https://access.redhat.com/solutions/6970061. (BZ#2112988)
Changes to the collectd component:
- In Red Hat OpenStack Platform 17.0, the collectd-write_redis plugin was removed. (BZ#2022714)
-
In Red Hat OpenStack Platform 17.0, a dependency has been removed from the distribution so that the subpackage
collectd-memcachec
cannot be built anymore. The collectd-memcached
plugin provides similar functionality to that ofcollectd-memcachec
. (BZ#2023893) - In Red Hat OpenStack Platform 17.0, the deprecated dbi and notify_email collectd plugins were removed. (BZ#2094409)
Changes to the distribution component:
- In Red Hat OpenStack Platform 17.0, panko and its API were removed from the distribution. (BZ#1966898)
- In Red Hat OpenStack Platform 17.0, the ability to deliver metrics from collectd to gnocchi was removed. (BZ#2065540)
Changes to the openstack-cinder component:
-
Before this update, if an operator defined a custom value for the
volume:accept_transfer
policy that referred to the project_id of the user making the volume transfer accept request, the request would fail. This update removes a duplicate policy check that incorrectly compared the project_id of the requestor to the project_id associated with the volume before transfer. The check done at the Block Storage API layer will now function as expected. (BZ#2050773) - With this enhancement, you can view a volume Encryption Key ID using the cinder client command 'cinder --os-volume-api-version 3.64 volume show <volume_name>'. You must specify microversion 3.64 to view the value. (BZ#1904086)
- Before this update, an issue existed with PowerFlex storage-assisted volume migration when volume migration was performed without conversion of volume type in cases where it should have been converted to thin from thick provisioned. With this update, this issue is fixed. (BZ#1883326)
- In this release, Block Storage service (cinder) backup support for Google Cloud Services (GCS) has been removed due to a reliance on libraries that are not FIPS compliant. (BZ#1984889)
Changes to the openstack-designate component:
- Before this update, a misconfiguration of communication parameters between the DNS service (designate) worker and deployed BIND instances caused Red Hat OpenStack Platform (RHOSP) 17.0 Beta deployments that have more than one Controller node to fail. With this update, this issue has been resolved, and you can now use the DNS service in a deployment with more than one Controller node. (BZ#1374002)
- In Red Hat OpenStack Platform 17.0, Secure RBAC is available for the DNS service (designate) as a technology preview. (BZ#1901687)
Changes to the openstack-ironic component:
- Before this update, Supermicro servers in UEFI mode would reboot from the network instead of from the local hard disk, causing a failed boot. With this update, Ironic sends the correct raw IPMI commands that request UEFI "boot from hard disk." Booting Supermicro nodes in UEFI mode with IPMI now works as expected. (BZ#1888069)
- This enhancement improves the operating performance of the Bare Metal Provisioning service (ironic) to optimize the performance of large workloads. (BZ#1954274)
-
Before this update, network interruptions caused a bare metal node’s power state to become
None
, and enter themaintenance
state. This is due to Ironic’s connection cache of Redfish node sessions entering a stale state and not being retried. This state cannot be recovered without restarting the Ironic service. With this update, the underlying REST client has been enhanced to return specific error messages. These error messages are used by Ironic to invalidate cached sessions. (BZ#2064019)
Changes to the openstack-ironic-inspector component:
Before this update, baremetal node introspection failed with an error and did not retry, when the node had a transient lock on it.
With this update, you can perform introspection even when the node has a lock. (BZ#1991657)
Changes to the openstack-manila component:
- With this update, the CephFS drivers in the OpenStack Shared File Systems service (manila) are updated so that you can manage provisioning and storage lifecycle operations by using the Ceph Manager API. When you create new file shares, the shares are created in a new format that is quicker for creating, deleting and operations. This transition does not affect pre-existing file shares. (BZ#1767084)
- With this update, you can restore snapshots with the CephFS Native and CephFS with NFS backends of the Shared File Systems service (manila) by creating a new share from a snapshot. (BZ#1699454)
Changes to the openstack-neutron component:
You can now migrate the mechanism driver to ML2/OVN from an ML2/OVS deployment that uses the iptables_hybrid firewall driver.
The existing instances keep using the hybrid plug mechanism after the migration, but security groups are implemented in OVN and there are no iptables rules present on the compute nodes. (BZ#2075038)
In an ML2/OVS deployment, Open vSwitch (OVS) does not support offloading OpenFlow rules that have the
skb_priority
,skb_mark
, or output queue fields set. Those fields are needed to provide quality-of-service (QoS) support for virtio ports.If you set a minimum bandwidth rule for a virtio port, the Neutron Open vSwitch agent marks the traffic of this port with a Packet Mark Field. As a result, this traffic cannot be offloaded, and it affects the traffic in other ports. If you set a bandwidth limit rule, all traffic is marked with the default 0 queue, which means no traffic can be offloaded.
As a workaround, if your environment includes OVS hardware offload ports, disable the packet marking in the nodes that require hardware offloading. After you disable the packet marking, it will not be possible to set rate limiting rules for virtio ports. However, differentiated services code point (DSCP) marking rules will still be available.
In the configuration file, set the
disable_packet_marking
flag totrue
. After you edit the configuration file, you must restart theneutron_ovs_agent
container. For example:$ cat `/var/lib/config-data/puppet-generated/neutron/etc/neutron/plugins/ml2/openvswitch_agent.ini` [ovs] disable_packet_marking=True
(BZ#2111015)
Changes to the openstack-nova component:
-
Before this update, the help text for the
max_disk_devices_to_attach
parameter did not state that0
is an invalid value. Also, when themax_disk_devices_to_attach
parameter was set to0
, thenova-compute
service started when it should have failed. With this update, themax_disk_devices_to_attach
parameter help option text states that a value of0
is invalid, and ifmax_disk_devices_to_attach
is set to0
, thenova-compute
service will now log an error and fail to start. (BZ#1801931)
Changes to the openstack-octavia component:
- With this update, the Red Hat OpenStack Platform (RHOSP) 17 Octavia amphora image now includes HAProxy 2.4.x as distributed in Red Hat Enterprise Linux (RHEL) 9. This improves the performance of Octavia load balancers; including load balancers using flavors with more than one vCPU core. (BZ#1813560)
- In Red Hat OpenStack Platform 17.0, secure role-based access control (RBAC) is available for the Load-balancing service (octavia) as a technology preview. (BZ#1901686)
Changes to the openstack-tripleo-common component:
- In RHOSP 17.0 you must use Ceph containers based on RHCSv5.2 GA content. (BZ#2111527)
Changes to the openstack-tripleo-heat-templates component:
-
With this update,
cephadm
andorchestrator
replace ceph-ansible. You can use director with cephadm to deploy the ceph cluster and additional daemons, and use a new `tripleo-ansible`role to configure and enable the Ceph backend. (BZ#1839169) -
With this update, Red Hat OpenStack Platform director deployed Ceph includes the RGW daemon, replacing the Object Storage service (swift) for object storage. To keep the Object Storage service, use the
cephadm-rbd-only.yaml
file instead ofcephadm.yaml
. (BZ#1758161) - With this update, you can now use Red Hat OpenStack Platform director to configure the etcd service to use TLS endpoints when deploying TLS-everywhere. (BZ#1848153)
-
In Red Hat OpenStack Platform 17.0, the
iscsi
deployment interface has been deprecated. The default deployment interface is nowdirect
. Bug fixes and support are provided while the feature is deprecated but Red Hat will not implement new feature enhancements. In a future release, the interface will be removed. (BZ#1874778) -
This enhancement changes the default machine type for each host architecture to Q35 (
pc-q35-rhel9.0.0
) for new Red Hat OpenStack Platform 17.0 deployments. The Q35 machine type provides several benefits and improvements, including live migration of instances between different RHEL 9.x minor releases, and the native PCIe hotplug that is faster than the ACPI hotplug used by thei440fx
machine type. (BZ#1946956) -
With this update, the default machine type is RHEL9.0-based Q35
pc-q35-rhel9.0.0
, with the following enhancements: - Live migration across RHEL minor releases.
- Native PCIe hotplug. This is also ACPI-based like the previous i440fx machine type.
- Intel input–output memory management unit (IOMMU) emulation helps protect guest memory from untrusted devices that are directly assigned to the guest.
- Faster SATA emulation.
- Secure boot. (BZ#1946978)
In Red Hat OpenStack Platform (RHOSP) 17.0 GA, for NIC-partitioned deployments, you can now pass through virtual functions (VFs) to VMs.
To pass through VFs, in a heat environment file, you must specify the VF product ID, vendor ID, and the physical function (PF) PCI addresses:
NovaPCIPassthrough: - product_id: "<VF_product_ID>" vendor_id: "<vendor_ID>" address: "<PF_PCI_addresses>" trusted: "true"
The PF PCI address parameter supports string and dict mapping. You can specify wildcard characters and use regular expressions when specifying one or more addresses.
Example
NovaPCIPassthrough: - product_id: "0x7b18" vendor_id: "0x8086" address: "0000:08:00.*" trusted: "true"
(BZ#1913862)
Before this update, the collectd smart plugin required the CAP_SYS_RAWIO capability to work. It was not added by default. With this update, you can add the capability to the collectd container and the smart plugin works. When you use the smart plugin, specify the following parameter in an environment file: CollectdContainerAdditionalCapAdd:
- "CAP_SYS_RAWIO" (BZ#1984556)
- In Red Hat OpenStack Platform 17.0, the collectd processes plugin has been removed from the default list of plugins. Loading the collectd processes plugin can cause logs to flood with messages, such as "procs_running not found". (BZ#2101948)
- In Red Hat OpenStack Platform (RHOSP) 17.0 GA, a technology preview is available for integration between the RHOSP Networking service (neutron) ML2/OVN and the RHOSP DNS service (designate). As a result, the DNS service does not automatically add DNS entries for newly created VMs. (BZ#1884782)
Changes to the openstack-tripleo-validations component:
- There is currently a known issue where 'undercloud-heat-purge-deleted' validation fails. This is because it is not compatible with Red Hat OpenStack Platform 17. Workaround: Skip 'undercloud-heat-purge-deleted' with '--skip-list' to skip this validation. (BZ#2105291)
Changes to the puppet-collectd component:
- With this enhancement you can use the PluginInstanceFormat parameter for collectd to specify more than one value. (BZ#1954103)
Changes to the python-octaviaclient component:
- This enhancement includes Octavia support for object tags. This allows users to add metadata to load balancer resources and filter query results based on tags. (BZ#1813573)
Changes to the python-openstackclient component:
- This enhancement includes OpenStack CLI (OSC) support for Block Storage service (cinder) API 3.42. This allows OSC to extend an online volume. (BZ#1689706)
Changes to the python-validations-libs component:
- This enhancement adds the '--limit' argument to the 'openstack tripleo validator show history' command. You can use this argument to show only a specified number of the most recent validations. (BZ#1944872)
With this update, the Validation Framework provides a configuration file in which you can set parameters for particular use. You can find an example of this file at the root of the code source or in the default location:
/etc/validation.cfg
.You can use the default file in
/etc/
or use your own file and provide it to the CLI with the argument--config
.When you use a configuration file there is an order for the variables precedence. The following order is the order of variable precedence:
- User’s CLI arguments
- Configuration file
- Default interval values (BZ#1971607)
-
With this update, you can supply a new argument
--skiplist
to thevalidation run
command. Use this command with ayaml
file containing services to skip when running validations. (BZ#2013120)
Changes to the tripleo-ansible component:
-
This security enhancement reduces the user privilege level required by the OpenStack Shared File System service (manila). You no longer need permissions to create and manipulate Ceph users, because the Shared File Systems service now uses the APIs exposed by the
Ceph Manager
service for this purpose. (BZ#1973356) -
You can now pre-provision bare metal nodes in your application by using the
overcloud node [un]provision
command. (BZ#2041429) - With this fix, traffic is distributed on VLAN provider networks in ML2/OVN deployments. Previously, traffic on VLAN provider networks was centralized even with the Distributed Virtual Router (DVR) feature enabled. (BZ#2101937)
Changes to the validations-common component:
- This update fixes a bug that incorrectly redirected registered non-stdout callback output from various Ansible processes to the validations logging directory. Output of other processes is no longer stored in validations logging directory. VF callbacks no longer receive information about plays, unless requested. (BZ#1944586)
4.2. RHBA-2023:0271 — Red Hat OpenStack Platform 17.0.1 bug fix and enhancement advisory
Changes to the openstack-aodh component:
- Before this update, the Alarming service (aodh) used a deprecated gnocchi API to aggregate metrics. This resulted in incorrect metric measures of CPU use in the gnocchi results. With this update, use of dynamic aggregation in gnocchi, which supports the ability to make reaggregations of existing metrics and the ability to make and transform metrics as required, resolves the issue. CPU use in gnocchi is computed correctly. (BZ#2133029)
Changes to the openstack-designate component:
- Before this update, the Red Hat OpenStack Platform (RHOSP) DNS service (designate) was unable to start its central process when TLS-everywhere was enabled. This was caused by an inability to connect to Redis over TLS. With this update in RHOSP 17.0.1, this issue has been resolved. (BZ#2121634)
Changes to the openstack-ironic-python-agent component:
- Before this update, deploying RHEL 8.6 images in UEFI mode caused a failure when using the ironic-python-agent because the ironic-python-agent service did not understand the RHEL 8.6 UEFI boot loader hint file. With this update, you can now deploy RHEL 8.6 in UEFI mode. (BZ#2135549)
Changes to the openstack-nova component:
- Before this update, an underlying RHEL issue caused a known issue with UEFI boot for instances. With this update, the underlying RHEL issue has now been fixed and the UEFI Secure Boot feature for instances is now available. (BZ#2106763)
Changes to the openstack-octavia component:
- Before this update, a race condition occurred in Octavia that may have caused OVN provider load balancers to become stuck in PENDING DELETE under certain conditions. This caused the load balancer to be immutable and unable to update. With this update, the race condition is fixed to resolve the issue. (BZ#2123658)
Changes to the openstack-tripleo-heat-templates component:
- Before this update, unavailability of the Podman log content caused the health check status script to fail. With this update, an update to the health check status script resolves the issue by using the Podman socket instead of the Podman log. As a result, API health checks, provided through sensubility for Service Telemetry Framework, are now operational. (BZ#2091076)
There is currently a known issue in RHOSP 17.0 where the Free Range Router (FRR) container does not start after the host on which it resides is rebooted. This issue is caused by a missing file in the BGP configuration. Workaround: Create the file,
/etc/tmpfiles.d/run-frr.conf
, and add the following line:d /run/frr 0750 root root - -
After you make this change,
tmpfiles
recreates/run/frr
after each reboot and the FRR container can start. (BZ#2127965)
Changes to the python-os-vif component:
-
Before this update,
ovsdb
connection time-outs caused thenova-compute
agent to become unresponsive. With this update, the issue has been fixed. (BZ#2085583)
Changes to the python-ovn-octavia-provider component:
-
Before this update, adding a member without subnet information when the subnet of the member is different than the subnet of the load balancer VIP caused the ovn-octavia provider to wrongly use the VIP subnet for the
subnet_id
which resulted in no error but no connectivity to the member. With this update, a check that the actual IP of the member belongs to the same CIDR that the VIP belongs to when there is no subnet information resolves the issue. If the two IP addresses do not match, the action is rejected, asking for thesubnet_id
. (BZ#2122926) Before this update, if an ovn-lb is created (VIP and members) in a LS (neutron network) that has 2 subnets (IPv4 and IPv6), and this LS is connected to a LR, removing the LS from the LR leads to the removal of the ovn-lb from the LS and consequently to remove it from the OVN SB DB as it is not associated to any datapath. When re-adding the LS to the LR (the network and subnets to the router) the ovn-lb will not be properly associated to the LR/LS at OVN level and there will be no connectivity
With this update the IP version is checked so that router ports that belong to other subnets are not considereed and the ovn-lb is not removed from the LS. This results in the ovn-lb having proper connectivity when a subnet is removed from the router. This resolves the issue. (BZ#2135270)
Changes to the tripleo-ansible component:
There is currently a known issue that causes tuned kernel configurations to not be applied after initial provisioning.
Workaround: You can use the following custom playbook to ensure that the tuned kernel command line arguments are applied. Save the following playbook as
/usr/share/ansible/tripleo-playbooks/cli-overcloud-node-reset-blscfg.yaml
on the undercloud node:- name: Reset BLSCFG of compute node(s) meant for NFV deployments hosts: allovercloud any_errors_fatal: true gather_facts: true pre_tasks: - name: Wait for provisioned nodes to boot wait_for_connection: timeout: 600 delay: 10 tasks: - name: Reset BLSCFG flag in grub file, if it is enabled become: true lineinfile: path: /etc/default/grub line: "GRUB_ENABLE_BLSCFG=false" regexp: "^GRUB_ENABLE_BLSCFG=.*" insertafter: '^GRUB_DISABLE_RECOVERY.*'
Configure the role in the node definition file,
overcloud-baremetal-deploy.yaml
, to run thecli-overcloud-node-reset-blscfg.yaml
playbook before the playbook that sets thekernelargs
:- name: ComputeOvsDpdkSriov count: 2 hostname_format: computeovsdpdksriov-%index% defaults: networks: - network: internal_api subnet: internal_api_subnet - network: tenant subnet: tenant_subnet - network: storage subnet: storage_subnet network_config: template: /home/stack/osp17_ref/nic-configs/computeovsdpdksriov.j2 config_drive: cloud_config: ssh_pwauth: true disable_root: false chpasswd: list: |- root:12345678 expire: False ansible_playbooks: - playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-node-reset-blscfg.yaml - playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-node-kernelargs.yaml extra_vars: reboot_wait_timeout: 600 kernel_args: 'default_hugepagesz=1GB hugepagesz=1G hugepages=32 iommu=pt intel_iommu=on isolcpus=1-11,13-23' tuned_profile: 'cpu-partitioning' tuned_isolated_cores: '1-11,13-23' - playbook: /usr/share/ansible/tripleo-playbooks/cli-overcloud-openvswitch-dpdk.yaml extra_vars: memory_channels: '4' lcore: '0,12' pmd: '1,13,2,14,3,15' socket_mem: '4096' disable_emc: false enable_tso: false revalidator: '' handler: '' pmd_auto_lb: false pmd_load_threshold: '' pmd_improvement_threshold: '' pmd_rebal_interval: '' nova_postcopy: true
(BZ#2107896)
-
Before this update, the
network_config
schema in the Bare Metal provisioning definition did not allow setting thenum_dpdk_interface_rx_queues
parameter which caused a schema validation error that blocked the Bare Metal node provisioning process. With this update, the schema validaton error no longer occurs when the 'num_dpdk_interface_rx_queues' parameter is used. This resolves the issue. (BZ#2140881)