Chapter 1. Red Hat Quay release notes

1.1. RHBA-2024:11113 - Red Hat Quay 3.9.9 release

Issued 2024-12-19

Red Hat Quay release 3.9.9 is now available. The bug fixes that are included in the update are listed in the RHBA-2024:11113 advisory.

1.2. RHBA-2024:3922 - Red Hat Quay 3.9.8 release

Issued 2024-06-13

Red Hat Quay release 3.9.8 is now available. The bug fixes that are included in the update are listed in the RHBA-2024:3922 advisory.

1.2.1. Bug fixes

PROJQUAY-7295. Upgrade from 3.8 to 3.9, postgresql data lost

1.3. RHBA-2024:2835 - Red Hat Quay 3.9.7 release

Issued 2024-05-15

Red Hat Quay release 3.9.7 is now available. The bug fixes that are included in the update are listed in the RHBA-2024:2835 advisory.

1.3.1. Bug fixes

PROJQUAY-7070. Repository list API does not scale when quota is enabled.

1.4. RHBA-2024:0103 - Red Hat Quay 3.9.6 release

Issued 2024-1-16

Red Hat Quay release 3.9.6 is now available. The bug fixes that are included in the update are listed in the RHBA-2024:0103 advisory.

1.5. RHBA-2023:6852 - Red Hat Quay 3.9.5 release

Issued 2023-11-14

Red Hat Quay release 3.9.5 is now available. The bug fixes that are included in the update are listed in the RHBA-2023:6852 advisory.

1.5.1. OpenShift Container Platform and FIPs compliance

The following note applies to Red Hat Quay Operator deployments, or users running OpenShift Container Platform:

OpenShift Container Platform is designed for FIPS. When running Red Hat Enterprise Linux (RHEL) booted in FIPS mode, OpenShift Container Platform core components use the RHEL cryptographic libraries that have been submitted to NIST for FIPS 140-2/140-3 Validation on only the x86_64, ppc64le, and s390x architectures.

For more information about the NIST validation program, see Cryptographic Module Validation Program. For the latest NIST status for the individual versions of RHEL cryptographic libraries that have been submitted for validation, see Compliance Activities and Government Standards.

1.6. RHBA-2023:6124 - Red Hat Quay 3.9.4 release

Issued 2023-11-3

Red Hat Quay release 3.9.4 is now available. The bug fixes that are included in the update are listed in the RHBA-2023:6124 advisory.

1.7. RHBA-2023:5799 - Red Hat Quay 3.9.3 release

Issued 2023-10-17

Red Hat Quay release 3.9.3 is now available. The bug fixes that are included in the update are listed in the RHBA-2023:5799 advisory.

1.7.1. Bug fixes

PROJQUAY-6143. Quay 3.9.3 High Images Vulnerability Reported by Redhat ACS

1.8. RHBA-2023:5345 - Red Hat Quay 3.9.2 release

Issued 2023-09-26

Red Hat Quay release 3.9.2 is now available.

As of September 25, 2023, the Code Ready Dependency Analytics (CRDA) service for Java vulnerability matching is no longer usable with Clair. The service’s API moved to a different endpoint and there are no plans to update Clair to support this new endpoint. Instead, users should upgrade to Red Hat Quay 3.9 in order to keep getting CVE reports on Java Maven packages indexed by Clair from container images stored in Red Hat Quay, with the additional benefit of offline support and without the need for separate API keys.

The bug fixes that are included in the update are listed in the RHBA-2023:5345 advisory.

1.8.1. Bug fixes

PROJQUAY-5174. Quay Operator doesn’t trust internal service CA when it is rotated.
PROJQUAY-5931. Duplicate Robot accounts
PROJQUAY-5256. Storage replication not triggered on manifest list mirror

1.9. RHBA-2023:4974 - Red Hat Quay 3.9.1 release

Issued 2023-09-05

Red Hat Quay release 3.9.1 is now available with Clair 4.7.1. The bug fixes that are included in the update are listed in the RHBA-2023:4974 advisory.

1.9.1. Bug fixes

PROJQUAY-5581. Should show total quota consumption for user account namespace in UI.
PROJQUAY-5691. CVE-2023-33733 python-reportlab: remote code execution via supplying a crafted PDF file [quay-3.9].
PROJQUAY-5702. CVE-2023-36464 quay-registry-container: pypdf: Possible Infinite Loop when a comment isn’t followed by a character [quay-3].
PROJQUAY-5874. CVE-2021-33194 Vulnerabilities in dependency usr/local/bin/pushgateway (gobinary).
PROJQUAY-5925. A lot of quotatotalworker error in quayregistry-quay-config-editor pod log.
PROJQUAY-5914. Bulk update Repo settings in Robot accounts tab.
PROJQUAY-5967. Quay 3.9.1 High Image Vulnerability reported by Redhat ACS.

1.10. RHBA-2023:3256 - Red Hat Quay 3.9.0 release

Issued 2023-08-14

Red Hat Quay release 3.9.0 is now available with Clair 4.7. The bug fixes that are included in the update are listed in the RHBA-2023:3256 advisory.

1.10.1. Red Hat Quay release cadence

With the next release of Red Hat Quay, version 3.10, the product will begin to align its release cadence and lifecycle with OpenShift Container Platform. As a result, Red Hat Quay 3.10 will be generally available within approximately four weeks of the OpenShift Container Platform 4.14 release, which is currently scheduled for release in early Q4, 2024.

With the current release model, the total support length of Red Hat Quay 3.8 and Red Hat Quay 3.9 would have been cut short due to the release of Red Hat Quay 3.10 being scheduled earlier than previous releases. In order to provide customers with proper time to prepare for updates, the full support and maintenance phases of Red Hat Quay 3.8 and Red Hat Quay 3.9 have been amended to go beyond the release of Red Hat Quay 3.10. This is a one time amendment. After the release of Red Hat Quay 3.10 and subsequent releases, customers can expect the support lifecycle phases of Red Hat Quay to align with OpenShift Container Platform releases.

For more information, see the Red Hat Quay Life Cycle Policy.

1.10.2. Red Hat Quay new features and enhancements

The following updates have been made to Red Hat Quay:

1.10.2.1. Clair 4.7

Clair 4.7 was released as part of Red Hat Quay 3.9.

As of September 25, 2023, the Code Ready Dependency Analytics (CRDA) service for Java vulnerability matching will no longer be usable with Clair. The service’s API moved to a different endpoint and there are no plans to update Clair to support this new endpoint. Instead, users should upgrade to Red Hat Quay 3.9 in order to keep getting CVE reports on Java Maven packages indexed by Clair from container images stored in Red Hat Quay, with the additional benefit of offline support and without the need for separate API keys.

Additional enhancements to Clair include the following:

Native support for indexing Golang modules and RubeGems in container images.
Change to OSV.dev as the vulnerability database source for any programming language package managers.
- This includes popular sources like GitHub Security Advisories or PyPA.
- This allows offline capability.
Use of pyup.io for Python and CRDA for Java is suspended.
Clair now supports Java, Golang, Python, and Ruby dependencies.

1.10.2.2. Removal of a single site in a geo-replicated environment

Red Hat Quay administrators can now remove a specific site from their geo-replicated environment.

For more information, see Removing a geo-replicated site from your Red Hat Quay Operator deployment.

1.10.2.3. Quota management enhancements

Prior to Red Hat Quay 3.9, the quota management feature created totals by combining the manifest sizes at the repository and namespace level. This created an issue wherein a single blob could be counted multiple times within the total. For example, in previous versions of Red Hat Quay, if blobs were referenced multiple times within a repository and namespace, the blob was counted towards the allotted quota for every time it was referenced.
With this release, individual blob sizes are summed at the repository and namespace level. For example, if two tags in the same repository reference the same blob, the size of that blob is now only counted once towards the repository total. This enhancement to the quota management feature works by calculating the size of existing repositories and namespace with a backfill worker, and then adding or subtracting from the total for every image that is pushed or garbage collected afterwords. Additionally, the subtraction from the total happens when the manifest is garbage collected, whereas in the past it occurred when the tag was deleted.
Note
Because subtraction occurs from the total when the manifest is garbage collected, there is a delay in the size calculation until it is able to be garbage collected. For more information about Red Hat Quay garbage collection, see Red Hat Quay garbage collection.
Additionally, manifest list totals are now counted toward the repository total, the total quota consumed when upgrading from a previous version of Red Hat Quay might be reportedly differently in Red Hat Quay 3.9. In some cases, the new total might go over a repository’s previously-set limit. Red Hat Quay administrators might have to adjust the allotted quota of a repository to account for these changes.
Collectively, the quota management feature in Red Hat Quay 3.9 provides a more accurate depiction of storage growth and registry consumption. As a result, users can place quota limits on the namespace and repository sizes based on the actual usage of storage by Red Hat Quay.
For more information, see Quota management for Red Hat Quay 3.9

1.10.2.4. Configuring action log storage for Splunk

With this release, Red Hat Quay administrators can forward logs to a Splunk deployment. This allows administrators to perform log analyses and offload the internal database.

For more information, see Configuring action log storage for Splunk.

1.10.2.5. Red Hat Quay UI v2 enhancements

In Red Hat Quay 3.8, a new UI was introduced as a technology preview. With Red Hat Quay 3.9, the following enhancements have been made to the UI v2:

A tab for robot account creation.
A tab for Organization settings.
A tab for image tags.
A tab for Repository settings.
Overview, Security Reports, and Package vulnerability reports.

For more information about UI v2 enablement, see Using the Red Hat Quay v2 UI.

1.10.2.6. Nutanix Object Storage

With this release, Nutanix Object Storage is now supported. For more information, see Nutanix Object Storage.

1.10.3. New Red Hat Quay configuration fields

The following configuration fields have been added to Red Hat Quay 3.9:

The following configuration fields have been added to the quota management feature:
- QUOTA_BACKFILL: Enables the quota backfill worker to calculate the size of pre-existing blobs. Because this parameter sums the de-duplicated totals in the database, it might increase database load.
  Default: True
- QUOTA_TOTAL_DELAY_SECONDS:The time delay for starting the quota backfill. Rolling deployments can cause incorrect totals. This field must be set to a time longer than it takes for the rolling deployment to complete.
  Default: 1800
- PERMANENTLY_DELETE_TAGS: Enables functionality related to the removal of tags from the time machine window.
  Default: False
- RESET_CHILD_MANIFEST_EXPIRATION: Resets the expirations of temporary tags targeting the child manifests. With this feature set to True, child manifests are immediately garbage collected.
  Default: False

For more information, see Configuration updates for Red Hat Quay 3.9.

The following configuration field has been added to enhance the Red Hat Quay security scanner feature:
- FEATURE_SECURITY_SCANNER_NOTIFY_ON_NEW_INDEX: Whether to allow sending notifications about vulnerabilities for new pushes.
  Default: True
  For more information, see Security scanner configuration fields.
The following configuration field has been added to configure whether Red Hat Quay automatically removes old persistent volume claims (PVCs) when upgrading from version 3.8 3.9:
- POSTGRES_UPGRADE_RETAIN_BACKUP: When set to True, persistent volume claims from PostgreSQL 10 are backed up.
  Default: False
The following configuration field has been added to track various events:
- ACTION_LOG_AUDIT_LOGINS: When set to True, tracks advanced events such as logging into, and out of, the UI, and logging in using Docker for regular users, robot accounts, and for application-specific token accounts.
  Default: True

1.10.4. Red Hat Quay Operator

The following updates have been made to the Red Hat Quay Operator:

Currently, the Red Hat Quay Operator and Clair use PostgreSQL 10. PostgreSQL 10 had its final release on November 10, 2022 and is no longer supported.
With this release, if your database is managed by the Red Hat Quay Operator, updating from Red Hat Quay 3.8 3.9 automatically handles upgrading PostgreSQL 10 to PostgreSQL 13.
Important
Users with a managed database will be required to upgrade their PostgreSQL database from 10 13.
If you do not want the Red Hat Quay Operator to upgrade your PostgreSQL deployment from 10 13, you must set the PostgreSQL parameter to managed: false in your quayregistry.yaml file. For more information about setting your database to unmanaged, see Using an existing Postgres database.
Important
- It is highly recommended that you upgrade to PostgreSQL 13. PostgreSQL 10 had its final release on November 10, 2022 and is no longer supported. For more information, see the PostgreSQL Versioning Policy.
If you want your PostgreSQL database to match the same version as your Red Hat Enterprise Linux (RHEL) system, see Migrating to a RHEL 8 version of PostgreSQL for RHEL 8 or Migrating to a RHEL 9 version of PostgreSQL for RHEL 9.

For more information about the Red Hat Quay 3.8 3.9 procedure, see Upgrading the Red Hat Quay Operator overview.

1.10.5. Red Hat Quay 3.9 known issues and limitations

The following sections note known issues and limitations for Red Hat Quay 3.9.

1.10.5.1. Known issues:

1.10.5.1.1. Upgrading known issues

There are two known issues when upgrading your Red Hat Quay deployment:

If your Red Hat Quay deployment is upgrading from one y-stream to the next, for example, from 3.8.10 3.8.11, you must not switch the upgrade channel from stable-3.8 to stable-3.9. Changing the upgrade channel in the middle of a y-stream upgrade will disallow Red Hat Quay from upgrading to 3.9. This is a known issue and will be fixed in a future version of Red Hat Quay.
When upgrading from Red Hat Quay 3.7 to 3.9, you might receive the following error: pg_dumpall: error: query failed: ERROR: xlog flush request 1/B446CCD8 is not satisfied --- flushed only to 1/B0013858. As a workaround to this issue, you can delete the quayregistry-clair-postgres-upgrade job on your OpenShift Container Platform deployment, which should resolve the issue.

1.10.5.1.2. Other known issues

Using conftest pull commands to obtain policies might return the following error: Error: download policies: client get: stat /policy/quayregistry-quay-quay-enterprise-847.apps.quaytest-847.qe.devcluster.openshift.com/conftest/policy:latest: no such file or directory. As a workaround, you can add the oci:// prefix on your registry host. For example:
```
$ conftest pull oci://mkoktest.quaydev.org/admin/conftest:v1
```
This is a known issue and will be fixed in a future version of Red Hat Quay. (PROJQUAY-5573)
Red Hat Quay 3.9 introduced changes to the quota management feature. One of these changes is that tags in the time machine window now count towards the quota total of your organization.
There is a known issue when the proxy cache feature is enabled and configured in a new organization with a hard quota check and time machine settings set to longer than a few seconds under their organization settings. In sum, tags in a proxy organization are all given a tag expiration that defaults to 1 day. If your proxy organization has a time machine policy set to longer than a few seconds under your organization settings, and the tag expires, it is not immediately available for garbage collection; it must wait to be outside of the time machine window before it can be garbage collected. Because subtraction happens upon garbage collection, and pruned tags are kept within the time frame allotted by your organization’s settings, image tags are not immediately garbage collected. This results in the quota consumption metric not being updated, and runs the risk of your proxy organization going over the allotted quota.
When a hard quota check is configured for a proxy organization, Red Hat Quay administrators will want to reclaim the space taken by tags within the time machine window to prevent organizations from hitting their allotted quota. As a temporary workaround, you can set the time machine expiration for proxy organizations to a few seconds under Organizations Settings on the Red Hat Quay UI. This immediately removes image tags and allows for more accurate quota consumption metrics.
This is a non-issue for proxy organizations employing a soft quota check and can be ignored.
When removing a site from your geo-replicated Red Hat Quay deployment, you might receive the following error when running python -m util.removelocation: /app/lib/python3.9/site-packages/tzlocal/unix.py:141: SyntaxWarning: "is not" with a literal. Did you mean "!="? while start is not 0: /app/lib/python3.9/site-packages/netaddr/strategy/{}init{}.py:189: SyntaxWarning: "is not" with a literal. Did you mean "!="? if word_sep is not ''. You can confirm the deletion of your site by entering y. The error is a known issue and will be removed in a future version of Red Hat Quay.

1.10.5.2. Red Hat Quay 3.9 limitations

You must use the Splunk UI to view Red Hat Quay action logs. At this time, viewing Splunk action logs on the Red Hat Quay Usage Logs page is unsupported, and returns the following message: Method not implemented. Splunk does not support log lookups.

1.10.6. Red Hat Quay bug fixes

Previously, on Red Hat Quay Lightweight Directory Access Protocol (LDAP) deployments, there was a bug that disallowed referrals from being used with team synchronization and in other circumstances. With this update, referrals can be turned off globally for Red Hat Quay to ensure proper behavior across all components.
Previously, only last access timestamps were recorded in Red Hat Quay. This issue has been fixed, and now the following timestamps are recorded:
- Login to the Red Hat Quay UI.
- Logout of the Red Hat Quay UI.
- Login via Docker CLI (registry API) for regular users.
- Login via Docker CLI (Registry API) for robot accounts.
- Login via Docker CLI (Registry API) for app-specific tokens accounts.
  You can disable this timestamp feature by setting ACTION_LOG_AUDIT_LOGINS to false in your config.yaml file. This field is set to true by default.
  Note
  Logout events from the client side (Docker or Podman) are not causing requests to the registry API and are therefore not trackable.
PROJQUAY-4614. Add conftest mediatypes to default Quay configuration.
PROJQUAY-4865. Remove unused dependencies.
PROJQUAY-4957. Limit indexing of manifests that continuously fail.
PROJQUAY-5009. secscan: add api client timeout.
PROJQUAY-5018. Ignore unknown media types in manifests.
PROJQUAY-5237. The number of repositories in organization is incorrect in new UI.
PROJQUAY-4993. Support Action Log Forward to Splunk.
PROJQUAY-4567. Robot Tokens.
PROJQUAY-5289. Create a new username for accounts that login via SSO in the new UI.
PROJQUAY-5362. API: Add filtering to Tags API.
PROJQUAY-5207. Phase 3: Quay.io Summit Deliverables.
PROJQUAY-4608. Quay Operator should install a fully supported version of Postgres for Quay and Clair.
PROJQUAY-5050. Can’t provide a link to quay directly to an image that works in both old UI and new UI.
PROJQUAY-5253. Don’t convert dashes to underscores during first login.
PROJQUAY-4303. Multi-arch images are ignored in storage consumption calculation.
PROJQUAY-4304. Empty repositories are reporting storage consumption.
PROJQUAY-5634. oci: Allow optional components in the image config to be set to "null".
PROJQUAY-5639. Quay 3.9.0 delete organization under normal user by superuser was failed with unauthorized error.
PROJQUAY-5642. Quay 3.9.0 image High Vulnerability reported by Redhat ACS.
PROJQUAY-5630. Quay 3.9.0 Quay image High vulnerability issue CVE-2022-28948.

1.10.7. Red Hat Quay feature tracker

New features have been added to Red Hat Quay, some of which are currently in Technology Preview. Technology Preview features are experimental features and are not intended for production use.

Some features available in previous releases have been deprecated or removed. Deprecated functionality is still included in Red Hat Quay, but is planned for removal in a future release and is not recommended for new deployments. For the most recent list of deprecated and removed functionality in Red Hat Quay, refer to Table 1.1. Additional details for more fine-grained functionality that has been deprecated and removed are listed after the table.

Table 1.1. Technology Preview tracker
Feature	Quay 3.9	Quay 3.8	Quay 3.7
Single site geo-replication removal	General Availability	-	-
Splunk log forwarding	General Availability	-	-
Nutanix Object Storage	General Availability	-	-
Docker v1 support	Deprecated	Deprecated	General Availability
FEATURE_UI_V2	Technology Preview	Technology Preview	-
FEATURE_LISTEN_IP_VERSION	General Availability	General Availability	-
LDAP_SUPERUSER_FILTER	General Availability	General Availability	-
LDAP_RESTRICTED_USER_FILTER	General Availability	General Availability	-
FEATURE_SUPERUSERS_FULL_ACCESS	General Availability	General Availability	-
GLOBAL_READONLY_SUPER_USERS	General Availability	General Availability	-
FEATURE_RESTRICTED_USERS	General Availability	General Availability	-
RESTRICTED_USERS_WHITELIST	General Availability	General Availability	-
Quota management and enforcement	General Availability	General Availability	General Availability
Red Hat Quay build enhancements	General Availability	General Availability	General Availability
Red Hat Quay as proxy cache for upstream registries	General Availability	General Availability	Technology Preview
Geo-replication - Red Hat Quay Operator	General Availability	General Availability	General Availability
Advanced Clair configuration	General Availability	General Availability	General Availability
Support for Microsoft Azure Government (MAG)	General Availability	General Availability	General Availability
Java scanning with Clair	Technology Preview	Technology Preview	Technology Preview