Chapter 14. Red Hat Quay quota management and enforcement overview
With Red Hat Quay, administrators with superuser privileges have the ability to report storage consumption and to contain registry growth by establishing configured storage quota limits. Red Hat Quay superusers can manage the capacity limits of their environment in the following ways:
- Quota reporting: With this setting, an administrator can track the storage consumption of all of their organizations. Additionally, users can track the storage consumption of their assigned organization.
- Quota management: With this setting, an administrator can define soft and hard checks for Red Hat Quay users. Soft checks tell users if the storage consumption of an organization reaches their configured threshold. Hard checks prevent users from pushing to the registry when storage consumption reaches the configured limit.
Quota can be set for the entire registry, or for a specific repository within the registry.
Collectively, these features allow service owners of a Red Hat Quay registry to define service level agreements and support a healthy resource budget.
14.1. Quota management limitations Copy linkLink copied to clipboard!
One limitation of the quota management feature is that calculating resource consumption on the push of an artifact results in the calculation becoming part of the push’s critical path. Without this, usage data might drift.
The maximum storage quota size is dependent on the selected database:
Variable | Description |
---|---|
Postgres | 8388608 TB |
MySQL | 8388608 TB |
SQL Server | 16777216 TB |
14.2. Setting a system-wide default quota Copy linkLink copied to clipboard!
To specify a system-wide default storage quota that is applied to every organization and user, you can use the DEFAULT_SYSTEM_REJECT_QUOTA_BYTES configuration flag. When this field is set, and the quota limit has been met, the system automatically rejects new artifacts. By default, this configuration field is disabled.
If you configure a specific quota for an organization or user, and then delete that quota, the system-wide default quota applies if one has been set. Similarly, if you have configured a specific quota for an organization or user, and then modify the system-wide default quota, the updated system-wide default overrides any specific settings.
The following procedure shows you how to configure a system-wide default quota.
Procedure
Set a system-wide default storage quota by including the
DEFAULT_SYSTEM_REJECT_QUOTA_BYTES
field in yourconfig.yaml
file. For example:# ... DEFAULT_SYSTEM_REJECT_QUOTA_BYTES: 100gb # ...
# ... DEFAULT_SYSTEM_REJECT_QUOTA_BYTES: 100gb # ...
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Restart your Red Hat Quay registry.
14.3. Establishing quota for an organization by using the Red Hat Quay UI Copy linkLink copied to clipboard!
The following procedure describes how you can report storage consumption and establish storage quota limits for a repository.
Prerequisites
- A superuser account.
- Enough storage to meet the demands of quota limitations.
Procedure
Set
FEATURE_QUOTA_MANAGEMENT: True
in yourconfig.yaml
file and then restart your registry. For example:# ... FEATURE_QUOTA_MANAGEMENT: True # ...
# ... FEATURE_QUOTA_MANAGEMENT: True # ...
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Create a new organization or choose an existing one.
- Log in to the registry as a superuser and navigate to the Manage Organizations tab on the Super User Admin Panel. Click the Options icon of the organization for which you want to create storage quota limits.
- Click Configure Quota.
- For Set storage quota, enter the initial quota, for example, 10 MiB. Then, click Apply.
Optional: For Quota policy select one of the following Actions. Then, enter a Quota Threshold and click Add Limit.
- Reject: When this option is selected, any artifact that exceeds the established quota is rejected.
Warning: When this option is selected, users are notified of pushed artifacts that exceed the configured quota, however, the artifact successfully pushes.
NoteThe quota threshold percent determines when Red Hat Quay starts warning users that the repository is approaching its assigned storage quota.
Verification
Pull a sample artifact by entering the following command:
podman pull busybox
$ podman pull busybox
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Tag the sample artifact by entering the following command:
podman tag docker.io/library/busybox quay-server.example.com/testorg/busybox:test
$ podman tag docker.io/library/busybox quay-server.example.com/testorg/busybox:test
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Push the sample artifact to the organization by entering the following command:
podman push --tls-verify=false quay-server.example.com/testorg/busybox:test
$ podman push --tls-verify=false quay-server.example.com/testorg/busybox:test
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - Navigate to the Super User Admin Panel on the Red Hat Quay UI, then click Manage Organizations. The Organizations page shows the total proportion of the quota used by the artifact.
Optional: Pull a second sample artifact with intentions of exceeding the established quota by entering the following command:
podman pull nginx
$ podman pull nginx
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: Tag the second artifact by entering the following command:
podman tag docker.io/library/nginx quay-server.example.com/testorg/nginx
$ podman tag docker.io/library/nginx quay-server.example.com/testorg/nginx
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Optional: Push the second artifact to the organization by entering the following command:
podman push --tls-verify=false quay-server.example.com/testorg/nginx
$ podman push --tls-verify=false quay-server.example.com/testorg/nginx
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the artifact exceeds the defined quota, and you set the Quota policy to Reject, the following error message is returned:
denied: Quota has been exceeded on namespace
denied: Quota has been exceeded on namespace
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the artifact exceeds the defined quota, and you set the Quota policy to Warning, no error message is returned, and the image is successfully pushed.
Notifications for both Reject and Warning policies are also returned on the Red Hat Quay UI by clicking the bell icon.
14.4. Managing quota limits by using the API Copy linkLink copied to clipboard!
When an organization is first created, it does not have an established quota. You can use the API to check, create, change, or delete quota limitations for an organization.
Prerequisites
- You have generated an OAuth access token.
14.4.1. Setting quota by using the API Copy linkLink copied to clipboard!
Use the following procedure to set quota limitations by using the Red Hat Quay API.
Procedure
To set a quota for an organization, you can use the
POST /api/v1/organization/{orgname}/quota
endpoint:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
"Created"
"Created"
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Use the
GET /api/v1/organization/{orgname}/quota
command to see if your organization already has an established quota:curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' https://<quay-server.example.com>/api/v1/organization/<organization_name>/quota | jq
$ curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' https://<quay-server.example.com>/api/v1/organization/<organization_name>/quota | jq
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
[{"id": 1, "limit_bytes": 10737418240, "limit": "10.0 GiB", "default_config": false, "limits": [], "default_config_exists": false}]
[{"id": 1, "limit_bytes": 10737418240, "limit": "10.0 GiB", "default_config": false, "limits": [], "default_config_exists": false}]
Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can use the
PUT /api/v1/organization/{orgname}/quota/{quota_id}
command to modify the existing quota limitation. For example:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
{"id": 1, "limit_bytes": 21474836480, "limit": "20.0 GiB", "default_config": false, "limits": [], "default_config_exists": false}
{"id": 1, "limit_bytes": 21474836480, "limit": "20.0 GiB", "default_config": false, "limits": [], "default_config_exists": false}
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
14.4.2. Viewing quota usage by using the API Copy linkLink copied to clipboard!
Use the following procedure to view quota usage by using the Red Hat Quay API.
Procedure
To view the storage consumed,
GET
data from the /api/v1/repository endpoint:Sample command
curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/repository?last_modified=true&namespace=testorg&popularity=true&public=true' | jq
$ curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/repository?last_modified=true&namespace=testorg&popularity=true&public=true' | jq
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Sample output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To view the quota report for the repositories in the organization, use the /api/v1/repository endpoint:
Sample command
curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/repository?last_modified=true&namespace=testorg&popularity=true&public=true'
$ curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/repository?last_modified=true&namespace=testorg&popularity=true&public=true'
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Sample output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow To view the quota information in the organization details, use the /api/v1/organization/{orgname} endpoint:
Sample command
curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg' | jq
$ curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' 'https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg' | jq
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Sample output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
14.4.3. Setting reject and warning limits by using the API Copy linkLink copied to clipboard!
You can set reject and warning limits by using the Red Hat Quay API.
Procedure
To set reject and warning limits,
POST
data to the /api/v1/organization/{orgname}/quota/{quota_id}/limit endpoint. For example:curl -k -X POST -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' -d '{"type":"Reject","threshold_percent":80}' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota/1/limit
$ curl -k -X POST -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' -d '{"type":"Reject","threshold_percent":80}' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota/1/limit
Copy to Clipboard Copied! Toggle word wrap Toggle overflow curl -k -X POST -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' -d '{"type":"Warning","threshold_percent":50}' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota/1/limit
$ curl -k -X POST -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' -d '{"type":"Warning","threshold_percent":50}' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota/1/limit
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
14.4.4. Viewing reject and warning limits by using the API Copy linkLink copied to clipboard!
You can use the Red Hat Quay API to view reject and warning limits.
Procedure
View the reject and warning limits by using the /api/v1/organization/{orgname}/quota endpoint. For example:
curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota | jq
$ curl -k -X GET -H "Authorization: Bearer <token>" -H 'Content-Type: application/json' https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org/api/v1/organization/testorg/quota | jq
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Example output
Copy to Clipboard Copied! Toggle word wrap Toggle overflow
14.5. Calculating the total registry size Copy linkLink copied to clipboard!
Use the following procedure to queue a registry total calculation.
This feature is done on-demand. Calculating a registry total is database intensive. Use with caution.
Prerequisites
- You are logged in as a Red Hat Quay superuser.
Procedure
-
On the Red Hat Quay UI, click your username
Super User Admin Panel. - In the navigation pane, click Manage Organizations.
-
Click Calculate
Ok. - After a few minutes, depending on the size of your registry, refresh the page. The Total Registry Size is now calculated.
14.6. Permanently deleting an image tag Copy linkLink copied to clipboard!
In some cases, users might want to delete an image tag outside of the time machine window. Use the following procedure to manually delete an image tag permanently.
The results of the following procedure cannot be undone. Use with caution.
14.6.1. Permanently deleting an image tag using the Red Hat Quay v2 UI Copy linkLink copied to clipboard!
Use the following procedure to permanently delete an image tag using the Red Hat Quay v2 UI.
Prerequisites
-
You have set
FEATURE_UI_V2
toTrue
in yourconfig.yaml
file.
Procedure
Ensure that the
PERMANENTLY_DELETE_TAGS
andRESET_CHILD_MANIFEST_EXPIRATION
parameters are set toTrue
in yourconfig.yaml
file. For example:PERMANENTLY_DELETE_TAGS: true RESET_CHILD_MANIFEST_EXPIRATION: true
PERMANENTLY_DELETE_TAGS: true RESET_CHILD_MANIFEST_EXPIRATION: true
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - In the navigation pane, click Repositories.
- Click the name of the repository, for example, quayadmin/busybox.
- Check the box of the image tag that will be deleted, for example, test.
Click Actions
Permanently Delete. ImportantThis action is permanent and cannot be undone.
14.6.2. Permanently deleting an image tag using the Red Hat Quay legacy UI Copy linkLink copied to clipboard!
Use the following procedure to permanently delete an image tag using the Red Hat Quay legacy UI.
Procedure
Ensure that the
PERMANENTLY_DELETE_TAGS
andRESET_CHILD_MANIFEST_EXPIRATION
parameters are set toTrue
in yourconfig.yaml
file. For example:PERMANENTLY_DELETE_TAGS: true RESET_CHILD_MANIFEST_EXPIRATION: true
PERMANENTLY_DELETE_TAGS: true RESET_CHILD_MANIFEST_EXPIRATION: true
Copy to Clipboard Copied! Toggle word wrap Toggle overflow - On the Red Hat Quay UI, click Repositories and the name of the repository that contains the image tag you will delete, for example, quayadmin/busybox.
- In the navigation pane, click Tags.
- Check the box of the name of the tag you want to delete, for example, test.
-
Click the Actions drop down menu and select Delete Tags
Delete Tag. - Click Tag History in the navigation pane.
On the name of the tag that was just deleted, for example,
test
, click Delete test under the Permanently Delete category. For example:ImportantThis action is permanent and cannot be undone.