Chapter 9. Namespace auto-pruning architecture
For the namespace auto-pruning feature, two distinct database tables within a database schema were created: one for namespaceautoprunepolicy
and another for autoprunetaskstatus
. An auto-prune worker carries out the configured policies.
Namespace auto prune policy database table
The namespaceautoprunepolicy
database table holds the policy configuration for a single namespace. There is only one entry per namespace, but there is support for multiple rows per namespace_id
. The policy
field holds the policy details, such as {method: "creation_date", olderThan: "2w"}
or {method: "number_of_tags", numTags: 100}
.
Field | Type | Attributes | Description |
---|---|---|---|
| character varying (225) | Unique, indexed | Unique identifier for this policy |
| Integer | Foreign Key | Namespace that the policy falls under |
| text | JSON | Policy configuration |
Auto-prune task status database table
The autoprunetaskstatus
table registers tasks to be executed by the auto-prune worker. Tasks are executed within the context of a single namespace. Only one task per namespace exists.
Field | Type | Attributes | Description |
---|---|---|---|
| Integer | Foreign Key | Namespace that this task belongs to |
| Big Integer (bigint) | Nullable, indexed | Last time that the worker executed the policies for this namespace |
| text | Nullable | Details from the last execution task |
9.1. Auto-prune worker
The following sections detail information about the auto-prune worker.
9.1.1. Auto-prune-task-creation
When a new policy is created in the namespaceautoprunepolicy
database table, a row is also created in the autoprunetask
table. This is done in the same transaction. The auto-prune worker uses the entry in the autoprunetask
table to identify which namespace it should execute policies for.
9.1.2. Auto-prune worker execution
The auto-pruning worker is an asynchronous job that executes configured policies. Its workflow is based on values in the autoprunetask
table. When a task begins, the following occurs:
- The auto-prune worker starts on a set interval, which defaults at 30 seconds.
The auto-prune worker selects a row from
autoprunetask
with the least, or null,last_ran_ms
andFOR UPDATE SKIP LOCKED
.-
A null
last_ran_ms
indicates that the task was never ran. - A task that hasn’t been ran in he longest amount of time, or has never been run at all, is prioritized.
-
A null
The auto-prune worker obtains the policy configuration from the
namespaceautoprunepolicy
table.-
If no policy configuration exists, the entry from
autoprunetask
is deleted for this namespace and the procedure stops immediately.
-
If no policy configuration exists, the entry from
The auto-prune worker begins a paginated loop of all repositories under the organization.
-
The auto-prune worker determines much pruning method to use based on
policy.method
.
-
The auto-prune worker determines much pruning method to use based on
The auto-prune worker executes the pruning method with the policy configuration retrieved earlier.
- For pruning by the number of tags: the auto-pruner worker gets the number of currently active tags sorted by creation date, and deletes the older tags to the configured number.
- For pruning by date: the auto-pruner worker gets the active tags older than the specified time span and any tags returned are deleted.
- The auto-prune worker adds audit logs of the tags deleted.
-
The
last_ran_ms
gets updated after a row fromautoprunetask
is selected. - The auto-prune worker ends.