Chapter 9. Namespace auto-pruning architecture
For the namespace auto-pruning feature, two distinct database tables within a database schema were created: one for namespaceautoprunepolicy and another for autoprunetaskstatus. An auto-prune worker carries out the configured policies.
Namespace auto prune policy database table
The namespaceautoprunepolicy database table holds the policy configuration for a single namespace. There is only one entry per namespace, but there is support for multiple rows per namespace_id. The policy field holds the policy details, such as {method: "creation_date", olderThan: "2w"} or {method: "number_of_tags", numTags: 100}.
| Field | Type | Attributes | Description |
|---|---|---|---|
|
| character varying (225) | Unique, indexed | Unique identifier for this policy |
|
| Integer | Foreign Key | Namespace that the policy falls under |
|
| text | JSON | Policy configuration |
Auto-prune task status database table
The autoprunetaskstatus table registers tasks to be executed by the auto-prune worker. Tasks are executed within the context of a single namespace. Only one task per namespace exists.
| Field | Type | Attributes | Description |
|---|---|---|---|
|
| Integer | Foreign Key | Namespace that this task belongs to |
|
| Big Integer (bigint) | Nullable, indexed | Last time that the worker executed the policies for this namespace |
|
| text | Nullable | Details from the last execution task |
9.1. Auto-prune worker
The following sections detail information about the auto-prune worker.
9.1.1. Auto-prune-task-creation
When a new policy is created in the namespaceautoprunepolicy database table, a row is also created in the autoprunetask table. This is done in the same transaction. The auto-prune worker uses the entry in the autoprunetask table to identify which namespace it should execute policies for.
9.1.2. Auto-prune worker execution
The auto-pruning worker is an asynchronous job that executes configured policies. Its workflow is based on values in the autoprunetask table. When a task begins, the following occurs:
- The auto-prune worker starts on a set interval, which defaults at 30 seconds.
The auto-prune worker selects a row from
autoprunetaskwith the least, or null,last_ran_msandFOR UPDATE SKIP LOCKED.-
A null
last_ran_msindicates that the task was never ran. - A task that hasn’t been ran in he longest amount of time, or has never been run at all, is prioritized.
-
A null
The auto-prune worker obtains the policy configuration from the
namespaceautoprunepolicytable.-
If no policy configuration exists, the entry from
autoprunetaskis deleted for this namespace and the procedure stops immediately.
-
If no policy configuration exists, the entry from
The auto-prune worker begins a paginated loop of all repositories under the organization.
-
The auto-prune worker determines much pruning method to use based on
policy.method.
-
The auto-prune worker determines much pruning method to use based on
The auto-prune worker executes the pruning method with the policy configuration retrieved earlier.
- For pruning by the number of tags: the auto-pruner worker gets the number of currently active tags sorted by creation date, and deletes the older tags to the configured number.
- For pruning by date: the auto-pruner worker gets the active tags older than the specified time span and any tags returned are deleted.
- The auto-prune worker adds audit logs of the tags deleted.
-
The
last_ran_msgets updated after a row fromautoprunetaskis selected. - The auto-prune worker ends.
