Add GraphQL support to manage the Packages Cleanup policy
♻ Context
The Packages Registry works with these core models (simplified):
flowchart LR
Group -- 1:n --> Project
Project -- 1:n --> Package
Package -- 1:n --> PackageFile
PackageFile -- 1:1 --> os((Object Storage file))
For some package formats, we allow republishing a package version. What happens is that we append the package files to the existing package.
With time, some packages can end up with many package files. All these package files take space on Object Storage.
With #346153 (closed), we're kicking off the work for adding cleanup policies for packages. Basically, users will be allowed to define cleanup rules and the backend will regularly execute the policy to remove packages/package files that are not kept by the policy.
In true iteration spirit, the first iteration will have a single rule. Users will be able to define how many duplicated package files (by filename) need to be kept.
Example: for maven package, a pom.xml
is uploaded on each publication. If you publish the same version 100 times, you end up with 100 pom.xml
package files. Users will be able to state that they only want to keep the 10 most recent pom.xml
files.
For this feature, there are 3 parts:
- The policy model. That's !85918 (merged).
- Expose the policy object through GraphQL
👈 This MR.- This is needed by the frontend to display the policy and its "rules".
- The background job that executes the cleanup policies.
As stated above, this MR introduces the GraphQL pieces to manage the packages cleanup policy.
Lastly, understand that we're setting up things for the first iteration of packages cleanup policies with only 1
rule. At some point down the road, we're going to expand these policies with more whistles and bells (eg. more rules that users can define).
What does this MR do and why?
- Add a GraphQL Query to read the packages cleanup policy of a given project.
- Add a GraphQL Mutation to create/update the packages cleanup policy for a given project.
- Add a service to create/update a given policy. This service is used by the GraphQL Mutation.
- Add all related specs.
- Authorization wise, we're using the
admin_package
that we introduced in !86281 (merged). The difference here is that we need that permission in the Project policy.- We introduced this permission to gate all Package Registry settings operations. Since a Packages cleanup policy is considered a Package Registry setting, we simply used that same permission.
- We're using the same rules here: only
maintainer
andowner
will have this permission.
📺 Screenshots or screen recordings
-
Let's see the policy of the first project:
-
GraphQL query
{ project(fullPath: "gitlab-org/gitlab-test") { id name packagesCleanupPolicy { keepNDuplicatedPackageFiles nextRunAt } } }
-
GraphQL response
{ "data": { "project": { "id": "gid://gitlab/Project/1", "name": "Gitlab Test", "packagesCleanupPolicy": { "keepNDuplicatedPackageFiles": "ALL_PACKAGE_FILES", "nextRunAt": null } } } }
-
-
Let's update the policy. We feel adventurous, so we want to keep only
1
package file when there are duplications:-
GraphQL query
mutation { updatePackagesCleanupPolicy(input: { projectPath: "gitlab-org/gitlab-test", keepNDuplicatedPackageFiles: ONE_PACKAGE_FILE }) { packagesCleanupPolicy { keepNDuplicatedPackageFiles nextRunAt } } }
-
GraphQL response
{ "data": { "updatePackagesCleanupPolicy": { "packagesCleanupPolicy": { "keepNDuplicatedPackageFiles": "ONE_PACKAGE_FILE", "nextRunAt": "2022-05-18T03:04:43Z" } } } }
-
The policy is created with the value we want
✅
-
-
Running the query again gives us:
-
GraphQL response
{ "data": { "project": { "id": "gid://gitlab/Project/1", "name": "Gitlab Test", "packagesCleanupPolicy": { "keepNDuplicatedPackageFiles": "ONE_PACKAGE_FILE", "nextRunAt": "2022-05-18T03:04:43Z" } } } }
-
All good
✅
-
Now, let's see what happens with a developer
member.
- Trying to read the policy with the query returns:
-
GraphQL response
{ "data": { "project": { "id": "gid://gitlab/Project/1", "name": "Gitlab Test", "packagesCleanupPolicy": null } } }
-
- Trying to update the policy with the mutation returns:
-
GraphQL response
{ "data": { "updatePackagesCleanupPolicy": null }, "errors": [ { "message": "The resource that you are attempting to access does not exist or you don't have permission to perform this action", "locations": [ { "line": 2, "column": 3 } ], "path": [ "updatePackagesCleanupPolicy" ] } ] }
-
- Both access are denied
✅
⚗ How to set up and validate locally
- Load the graphql editor:
<gdk host>/-/graphql-explorer
- Run the queries the we used above.
🚓 MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.