Make "allow anyone to pull" work with group-level Maven endpoints

Context

In GitLab package registry, there's a project-level setting that can allow anyone to pull from the package registry, regardless of the project's visibility.

That works fine for the project-level endpoints. However, it's not supported for the group-level endpoints.

In Maven Repository, we have one endpoint that we need to support the allow anyone to pull setting for: the Download a package file at the group-level endpoint.

Solution

We have a SQL query that says: within this group, collect all the public projects + all the projects where the user has reporter access.

We will need to update that to: within this group, collect all the public projects + all the projects where the user has reporter access + all the projects that have a public package registry.

The change is gated behind a feature flag.

What does this MR do and why?

• Modify Packages::Maven::PackageFinder & API::MavenPackages classes to support allow anyone to pull setting.
• Add the related specs.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Make sure you have a private project in a private or public group to use it as the package registry.

2. Open rails console:

   # Enable the ~"feature flag"
   Feature.enable(:allow_anyone_to_pull_public_maven_packages_on_group_level)
   
   # Enable `package_registry_allow_anyone_to_pull_option` application setting
   ApplicationSetting.last.update(package_registry_allow_anyone_to_pull_option: true)
   
   # Enable Allow anyone to pull from Package Registry in the private project from step 1
   Project.find(<id>).project_feature.update(package_registry_access_level: ::ProjectFeature::PUBLIC)
   
   # Create an external user that we are sure they dont have access to the group or project
   user = FactoryBot.create(:user, :external)
   # Keep the username of the user, we will use it later
   user.username
   
   # Create PAT for the external user, we will use it later
   pat =  FactoryBot.create(:personal_access_token, user: ext).token
   
   # stub file upload
   def fixture_file_upload(*args, **kwargs)
     Rack::Test::UploadedFile.new(*args, **kwargs)
   end
   
   # Create a maven package in the private project from step 1
   package = FactoryBot.create(:maven_package, project_id: <private_project_id>)
   
   # Keep the package path, we will use it in the download endpoint
   package.maven_metadatum.path
   
   # get a name of one of the package's files to test with:
   package.package_files.last.file_name

3. Try downloading the package using the group-level endpoint. The endpoint requires two params: path & file_name. We already have them from the console steps.

   It should work without passing any token and also with passing the external user PAT as the Private-Token header:

   curl --header "Private-Token: <personal_access_token>" "http://gdk.test:3000/api/v4/groups/<group_id>/-/packages/maven/<path>/<file_name>"

Related to #468059 (closed)

changed milestone to %17.4

added Category:Package Registry Maven Repository Package:P2 Stretch backend customer devopspackage featureenhancement grouppackage registry sectionci typefeature workflowin dev labels

assigned to @mkhalifa3

added pipelinetier-1 label

added database databasereview pending feature flag labels

	2 Warnings
	0d0d71e0: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.

	1 Message
	CHANGELOG missing: If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog. If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@panoskanell (UTC+3, 1 hour ahead of author)	@wandering_person (UTC+3, 1 hour ahead of author)
database	@johnmason (UTC-4, 6 hours behind author)	@tigerwnz (UTC+12, 10 hours ahead of author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

resolved all threads

added 1 commit

• 832b6452 - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

mentioned in issue #468059 (closed)

changed the description

requested review from @panoskanell

requested review from @johnmason

Hi @panoskanell

Are you available for the initial backend review here?

I would like to pass it down to one of the grouppackage registry's maintainers after you're done with your review: @10io, @dmeshcharakou or @radbatnag should be fine :)

---

Hi @johnmason

Are you available for the initial database review here?

Thanks both

changed the description

added workflowin review label and removed workflowin dev label

Thank you @mkhalifa3, just 1 comment from me!

Over to you

requested review from @panoskanell

approved this merge request

requested review from @radbatnag and removed review request for @panoskanell

added pipeline:mr-approved label

added pipelinetier-2 label and removed pipelinetier-1 label

Before you set this MR to auto-merge

This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.

Before you set this MR to auto-merge, please check the following:

• You are the last maintainer of this merge request
• The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
• This pipeline is recent enough (created in the last 8 hours)

If all the criteria above apply, please set auto-merge for this merge request.

See pipeline tiers and merging a merge request for more details.

E2E Test Result Summary

allure-report-publisher generated test report!

e2e-test-on-gdk: test report for a20e0ed0

expand test summary

+------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Govern      | 72     | 0      | 0       | 0     | 72    | ✅     |
| Create      | 128    | 0      | 16      | 0     | 144   | ✅     |
| Plan        | 73     | 0      | 0       | 0     | 73    | ✅     |
| Package     | 40     | 0      | 24      | 0     | 64    | ✅     |
| Verify      | 44     | 0      | 2       | 0     | 46    | ✅     |
| Data Stores | 31     | 0      | 1       | 0     | 32    | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Release     | 5      | 0      | 0       | 0     | 5     | ✅     |
| Secure      | 3      | 0      | 0       | 0     | 3     | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Fulfillment | 2      | 0      | 0       | 0     | 2     | ✅     |
| Manage      | 1      | 0      | 1       | 0     | 2     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 409    | 0      | 44      | 0     | 453   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+

e2e-test-on-omnibus: test report for a20e0ed0

expand test summary

+--------------------------------------------------------------+
|                        suites summary                        |
+---------+--------+--------+---------+-------+-------+--------+
|         | passed | failed | skipped | flaky | total | result |
+---------+--------+--------+---------+-------+-------+--------+
| Package | 48     | 0      | 28      | 0     | 76    | ✅     |
+---------+--------+--------+---------+-------+-------+--------+
| Total   | 48     | 0      | 28      | 0     | 76    | ✅     |
+---------+--------+--------+---------+-------+-------+--------+

added 1185 commits

• 832b6452...85ac4f1e - 1183 commits from branch master
• 18a4296e - Make "allow anyone to pull" work with group-level Maven endpoints
• 06c9d8bf - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

added 150 commits

• 06c9d8bf...14c3ea6d - 148 commits from branch master
• 0d0d71e0 - Make "allow anyone to pull" work with group-level Maven endpoints
• a20e0ed0 - Apply 1 suggestion(s) to 1 file(s)

Compare with previous version

approved this merge request

requested review from @tigerwnz and removed review request for @johnmason

approved this merge request

added databaseapproved label and removed databasereview pending label

added pipelinetier-3 pipeline:run-e2e-omnibus-once labels and removed pipelinetier-2 label

removed pipeline:run-e2e-omnibus-once label

approved this merge request

resolved all threads

started a merge train

removed this merge request from the merge train because the pipeline did not succeed. Learn more.

started a merge train

removed this merge request from the merge train because the pipeline did not succeed. Learn more.

started a merge train

merged

Hello @mkhalifa3

The database team is looking for ways to improve the database review process and we would love your help!

If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

@gitlab-org/database-team

And someone will be by shortly!

Thanks for your help!

This message was generated automatically. You're welcome to improve it.

mentioned in commit 9ed44224

added workflowstaging-canary label and removed workflowin review label

added workflowcanary label and removed workflowstaging-canary label

added workflowstaging label and removed workflowcanary label

added workflowproduction label and removed workflowstaging label

mentioned in issue #467396 (closed)

added workflowpost-deploy-db-staging label and removed workflowproduction label

added releasedcandidate label

mentioned in merge request !166115 (merged)

added releasedpublished label and removed releasedcandidate label