Skip to content

Cascading namespace settings for package forwarding

Steve Abrams requested to merge 360267-cascade-package-forwarding-settings into master

🌳 What does this MR do and why?

When working with the package registry, we have the ability to turn on and off package forwarding at the instance level using a few application settings:

npm_package_requests_forwarding
pypi_package_requests_forwarding
maven_package_requests_forwarding

Package forwarding just means if a package is not found in the GitLab package registry, the request is forwarded (or redirected) to the standard public registry for that package format (for example npmjs.org or maven-central).

In !99274 (merged) we added columns at the namespace level to allow for namespace-level settings that can override the application setting. In this MR, we put these columns to use, using the cascading settings framework. This framework adds logic for when a namespace level setting should and should not override its parent namespace and/or the application setting. If the setting is NULL for a given namespace, then it finds the value from its parent group all the way up to the application setting. There is also a "lock" option where a group can enforce its setting value for all of its subgroups.

I've placed the usage of the namespace-level setting behind a feature flag so we can be certain it is safely working when we enable it on GitLab.com and revert back to only using the application setting if needed.

You'll notice the bulk of changes in this MR are the refactoring of a test file. There was only one cascading setting prior to this MR, so the test was specific to that setting. I refactored it so it could be applied to any cascading boolean setting. The content of that test file remains largely unchanged and there are no added or removed test cases, I just made the setting names variable so this can be used as a shared_examples for any future boolean cascading setting.

📸 Screenshots or screen recordings

See the section below, they contain the screenshots from my direct testing.

💻 How to set up and validate locally

You can test this with Maven, PyPI, or NPM. This explains how to do it with NPM:

  1. Create a group, a subgroup, and a project in that subgroup. Note the IDs of all of them.
  2. On your machine, create a new npm project to install packages to: 
    mkdir npm-forward && cd npm-forward
npm init # just enter through all of the options
  3. Using a personal access token with at least api scope, configure your npm project to point to your GitLab project. 
    npm config set @types:registry http://gdk.test:3000/api/v4/projects/35/packages/npm/
npm config set -- '//gdk.test:3000/api/v4/projects/35/packages/npm/:_authToken' "dNqw72htFTUiFFrdVbYR"
  4. The application setting npm_package_requests_forwarding defaults to true, so forwarding is already enabled.
  5. Install the package. It should be successful. You can watch log/development.log to see the requests coming into the GitLab instance. 
    ~ npm install @types/jest

added 50 packages, and audited 51 packages in 4s

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
  6. Remove the package so we can reinstall it again: rm -rf node_modules && rm package-lock.json
  7. Disable forwarding on the subgroup using the rails console 
    Group.last.package_settings.update!(npm_package_requests_forwarding: false)
  8. Install the package. It should still succeed because we have not yet enabled the feature flag. 
    ~ npm install @types/jest

added 50 packages, and audited 51 packages in 4s

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
  9. Remove the package so we can reinstall it again: rm -rf node_modules && rm package-lock.json
  10. Enable the feature flag 
    Feature.enable(:cascade_package_forwarding_settings)
  11. Install the package. It should now fail due to the subgroup setting overriding the application setting. 
    npm install @types/jest
npm ERR! code E404
npm ERR! 404 Not Found - GET http://gdk.test:3000/api/v4/projects/35/packages/npm/@types%2fjest
npm ERR! 404
npm ERR! 404  '@types/jest@^29.1.1' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/steveabrams/.npm/_logs/2022-10-06T14_16_55_607Z-debug-0.log
  12. Enable forwarding on the parent group and lock it: 
    Group.last.parent.package_settings.update!(npm_package_requests_forwarding: true, lock_npm_package_requests_forwarding: true)
  13. Install the package. It should be successful.
~ npm install @types/jest

added 50 packages, and audited 51 packages in 4s

4 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
  1. Remove the package so we can reinstall it again: rm -rf node_modules && rm package-lock.json
  2. Disable the lock on the parent group
Group.last.parent.package_settings.update!(lock_npm_package_requests_forwarding: false)
  1. Install the package. It should fail again because we are using the subgroup setting again.
npm install @types/jest
npm ERR! code E404
npm ERR! 404 Not Found - GET http://gdk.test:3000/api/v4/projects/35/packages/npm/@types%2fjest
npm ERR! 404
npm ERR! 404  '@types/jest@^29.1.1' is not in this registry.
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/steveabrams/.npm/_logs/2022-10-06T14_22_11_159Z-debug-0.log

🛃 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #360267 (closed)

Edited by Steve Abrams

Merge request reports