Add NPM settings to the dependency proxy (!140848) · Merge requests · GitLab.org / GitLab

David Fernandez requested to merge 435642-add-npm-settings-to-dependency-proxy-for-packages into master Jan 03, 2024

🐝 Context

We are working on the "second" iteration of the dependency proxy for packages. This time around, we're targeting the NPM package format.

In a few words, with the dependency proxy, we can put GitLab as a middle man between a package manager (in this case $ npm or $ yarn) and an external package registry. GitLab will act as a pull-through cache.

This is the very first step for supporting the NPM package format. The challenge accessing the npm external registry is that $ npm has not one but two ways to send credentials (_auth and _authToken). Yes, I know 🤷.

Well, we will need to support both ways since external npm registries will usually support one or the other and the plan with the dependency proxy is to support as many of external registries as possible.

Thus, here are the settings:

the url. Well, that's obvious I guess 😸 . Required.
the basic_auth. The credentials string sent as basic auth. (We will need to send those with authorization: Basic <x>).
the auth_token. The token used as credentials. (We will need to send this as authorization: Bearer <x>).

Obviously, we will not accept both ways to authenticate to be set. Users will need to choose the correct way. This will be documented (outside of this MR). For this MR, this simply means that we have a model and database validation that will not allow both columns to be set.

Lasty but that's pretty obvious, both credentials columns will be encrypted. We are pretty much in the same situation as we were when adding the maven credentials: the documentation points to #26243 but unfortunately, ActiveRecord encryption is not ready to be used yet.

See NPM dependency proxy: add columns to settings m... (#435642 - closed).

The next MR/steps will be implementing the backend logic that will use those settings.

🤔 What does this MR do and why?

Add the settings to support the external npm package registries.
Add the related specs

🛵 MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

✅

🔭 Screenshots or screen recordings

Mmmmh. Model changes only so, no screenshots.

⚙ How to set up and validate locally

Again, it's just a model so not much to play with. Still, you can check the spec factory and play around with the validators.

In a rails console:

s = FactoryBot.create(:dependency_proxy_packages_setting, :npm)

s.npm_external_registry_basic_auth = "basic_auth"
=> "basic_auth"

s.npm_external_registry_auth_token = "auth token"
=> "auth token"

s.valid?
=> false

s.errors
=> #<ActiveModel::Errors [#<ActiveModel::Error attribute=base, type=Npm external registry basic auth and auth token can't be set at the same time, options={}>]>

💾 Database review

🔼 Migration up

main: == [advisory_lock_connection] object_id: 183080, pg_backend_pid: 86969
main: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: migrating =====
main: -- change_table(:dependency_proxy_packages_settings)
main:    -> 0.0051s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_12c046b67f\nCHECK ( char_length(npm_external_registry_url) <= 255 )\nNOT VALID;\n")
main:    -> 0.0019s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0001s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_12c046b67f;")
main:    -> 0.0006s
main: -- execute("RESET statement_timeout")
main:    -> 0.0001s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_54126e21c1\nCHECK ( octet_length(encrypted_npm_external_registry_basic_auth) <= 1020 )\nNOT VALID;\n")
main:    -> 0.0003s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_54126e21c1;")
main:    -> 0.0003s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_7fafb5606e\nCHECK ( octet_length(encrypted_npm_external_registry_basic_auth_iv) <= 1020 )\nNOT VALID;\n")
main:    -> 0.0003s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_7fafb5606e;")
main:    -> 0.0002s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_48643112c8\nCHECK ( octet_length(encrypted_npm_external_registry_auth_token) <= 1020 )\nNOT VALID;\n")
main:    -> 0.0003s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_48643112c8;")
main:    -> 0.0002s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_c8613a3d35\nCHECK ( octet_length(encrypted_npm_external_registry_auth_token_iv) <= 1020 )\nNOT VALID;\n")
main:    -> 0.0003s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_c8613a3d35;")
main:    -> 0.0002s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- transaction_open?(nil)
main:    -> 0.0000s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_93afb1690f\nCHECK ( num_nulls(encrypted_npm_external_registry_basic_auth, encrypted_npm_external_registry_auth_token) > 0 )\nNOT VALID;\n")
main:    -> 0.0003s
main: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_93afb1690f;")
main:    -> 0.0003s
main: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: migrated (0.0815s) 

main: == [advisory_lock_connection] object_id: 183080, pg_backend_pid: 86969
ci: == [advisory_lock_connection] object_id: 183580, pg_backend_pid: 86976
ci: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: migrating =====
ci: -- change_table(:dependency_proxy_packages_settings)
ci:    -> 0.0024s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_12c046b67f\nCHECK ( char_length(npm_external_registry_url) <= 255 )\nNOT VALID;\n")
ci:    -> 0.0015s
ci: -- execute("SET statement_timeout TO 0")
ci:    -> 0.0001s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_12c046b67f;")
ci:    -> 0.0006s
ci: -- execute("RESET statement_timeout")
ci:    -> 0.0001s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_54126e21c1\nCHECK ( octet_length(encrypted_npm_external_registry_basic_auth) <= 1020 )\nNOT VALID;\n")
ci:    -> 0.0003s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_54126e21c1;")
ci:    -> 0.0002s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_7fafb5606e\nCHECK ( octet_length(encrypted_npm_external_registry_basic_auth_iv) <= 1020 )\nNOT VALID;\n")
ci:    -> 0.0002s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_7fafb5606e;")
ci:    -> 0.0004s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_48643112c8\nCHECK ( octet_length(encrypted_npm_external_registry_auth_token) <= 1020 )\nNOT VALID;\n")
ci:    -> 0.0002s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_48643112c8;")
ci:    -> 0.0002s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_c8613a3d35\nCHECK ( octet_length(encrypted_npm_external_registry_auth_token_iv) <= 1020 )\nNOT VALID;\n")
ci:    -> 0.0002s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_c8613a3d35;")
ci:    -> 0.0002s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- transaction_open?(nil)
ci:    -> 0.0000s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings\nADD CONSTRAINT check_93afb1690f\nCHECK ( num_nulls(encrypted_npm_external_registry_basic_auth, encrypted_npm_external_registry_auth_token) > 0 )\nNOT VALID;\n")
ci:    -> 0.0003s
ci: -- execute("ALTER TABLE dependency_proxy_packages_settings VALIDATE CONSTRAINT check_93afb1690f;")
ci:    -> 0.0002s
ci: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: migrated (0.0277s) 

ci: == [advisory_lock_connection] object_id: 183580, pg_backend_pid: 86976

🔽 Migration down

main: == [advisory_lock_connection] object_id: 182640, pg_backend_pid: 88123
main: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: reverting =====
main: -- change_table(:dependency_proxy_packages_settings)
main:    -> 0.0037s
main: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: reverted (0.0063s) 

main: == [advisory_lock_connection] object_id: 182640, pg_backend_pid: 88123

ci: == [advisory_lock_connection] object_id: 182580, pg_backend_pid: 88529
ci: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: reverting =====
ci: -- change_table(:dependency_proxy_packages_settings)
ci:    -> 0.0012s
ci: == 20231228134512 NpmSettingsToDependencyProxyPackagesSettings: reverted (0.0076s) 

ci: == [advisory_lock_connection] object_id: 182580, pg_backend_pid: 88529

Edited Jan 30, 2024 by David Fernandez

Add NPM settings to the dependency proxy