Include source in composer json payload (!56693) · Merge requests · GitLab.org / GitLab

Steve Abrams requested to merge 247531-composer-source-jsonn into master Mar 15, 2021

🏠 Context

When working with Composer PHP packages, a user might run composer update or composer req my-pkg:1.0.0 to install a package. There are multiple ways for the Composer client to then fetch the package. The default way is to let the Composer client request the package version metadata, and then use whatever URL the registry returns to retrieve the package. This is currently how the GitLab Composer registry operates. In the response to the request for package version metadata, we include a 'dist' section, that includes a URL pointing to the download route for the archived package.

There is another option to allow users to download a package directly from source, meaning, directly from the git repository where the package code lives. This is the --prefer-source option. So if a user uses composer update --prefer-source, the Composer client knows to not use the 'dist' url, but instead, favor the 'source' url. The source url points to a git repository, has a reference (commit hash), and specifies that the source is a 'git' repository. With that information, the Composer client can then download the code directly from the repository.

The problem is, GitLab currently does not supply a 'source' section in the metadata payload, so the GitLab registry does not support use of --prefer-source.

🔍 What does this MR do?

Adds a 'source' section to the Composer version metadata response that contains the git repository URL. This allows users to use the --prefer-source option when installing Composer packages.

📸 Screenshots (strongly suggested)

Without --prefer-source option

→ composer update -vvv Reading ./composer.json (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.json) Loading config file /Users/steveabrams/.composer/config.json Loading config file /Users/steveabrams/.composer/auth.json Loading config file ./composer.json (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.json) Checked CA file /private/etc/ssl/cert.pem: valid

Running 2.0.11 (2021-02-24 14:57:23) with PHP 7.3.11 on Darwin / 19.6.0 Loading composer repositories with package information Warning: Accessing gdk.test over http which is an insecure protocol. Using HTTP basic authentication with username "token" Downloading http://gdk.test:3001/api/v4/group/153/-/packages/composer/packages.json [200] http://gdk.test:3001/api/v4/group/153/-/packages/composer/packages.json Writing /Users/steveabrams/.composer/cache/repo/http---gdk.test-3001-api-v4-group-153---packages-composer-packages.json/packages.json into cache Downloading http://gdk.test:3001/api/v4/group/153/-/packages/composer/p2/foo/composer-test.json [200] http://gdk.test:3001/api/v4/group/153/-/packages/composer/p2/foo/composer-test.json Writing /Users/steveabrams/.composer/cache/repo/http---gdk.test-3001-api-v4-group-153---packages-composer-packages.json/provider-foo~composer-test.json into cache Updating dependencies Generating rules Resolving dependencies through SAT Looking at all rules.

Dependency resolution completed in 0.000 seconds Analyzed 86 packages to resolve dependencies Analyzed 86 rules to resolve dependencies Lock file operations: 1 install, 0 updates, 0 removals Installs: foo/composer-test:1.0.0 - Locking foo/composer-test (1.0.0) Writing lock file Installing dependencies from lock file (including require-dev) Reading ./composer.lock (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.lock) Package operations: 1 install, 0 updates, 0 removals Installs: foo/composer-test:1.0.0 - Downloading foo/composer-test (1.0.0) Downloading http://gdk.test:3001/api/v4/projects/61/packages/composer/archives/foo/composer-test.zip?sha=98298a129ca79d3c1c55a6651993ac01109e34ae [200] http://gdk.test:3001/api/v4/projects/61/packages/composer/archives/foo/composer-test.zip?sha=98298a129ca79d3c1c55a6651993ac01109e34ae Writing /Users/steveabrams/.composer/cache/files/foo/composer-test/587bfaa79eec55a422b392a5529929f8e5b816f3.zip into cache from /Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/composer/tmp-f523c084f1c57cef4b6570984c5c6ff5.zip - Installing foo/composer-test (1.0.0): Extracting archive Executing async command (CWD): unzip -qq -o '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/composer/tmp-f523c084f1c57cef4b6570984c5c6ff5.zip' -d '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/composer/db4aeeca' Executing command (CWD): rm -rf '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test' Executing command (CWD): rm -rf '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/composer/db4aeeca' Executing command (CWD): rm -rf '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/composer/' Generating autoload files

With --prefer-source option

→ composer update -vvv --prefer-source Reading ./composer.json (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.json) Loading config file /Users/steveabrams/.composer/config.json Loading config file /Users/steveabrams/.composer/auth.json Loading config file ./composer.json (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.json) Checked CA file /private/etc/ssl/cert.pem: valid

Dependency resolution completed in 0.000 seconds Analyzed 86 packages to resolve dependencies Analyzed 86 rules to resolve dependencies Lock file operations: 1 install, 0 updates, 0 removals Installs: foo/composer-test:1.0.0 - Locking foo/composer-test (1.0.0) Writing lock file Installing dependencies from lock file (including require-dev) Reading ./composer.lock (/Users/steveabrams/workspace/playground/composer/composer-local-install/composer.lock) Package operations: 1 install, 0 updates, 0 removals Installs: foo/composer-test:1.0.0 - Syncing foo/composer-test (1.0.0) into cache Cloning to cache at '/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/' Executing command (CWD): git clone --mirror 'http://gdk.test:3001/foo/cp-2.git' '/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/' Executing command (/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/): git rev-parse --git-dir Executing command (/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/): git rev-parse --quiet --verify '98298a129ca79d3c1c55a6651993ac01109e34ae^{commit}' - Installing foo/composer-test (1.0.0): Cloning 98298a129ca79d3c1c55a6651993ac01109e34ae from cache Executing command (CWD): git clone --no-checkout '/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/' '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test' --dissociate --reference '/Users/steveabrams/.composer/cache/vcs/http---gdk.test-3001-foo-cp-2.git/' && cd '/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test' && git remote set-url origin 'http://gdk.test:3001/foo/cp-2.git' && git remote add composer 'http://gdk.test:3001/foo/cp-2.git' Executing command (/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test): git branch -r Executing command (/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test): (git checkout '1.0.0' -- || git checkout -B '1.0.0' 'composer/1.0.0' --) && git reset --hard '98298a129ca79d3c1c55a6651993ac01109e34ae' -- Executing command (/Users/steveabrams/workspace/playground/composer/composer-local-install/vendor/foo/composer-test): git checkout '98298a129ca79d3c1c55a6651993ac01109e34ae' -- && git reset --hard '98298a129ca79d3c1c55a6651993ac01109e34ae' -- Generating autoload files

☑ Does this MR meet the acceptance criteria?

Conformity

📋 Does this MR need a changelog?
- I have included a changelog entry.
- [-] I have not included a changelog entry because _____.
Documentation (if required)
Code review guidelines
Merge request performance guidelines
Style guides
[-] Database guides
[-] Separation of EE specific content

Availability and Testing

Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
[-] Tested in all supported browsers
[-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

[-] Label as security and @ mention @gitlab-com/gl-security/appsec
[-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
[-] Security reports checked/validated by a reviewer from the AppSec team

Related to #247531 (closed)

Edited Mar 16, 2021 by Steve Abrams

Include source in composer json payload