You use the GitLab Composer repository to publish and download your Composer dependencies to your private project. You can authenticate with a personal access token or a job token.
Problem to solve
The problem is that personal access tokens and job tokens are tied to a specific user. When you want to finalize your production workflows, you'd like to use a deploy token, which is not tied to a specific user or their permissions.
Proposal
Update the permissions model for group deploy tokens to allow for publishing and downloading of Composer dependencies using the write_package_registry and read_package_registry scope respectively.
Hi @henning.cash, thank you for taking the time to open this issue! Unfortunately composer does not yet support deploy tokens and personal access tokens are the only way to authenticate at the moment. We will work to add support for deploy tokens as well as CI Job tokens soon.
Tim Rizzichanged title from {-Group deploy tokens with read_package_registry scope do not grant access to the Composer Package-} registry to Use Deploy tokens to grant access to the Composer registry
changed title from {-Group deploy tokens with read_package_registry scope do not grant access to the Composer Package-} registry to Use Deploy tokens to grant access to the Composer registry
UPDATE: Unfortunately we won't be able to get to this in milestone %13.6. We are accepting merge requests for this issue and it could be a good chance to contribute.
Thanks for the ping @JeroenVanOort, this issue and it's associated MRs detail the original work for adding Deploy token support for Packages. This would be a good place to start.
The tricky part with authentication in these APIs is it all is injected before we even reach these files in the top level api.rb file in the included APIGuard module.
If you dig down into that module, you'll find your way to a method named deploy_token_from_request. The first thing this method does is check for a specific Grape API decorator route_setting.
If you look back at the composer API code, you'll see most routes are prefaced with a route_setting line, like this. To enable deploy tokens for a given route, you just need to include deploy_token_allowed: true in the route_setting similar to what you might see in some of the other package APIs like npm.
I'm not entirely familiar with how composer passes credentials, but I'd start by seeing if that works, and then dig into the referenced auth code if it looks like it's not going to be as simple as that.
Feel free to ping me when you open an MR if you need any help!
Also, if you've never contributed to GitLab before, check out the Contribution docs for details on how to pull the code, run it using the GitLab Development Kit, and get a merge request started. I'm happy to answer any questions about that process as well.
Adding the ~"good for new contributors" as this could be a good option for the upcoming GitLab hackathon. Consider joining the upcoming Package Office Hours this Wednesday: #326138 (closed).
Hello,
I have pushed changes on my local fork to support the deploy tokens for composer package registry based on details written on #240897 (comment 440323332).
@Zalitis I was just rereading your comment. Can you open an MR with your committee changes? We can help with the tests during the review process. Just ping me in the MR.
@Zalatis, I agree with @trizzi, opening an MR is the best way to get started. The MR will automatically run the full set of tests. To run locally without GDK, I believe you need postgres running with a valid test DB which can be created with the usual rails commands: rake db:create, rake db:schema:load. Then you should be able to run the specific tests: bin/rspec spec/path/to/your/spec_file.rb. I actually have never run tests outside of GDK so I may be missing a step or two.
For anyone who's found this issue looking to be able to authenticate with composer using a job token, and you run a self-hosted instance, I made a small patch that enables it, similar to the one by @Zalitisabove. The only change is job_token_allowed: true to job_token_allowed: :basic_auth for all the composer routes. Patch is against 13.11.1.
To use,
Patch the file
sudo gitlab-ctl usr1 puma
In your ci job,
before_script:-composer config http-basic.<YOUR SITE URL> gitlab-ci-token "$CI_JOB_TOKEN"
[Composer\Downloader\TransportException] The "https://gitlab.com/api/v4/group/XXSubgroupIDXXXXX/-/packages/composer/packages.json" file could not be downloaded (HTTP/2 404 ): {"message":"404 Group Not Found"}
Exception trace: () at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:512 Composer\Util\Http\CurlDownloader->failResponse() at phar:///usr/local/bin/composer/src/Composer/Util/Http/CurlDownloader.php:354 Composer\Util\Http\CurlDownloader->tick() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:371 Composer\Util\HttpDownloader->countActiveJobs() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:342 Composer\Util\HttpDownloader->wait() at phar:///usr/local/bin/composer/src/Composer/Util/HttpDownloader.php:93 Composer\Util\HttpDownloader->get() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:1100 Composer\Repository\ComposerRepository->fetchFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:849 Composer\Repository\ComposerRepository->loadRootServerFile() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:527 Composer\Repository\ComposerRepository->hasProviders() at phar:///usr/local/bin/composer/src/Composer/Repository/ComposerRepository.php:349 Composer\Repository\ComposerRepository->loadPackages() at phar:///usr/local/bin/composer/src/Composer/Repository/RepositorySet.php:166 Composer\Repository\RepositorySet->findPackages() at phar:///usr/local/bin/composer/src/Composer/Package/Version/VersionSelector.php:70 Composer\Package\Version\VersionSelector->findBestCandidate() at phar:///usr/local/bin/composer/src/Composer/Command/InitCommand.php:834 Composer\Command\InitCommand->findBestVersionAndNameForPackage() at phar:///usr/local/bin/composer/src/Composer/Command/InitCommand.php:511 Composer\Command\InitCommand->determineRequirements() at phar:///usr/local/bin/composer/src/Composer/Command/RequireCommand.php:189 Composer\Command\RequireCommand->execute() at /var/www/vendor/symfony/flex/src/Command/RequireCommand.php:54 Symfony\Flex\Command\RequireCommand->execute() at phar:///usr/local/bin/composer/vendor/symfony/console/Command/Command.php:245 Symfony\Component\Console\Command\Command->run() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:835 Symfony\Component\Console\Application->doRunCommand() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:185 Symfony\Component\Console\Application->doRun() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:312 Composer\Console\Application->doRun() at phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:117 Symfony\Component\Console\Application->run() at phar:///usr/local/bin/composer/src/Composer/Console/Application.php:125 Composer\Console\Application->run() at phar:///usr/local/bin/composer/bin/composer:64 require() at /usr/local/bin/composer:24
You may be right, as I've used the workaround, not as a composer package registry, but as a composer package (used a subgroup, created a deploy token in the subgroup, added all my composer packages as subprojects of that subgroup and tagged the version)
I pulled the branch from @Zalatis's fork and opened an MR. I went through some of the updates that were discussed regarding the DeployToken model and started working through the tests.
The remaining work consists of working through the test changes. I expect this to be in review early next week.
I have verified deploy token access to Composer on GitLab.com. This feature will be included in the %14.6 release. Please feel free to ping me if you have any questions. Closing the issue.