When downloading Composer dependencies, you have two options. You can use source or dist. For stable versions, Composer will use the dist by default. The source is a version control repository, like your GitLab Project. If --prefer-source is enabled, Composer will install from source as a git clone instead of downloading a packaged zip file. This is useful if you want to make a bugfix to a project and get a local git clone of the dependency directly.
The problem is that until recently, you could not use the prefer-source and preferred-install commands and configurations when downloading Composer dependencies. This prevented many of you from using the GitLab Package Registry for your Composer dependencies.
Proposal
Add support for prefer-source and preferred-install so that PHP Developers can configure Composer to download packages from git instead of as a zip file.
Steps to reproduce
Create a project in a group.
Enable the Packages feature if not already enabled.
Make sure the project has a valid composer.json:
{ "name": "m4rw3r/composer-test-project", "description": "Composer test project for prefer-source install", "license": "MIT"}
Publish the package using curl: curl --data branch=master 'https://__token__:<token>@gitlab.com/api/v4/projects/21057249/packages/composer.
Create a new folder and add the following to a composer.json:
Attempt to install the package using composer install -vvv --prefer-source.
The package will install successfully, but not from source. In the log produced
it will list that it is downloading the package as a zip-file instead of
checking out a clone, despite --prefer-source being specified.
Well, we didn't get to adding v2 support of Composer in 13.9. #259840 (closed). So, this issue will have to move to 13.11 and the former to 13.10. I apologize for the delay everyone, we are trying to get to this soon.
That's really unfortunate to hear. These features are part of composer regardless of v1 or v2. I do not plan on realistically shifting our many many project over to composer 2 for at least another year, placing a big blocker on the transition plan.
Tim Rizzichanged title from The built-in GitLab Composer Repository does not allow source install to {+Use prefer-source and preferred-install with the GitLab Composer Repository+}
changed title from The built-in GitLab Composer Repository does not allow source install to {+Use prefer-source and preferred-install with the GitLab Composer Repository+}
I used the following workaround for the modules where i need to have the source locally: I added them directly as git repo in the repositories section of the composer.json. This way it's loading the needed modules (in this case merchant_customizations and merchant_theme) directly via Git and I can have the source for development.
Yeah, basically I am not using composer for these 2 modules but it works and I don't need another time consuming or complicated solution :-)
Coming up to speed with Composer, but I believe I have identified the areas of code that need to be updated to support prefer-source. I've opened an MR based on the investigation below.
To do:
When the get ':id/-/packages/composer/*package_name' endpoint is requested, we end up generating a versions index using ::Gitlab::Composer::VersionIndex. This class generates the "dist", but not "source". to be included in the json response.
Looking at the "source" format, it seems we need to add:
I was able to manually test and verify the --prefer-source option works with the updates. I updated the docs, fixed the failing test, and have pushed this into review.
Awesome I am looking so much forward to this and finally get rid of our current packages server (which is slow) and move everything to the Gitlab Composer Repo <3
Any chance we can get this bugfix into GitLab earlier than 13.11? That's a month of waiting for a fix some people really need :/ Why is this tagged as "feature"?
I confirmed this is not possible, in order to be picked into a minor release, we would have to label the MR prior to it being merged. Sorry, @bernhardberger. If you are on Self-Managed it will be available on April 22nd. (It is already available on Gitlab.com)
Tim Rizzichanged the descriptionCompare with previous version
changed the description
Tim Rizzichanged title from {-Use prefer-source and preferred-install with the GitLab-} Composer Repository to Download Composer dependencies from version control
changed title from {-Use prefer-source and preferred-install with the GitLab-} Composer Repository to Download Composer dependencies from version control
@trizzi, I opened #329246 (closed) to address supporting ssh-urls. Looking at the documentation, I don't see a way to include ssh and http urls in the same source payload, so we will need to do some testing to ensure that public packages are accessible without ssh keys, and private packages are easily accessed when ssh keys are present.