### Problem to solve [Composer](https://getcomposer.org/), is a dependency manager for PHP that allows you to declare the libraries your project depends on and it will manage (install/update) them for you. PHP developers need a mechanism to share and consume packages and view metadata in one central location. ### Intended users * [Software Developer](https://design.gitlab.com/research/personas#persona-sasha) * [DevOps Engineer](https://design.gitlab.com/research/personas#persona-devon) ### Proposal Provide support for users coding in PHP by integrating with Composer and allowing developers to share and consume packages alongside their source code and pipelines. #### Proposed MVC Scope * Publish dependencies to your GitLab project * Install dependencies using your group-level endpoint. * Authenticate using your GitLab Personal Access Token. * View your dependencies and their respective metadata using the Package Registry UI #### Package Naming and Scope The package name must be unique across the instance to allow easy integration. For the MVP the package name will be claimed by the first use of a given name. For example, when a package named `my-package` is first pushed to project `project-1`, then that name is owned by `project-1`. From then on only users with permission to project P can access the package. #### Packages are not uploaded to the Composer Repository, they are linked. It's important to clarify that Composer doesn't handle uploading packages to the registry as it's only the meta-layer for mapping package names to actual resources (which can be archives or Git references). As Gitlab already hosts ZIP files for branches and tags the only thing that needs to be done here is generating JSON with package information and pointing packages to existing resources. ### Further details #### Implementation Composer relies heavily on VCS for versioning by using a naming convention for branches and tags ([See composer docs](https://getcomposer.org/doc/articles/versions.md#composer-versions-vs-vcs-versions)). A simple way to publish a package would be to simply expose the package archive straight from GitLab by using the [Blob Api](https://docs.gitlab.com/ee/api/repositories.html#raw-blob-content). A package could then be published after it's on git by posting to the API: ```bash curl -X POST -F -F 'name=package_name' -F 'branch=my-feature' http://gitlab.com/<project_id>/packages/composer ``` This would expose the specific branch as a composer package. Or to expose a tag: ```bash curl -X POST -F 'name=package_name' -F 'tag=v1.0' http://gitlab.com/<project_id>/packages/composer ``` This API call creates the package inside GitLab and then parses the version accordingly based on composer rules. #### Composer Package API Composer package install starts by fetching the Packagist index from `https://repo.packagist.org/packages.json`. This top-level JSON works as a pagination mechanism by pointing to other indexed JSON files. ##### Sample index ```json { "metadata-url": "/p2/%package%.json", "notify": "https://packagist.org/downloads/%package%", "notify-batch": "https://packagist.org/downloads/", "packages": [], "provider-includes": { "p/provider-2013$%hash%.json": { "sha256": "c9f9d0aa9b4bc58886635d7768e599bf283c0b6d97ac7e877a9d0858a78903d7" }, ... "p/provider-2020-01$%hash%.json": { "sha256": "6c3bb1746ab8ddda1503240e416b73d15ee86c50df0d1b39a390b32df78b4c6c" }, "p/provider-archived$%hash%.json": { "sha256": "95b7476cfaab2a59b34e1fbc07c1b5eaea0ffab22762f7f3baab102b0cc62822" }, "p/provider-latest$%hash%.json": { "sha256": "490b8c4812dd4535ff03c1b378c4f6a5caa855bfa961152cee65104575de60d4" } }, "providers-api": "https://packagist.org/providers/%package%.json", "providers-url": "/p/%package%$%hash%.json", "search": "https://packagist.org/search.json?q=%query%&type=%type%" } ``` ##### Index files The second level index contains a list of URLs for the packages: `GET http://repo.packagist.org/p/provider-2020-01%246c3bb1746ab8ddda1503240e416b73d15ee86c50df0d1b39a390b32df78b4c6c.json` ```json { "providers": { "0.0.0/laravel-env-shim": { "sha256": "3b765076f87ac782e172d802b08d84c6d6c22e7845263c7bbcdce475f3bdf84c" }, ... "symphonycms/symphony-2": { "sha256": "08bd8ae21aad64fb0f7965a086f0d386523eda3b0a24911b42ee485028f95adf" }, } } ``` ##### Package versions file The 3rd level index file has information about the versions of a specific package: `GET http://repo.packagist.org/p/symphonycms/symphony-2%2408bd8ae21aad64fb0f7965a086f0d386523eda3b0a24911b42ee485028f95adf.json` ```json { "packages": { "symphonycms/symphony-2": { "2.6.0": { "name": "symphonycms/symphony-2", "description": "Symphony is a PHP & MySQL based CMS that utilises XML and XSLT as its core technologies.", "keywords": [], "homepage": "http://www.getsymphony.com", "version": "2.6.0", "version_normalized": "2.6.0.0", "license": [ "MIT" ], "authors": [ { "name": "Symphony Team", "email": "team@getsymphony.com" } ], "source": { "type": "git", "url": "https://github.com/symphonycms/symphony-2.git", "reference": "5a4883563627d6e9f5a941a7e01328d2e84ab00f" }, "dist": { "type": "zip", "url": "https://api.github.com/repos/symphonycms/symphony-2/zipball/5a4883563627d6e9f5a941a7e01328d2e84ab00f", "reference": "5a4883563627d6e9f5a941a7e01328d2e84ab00f", "shasum": "" }, "type": "library", "time": "2015-03-11T09:23:21+00:00", "autoload": { "classmap": [ "symphony/content", "symphony/lib", "symphony/template", "install" ], "files": [ "symphony/lib/boot/func.utilities.php", "symphony/lib/boot/defines.php", "symphony/lib/toolkit/util.validators.php" ] }, "uid": 354000 }, ... } } } ``` From there the specific package zip file for a given version can be retrieved (in this case straight from github): `GET https://api.github.com/repos/symphonycms/symphony-2/zipball/68f44f0c36ad3345068676bfb8a61c2e6a2e51f4` ##### Package Index Pagination The `packages.json` top-level entry point should return a paginated list of packages. For the MVC I believe we could paginate by the package name initial letters like so: `gitlab.com/api/v4/packages/composer/packages/aa.json` `gitlab.com/api/v4/packages/composer/packages/ab.json` ... `gitlab.com/api/v4/packages/composer/packages/zz.json` ##### Entry Points `GET gitlab.com/api/v4/packages/composer/packages.json`: Instance level, paginated top level index for the list of available packages. `GET gitlab.com/api/v4/packages/composer/packages/:page.json`: The paginated list of package names. Returns the list of packages contained in a specific `:page`. `GET gitlab.com/api/v4/projects/:id/packages/composer/*package_name.json`: Provides the package metadata json with the data about the package versions. `GET gitlab.com/api/v4/projects/:id/packages/composer/*package_name/-/*file_name`: Endpoint to download a specific package file. `POST gitlab.com/api/v4/projects/:id/packages/composer`: Endpoint to upload package files. ### Permissions and Security - Access to packages should follow standard project read/write permissions. ### Documentation - Add the Composer Repository to https://docs.gitlab.com/ee/user/packages/ - Screenshots should also be updated to reflect that Composer is now an option ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? Success looks like we enable PHP developers to use their GitLab project as a Composer Repository and drive usage of the Package Registry and CI/CD by giving them a new GitLab workflow they can use. #### Metrics - Our North Star Metric of number of packages published/installed will suffice to measure adoption. - We can also monitor page views and the number of times someone uses the quick setup/install commands in the UI.