Support publishing Composer package to one central project

What does this MR do and why?

Implements #250633.

This MR adds source_project parameter to Composer package creation endpoint.

It achieves that by:

• Adding source_project_id column to packages_composer_metadata table.
• Migrating all existing composer packages to fill in source_project_id from packages_packages.project_id - before this change the source was always in the same project as composer repository. Note that since this migration runs in post_migrate, I had to allow null value in source_project_id for now. In the future version we could rollback commit Allow for composer source_project_id column to be null.
• Using the new column source_project_id when generating links in composer responses (both for source and distribution) to make source composer uses the source project when fetching data.

Screenshots or screen recordings

This is an API only change. Here's example of composer repository with packages from multiple projects added:

Database migration log

Migration log

$ rake db:migrate

== 20211212174031 AddComposerPackageSourceProjectId: migrating ================
-- add_column(:packages_composer_metadata, :source_project_id, :integer)
   -> 0.0006s
== 20211212174031 AddComposerPackageSourceProjectId: migrated (0.0006s) =======

== 20211212174638 AddIndexForComposerPackageSourceProjectId: migrating ========
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:packages_composer_metadata, :source_project_id, {:name=>"index_packages_composer_metadata_on_source_project_id", :algorithm=>:concurrently})
   -> 0.0015s
-- execute("SET statement_timeout TO 0")
   -> 0.0003s
-- add_index(:packages_composer_metadata, :source_project_id, {:name=>"index_packages_composer_metadata_on_source_project_id", :algorithm=>:concurrently})
   -> 0.0074s
-- execute("RESET statement_timeout")
   -> 0.0004s
== 20211212174638 AddIndexForComposerPackageSourceProjectId: migrated (0.0107s) 

== 20211213062010 AddFkForComposerPackageSourceProjectId: migrating ===========
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:packages_composer_metadata)
   -> 0.0016s
-- transaction_open?()
   -> 0.0000s
-- execute("ALTER TABLE packages_composer_metadata\nADD CONSTRAINT fk_b959c6a2d5\nFOREIGN KEY (source_project_id)\nREFERENCES projects (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0011s
-- execute("ALTER TABLE packages_composer_metadata VALIDATE CONSTRAINT fk_b959c6a2d5;")
   -> 0.0015s
== 20211213062010 AddFkForComposerPackageSourceProjectId: migrated (0.0063s) ==

== 20211213062612 BackfillComposerPackageSourceProjectId: migrating ===========
-- transaction_open?()
   -> 0.0000s
-- execute("UPDATE packages_composer_metadata\nSET source_project_id = project_id\nFROM packages_packages\nWHERE source_project_id IS NULL AND package_id = packages_packages.id AND package_id BETWEEN 1 AND 37\n")
   -> 0.0019s
== 20211213062612 BackfillComposerPackageSourceProjectId: migrated (0.0191s) ==

$ rake db:migrate:down VERSION=20211213062612
== 20211213062612 BackfillComposerPackageSourceProjectId: reverting ===========
== 20211213062612 BackfillComposerPackageSourceProjectId: reverted (0.0000s) ==

$ rake db:migrate:down VERSION=20211213062010
== 20211213062010 AddFkForComposerPackageSourceProjectId: reverting ===========
-- transaction_open?()
   -> 0.0000s
-- remove_foreign_key(:packages_composer_metadata, {:column=>:source_project_id})
   -> 0.0028s
== 20211213062010 AddFkForComposerPackageSourceProjectId: reverted (0.0060s) ==

$ rake db:migrate:down VERSION=20211212174638
== 20211212174638 AddIndexForComposerPackageSourceProjectId: reverting ========
-- transaction_open?()
   -> 0.0000s
-- indexes(:packages_composer_metadata)
   -> 0.0015s
-- execute("SET statement_timeout TO 0")
   -> 0.0003s
-- remove_index(:packages_composer_metadata, {:algorithm=>:concurrently, :name=>"index_packages_composer_metadata_on_source_project_id"})
   -> 0.0036s
-- execute("RESET statement_timeout")
   -> 0.0003s
== 20211212174638 AddIndexForComposerPackageSourceProjectId: reverted (0.0068s) 

$ rake db:migrate:down VERSION=20211212174031
== 20211212174031 AddComposerPackageSourceProjectId: reverting ================
-- remove_column(:packages_composer_metadata, :source_project_id, :integer)
   -> 0.0006s
== 20211212174031 AddComposerPackageSourceProjectId: reverted (0.0013s) =======

How to set up and validate locally

The easiest option is to run bundle exec rake db:seed_fu FILTER=composer SEED_COMPOSER=1. This will create project composer/composer-repository that will have composer packages with sources from another repo. After running this seed you can go to <gdk-host>/composer/composer-repository/-/packages.

To test it manually and check the API endpoint you can:

1. Create a project A with a valid composer.json.
2. Create a separate project B (in a group) that has package registry enabled.
3. Create a composer package in a registry in project B, but with source from project A by using a new parameter source_project in Composer package creation endpoint.
4. Verify the dist and source links in composer package metadata endpoint.

Next, follow https://docs.gitlab.com/ee/user/packages/composer_repository/#install-a-composer-package to test with composer that package is installable.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.