GitLab Next

🌲 Context

The GitLab package registry allows users to publish their own packages to GitLab projects. Currently, if a package is published for the very first time using GitLab CI, a build_infos record is created that associating the package record with the pipeline that created it.

Users would like to be able to not only see the first pipeline, but anytime a pipeline may have republished a version. This is a common pattern with Maven SNAPSHOT packages. The same version may be published many times, but due to the current limitations, we only see the pipeline from the very first one that was published which may no longer be relevant.

🔍 What does this MR do?

This MR updates the relationships between packages and pipelines to change from a one-to-one to a one-to-many relationship. We do this by:

• Updating the relationships on Packages::Package and Packages::BuildInfo
• Removing the UNIQUE index on packages_build_infos that prevented us from having more than one record per package.
• Update the logic of when a build_info is created to not only occur when a package is first created, but also when it is updated.

The MR creates a new relationship between package files and pipelines so in addition to seeing when a package record was created/updated, we can see the pipelines that created the individual files belonging to that package. There are cases where the package files may be created in a different pipeline from when the package was created. We do this by:

• Create a new model: Packages::PackageFileBuildInfo that relates package files to pipelines.
• Add logic to create these records anytime a new package_file record is created.

A few notes:

• This MR only updates the backend and the API payload. We add a method to Packages::Package#build_info to mimic the previously existing association and allow the front end to continue to function, displaying only the first build_info
• We have pipeline and pipelines now returned in the API entity. Once the frontend is updated to use pipelines, we will remove pipeline and the #build_info method.
• This only updates Maven, NPM, and generic packages. The other package types do not yet track pipeline data, which will be added in a separate MR
• The frontend updates to display the new pipeline information will be implemented in a separate MR.

🐘 Database

UpdatePackagesBuildInfosIndex

Up:

== 20201103202213 UpdatePackagesBuildInfosIndex: migrating ====================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:packages_build_infos, :package_id, {:name=>"idx_packages_build_infos_on_package_id", :algorithm=>:concurrently})
   -> 0.0029s
-- execute("SET statement_timeout TO 0")
   -> 0.0003s
-- add_index(:packages_build_infos, :package_id, {:name=>"idx_packages_build_infos_on_package_id", :algorithm=>:concurrently})
   -> 0.0030s
-- execute("RESET ALL")
   -> 0.0003s
-- transaction_open?()
   -> 0.0000s
-- indexes(:packages_build_infos)
   -> 0.0028s
-- remove_index(:packages_build_infos, {:algorithm=>:concurrently, :name=>"index_packages_build_infos_on_package_id"})
   -> 0.0032s
== 20201103202213 UpdatePackagesBuildInfosIndex: migrated (0.0140s) ===========

Down:

== 20201103202213 UpdatePackagesBuildInfosIndex: reverting ====================
== 20201103202213 UpdatePackagesBuildInfosIndex: reverted (0.0000s) ===========

CreatePackagesPackageFileBuildInfos

Up:

== 20201103184333 CreatePackagesPackageFileBuildInfos: migrating ==============
-- table_exists?(:packages_package_file_build_infos)
   -> 0.0009s
-- create_table(:packages_package_file_build_infos)
   -> 0.0112s
== 20201103184333 CreatePackagesPackageFileBuildInfos: migrated (0.0189s) =====

Down:

== 20201103184333 CreatePackagesPackageFileBuildInfos: reverting ==============
-- drop_table(:packages_package_file_build_infos)
   -> 0.0032s
== 20201103184333 CreatePackagesPackageFileBuildInfos: reverted (0.0092s) =====

Screenshots

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• [-] Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• [-] Tested in all supported browsers
• [-] Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• [-] Label as security and @ mention @gitlab-com/gl-security/appsec
• [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• [-] Security reports checked/validated by a reviewer from the AppSec team

Related to #239518 (closed)