Skip to content

Add new packages build infos finder

David Fernandez requested to merge 347220-new-build-infos-finder into master

🦑 Context

Similar to !84387 (merged). This is a changes extraction from !82496 (merged) where we want to fix several caveats around pipelines object through packages:

  • too many objects loaded
  • n+1 situations.

That's issue #347220 (closed).

To support this fix, we need a highly specialized finder that this MR introduces. This finder will be used by a GraphQL resolver that will need to load build infos objects given a set of packages objects.

The changes presented here have been reviewed as part of hhttps://gitlab.com/gitlab-org/gitlab/-/merge_requests/82496. As such, we will directly go for the maintainer review phase.

🤔 What does this MR do and why?

  • Add Packages::BuildInfosForManyPackagesFinder.
    • This will load Packages::BuildInfos objects given a set of package_ids. In a sense it loads "partitions" of build infos objects for a set of packages. In other words, for each package id, it loads a set of build infos.
    • Supports pagination parameters (first, last, after, before, support_next_page). Those are applied on the sets of build infos. Example, using first: 1 means that for each package we want the most recent build info object.
    • Given the highly custom SQL query that is executed, this finder will return an array of Active Records.
  • To support the underlying SQL query, replaces an existing index on the packages_build_infos table.

Again, no code will use this finder. As such, these changes will have no impact on any user facing feature.

A changelog has been included because this MR includes a migration.

📺 Screenshots or screen recordings

n / a

🔭 How to set up and validate locally

  1. Create a Group

  2. Create several Projects

  3. For each project, open a rails console and execute:

    def fixture_file_upload(*args, **kwargs)
      Rack::Test::UploadedFile.new(*args, **kwargs)
    end
    
    pkg = FactoryBot.create(:npm_package, project: Project.find(<project_id>))
    
    # creates 5 pipelines
    5.times { Packages::BuildInfo.create!(package_id: pkg.id, pipeline_id: FactoryBot.create(:ci_pipeline).id) }
  4. Use the new finder:

    Packages::BuildInfosForManyPackagesFinder.new(<package_ids>)
    Packages::BuildInfosForManyPackagesFinder.new(<package_ids>, first: 1)

🚥 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

💽 Database review

Migration up

== 20220413075921 UpdateIndexOnPackagesBuildInfos: migrating ==================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:packages_build_infos, [:package_id, :pipeline_id, :id], {:name=>"index_packages_build_infos_package_id_pipeline_id_id", :algorithm=>:concurrently})
   -> 0.0059s
-- execute("SET statement_timeout TO 0")
   -> 0.0008s
-- add_index(:packages_build_infos, [:package_id, :pipeline_id, :id], {:name=>"index_packages_build_infos_package_id_pipeline_id_id", :algorithm=>:concurrently})
   -> 0.0065s
-- execute("RESET statement_timeout")
   -> 0.0010s
-- transaction_open?()
   -> 0.0000s
-- indexes(:packages_build_infos)
   -> 0.0029s
-- remove_index(:packages_build_infos, {:algorithm=>:concurrently, :name=>"index_packages_build_infos_package_id_pipeline_id"})
   -> 0.0036s
== 20220413075921 UpdateIndexOnPackagesBuildInfos: migrated (0.0351s) =========

Migration down

== 20220413075921 UpdateIndexOnPackagesBuildInfos: reverting ==================
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:packages_build_infos, [:package_id, :pipeline_id], {:name=>"index_packages_build_infos_package_id_pipeline_id", :algorithm=>:concurrently})
   -> 0.0048s
-- execute("SET statement_timeout TO 0")
   -> 0.0009s
-- add_index(:packages_build_infos, [:package_id, :pipeline_id], {:name=>"index_packages_build_infos_package_id_pipeline_id", :algorithm=>:concurrently})
   -> 0.0078s
-- execute("RESET statement_timeout")
   -> 0.0008s
-- transaction_open?()
   -> 0.0000s
-- indexes(:packages_build_infos)
   -> 0.0028s
-- remove_index(:packages_build_infos, {:algorithm=>:concurrently, :name=>"index_packages_build_infos_package_id_pipeline_id_id"})
   -> 0.0050s
== 20220413075921 UpdateIndexOnPackagesBuildInfos: reverted (0.0391s) =========

📊 Queries

Edited by David Fernandez

Merge request reports