Skip to content

Adding BuildNameFinder to enable searching builds by name

Max Fan requested to merge 446304-search-build-by-name into master

What does this MR do and why?

Enable searching build(s) by name

Introducing a new Finder just for build names due to the increased complexity of database queries.

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/28271/commands/88237 Is the query that this finder does.

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/28271/commands/88236 - Is the query of the graphQL endpoint (because it paginates again) - same db cost

Note: This should also work for REST API queries that support keyset, for example project jobs: (not included in this MR to keep it smaller)

          builds = ::Ci::BuildNameFinder.new(
            relation: builds,
            pagination_type: :keyset,
            name: params[:name],
            project: user_project,
            params: {
              after: Gitlab::Json.parse(Base64.decode64(params[:cursor])).fetch("id"),
              first: params[:per_page]
            }
          ).execute

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Query Result
image.png image.png

How to set up and validate locally

  1. Enable the FF populate_and_use_build_names_table and run some pipelines
  2. GraphQL queries asking for project_jobs by name should work now.
{
  project(fullPath: "root/pretendexternal"){
    jobs(name: "deploy", before: "eyJpZCI6IjEzMTIwIn0", first: 2) {
      count
      pageInfo {
        endCursor
        hasNextPage
        hasPreviousPage
        startCursor
      }
      nodes {
        id
        name
      }
    }
  }
}
  1. Play around with the name, before, after, first, last parameters and it should return a logical ordering (remember everything's reversed by default as this endpoint currently sorts id: :desc)
  2. Note: count is off -> known issue #459005

How the PSQL Queries Were populated

  1. Populate with 2 mill records referenced by a specific ID that we can use to try pagination
exec INSERT INTO p_ci_build_names (partition_id, build_id, project_id, name) SELECT build.partition_id, build.id, build.project_id, coalesce(build.name, 'p') as name FROM p_ci_builds as build WHERE build.project_id = 278964 and build.id < 6729873788 and ("build"."status" NOT IN ('created')) ORDER BY id desc LIMIT 1000000; 

exec INSERT INTO p_ci_build_names (partition_id, build_id, project_id, name) SELECT build.partition_id, build.id, build.project_id, coalesce(build.name, 'p') as name FROM p_ci_builds as build WHERE build.project_id != 278964 ORDER BY id desc LIMIT 1000000;

Related to #446304

Edited by Tianwen Chen

Merge request reports