Skip to content
Snippets Groups Projects

Add and backfill project_id for p_ci_pipeline_variables

Merged Add and backfill project_id for p_ci_pipeline_variables
morefice/add-sharding-key-p-ci-pipeline-variables into master
2 unresolved threads
Merged Max Orefice requested to merge morefice/add-sharding-key-p-ci-pipeline-variables into master
2 unresolved threads

Ref: #463244 (closed)

What does this MR do and why?

This MR adds and backfills project_id for p_ci_pipeline_variables as this is a requirement for Cells1.0.

We have about ~2B records to update which should take ~6 weeks to complete on gitlab.com.

Edited by Max Orefice

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Max Orefice changed milestone to %17.4

    changed milestone to %17.4

  • Max Orefice added Category:Continuous Integration Deliverable Engineering Time VerifyP1 cicdactive devopsverify grouppipeline security maintenancescalability sectionci typemaintenance workflowready for development labels

    added Category:Continuous Integration Deliverable Engineering Time VerifyP1 cicdactive devopsverify grouppipeline security maintenancescalability sectionci typemaintenance workflowready for development labels

  • Max Orefice assigned to @morefice

    assigned to @morefice

  • Max Orefice changed the description

    changed the description

  • 🤖 GitLab Bot 🤖 added pipelinetier-1 label

    added pipelinetier-1 label

    • Ghost User
      Ghost User @ghost1 ·
      Contributor
      Resolved by Marius Bobin

      Database migrations (on the main database)

      Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

      Migration Type Total runtime Result DB size change
      20240827114849 - AddProjectIdToPCiPipelineVariables Regular 4.4 s :white_check_mark: +0.00 B
      20240827114958 - IndexPCiPipelineVariablesOnProjectId Post deploy 6.2 s :white_check_mark: +0.00 B
      20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint Post deploy 5.9 s :white_check_mark: +0.00 B
      20240828074159 - QueueBackfillPCiPipelineVariablesProjectId Post deploy 4.3 s :white_check_mark: +0.00 B
      Runtime Histogram for all migrations
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 19
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827114849 - AddProjectIdToPCiPipelineVariables

      • Type: Regular
      • Duration: 4.4 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      1 5.0 ms 5.0 ms 5.0 ms 0 
      ALTER TABLE "p_ci_pipeline_variables" ADD "project_id" bigint
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for AddProjectIdToPCiPipelineVariables
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 3
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827114958 - IndexPCiPipelineVariablesOnProjectId

      • Type: Post deploy
      • Duration: 6.2 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      2 5.2 ms 5.1 ms 2.6 ms 2 
      INSERT INTO "postgres_async_indexes" ("created_at", "updated_at", "name", "definition", "table_name") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
      1 3.9 ms 3.9 ms 3.9 ms 2 
      SELECT "postgres_partitions".*
        FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
        ORDER BY "postgres_partitions"."name" ASC
      1 0.3 ms 0.3 ms 0.3 ms 1 
      SELECT "postgres_partitioned_tables".*
        FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
        LIMIT $3
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for IndexPCiPipelineVariablesOnProjectId
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 6
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint

      • Type: Post deploy
      • Duration: 5.9 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      1 4.5 ms 4.5 ms 4.5 ms 2 
      SELECT "postgres_partitions".*
        FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
        ORDER BY "postgres_partitions"."name" ASC
      1 2.9 ms 2.9 ms 2.9 ms 0 
      ALTER TABLE gitlab_partitions_dynamic.ci_pipeline_variables_102 ADD CONSTRAINT check_5969fa62a9 CHECK ( project_id IS NOT NULL ) NOT VALID
      1 2.4 ms 2.4 ms 2.4 ms 0 
      ALTER TABLE public.ci_pipeline_variables ADD CONSTRAINT check_aa63343bdf CHECK ( project_id IS NOT NULL ) NOT VALID
      2 2.2 ms 1.4 ms 1.1 ms 2 
      SELECT c.is_nullable
        FROM information_schema.columns c
        WHERE c.table_schema = $1 AND c.table_name = $2 AND c.column_name = $3
      1 0.3 ms 0.3 ms 0.3 ms 1 
      SELECT "postgres_partitioned_tables".*
        FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
        LIMIT $3
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for AddPCiPipelineVariablesProjectIdNullConstraint
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 8
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240828074159 - QueueBackfillPCiPipelineVariablesProjectId

      • Type: Post deploy
      • Duration: 4.3 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for QueueBackfillPCiPipelineVariablesProjectId
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 2
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Other information

      Other migrations pending on GitLab.com
      Migration Type Total runtime Result DB size change
      20240820133200 - AddPagesSettingToApplicationSettings Regular 5.4 s :white_check_mark: +0.00 B
      20240822080620 - AddExpiresAtToPagesDeployments Regular 4.5 s :white_check_mark: +0.00 B
      20240823114755 - AddIndexOnPagesDeploymentExpiresAt Regular 8.5 s :white_check_mark: +8.00 KiB [note]
      20240825000001 - CreateWorkspacesAgentConfigs Regular 5.6 s :white_check_mark: +48.00 KiB
      20240825000002 - AddWorkspacesAgentConfigsProjectsFk Regular 5.5 s :warning: +0.00 B
      20240825000003 - AddWorkspacesAgentConfigsClusterAgentsFk Regular 5.2 s :white_check_mark: +0.00 B
      20240825000004 - AddWorkspacesAgentConfigsConstraints Regular 6.3 s :white_check_mark: +0.00 B
      20240825000005 - MigrateRemoteDevelopmentAgentConfigsToWorkspacesAgentConfigs Regular 4.6 s :warning: +64.00 KiB
      20240815132712 - RemoveProjectsVulnerabilityMergeRequestLinksProjectIdFk Post deploy 5.1 s :white_check_mark: +0.00 B
      20240815132940 - RemoveMergeRequestsVulnerabilityMergeRequestLinksMergeRequestIdFk Post deploy 5.1 s :white_check_mark: +0.00 B
      Clone details
      Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
      database-testing-3650270-15172962-main 2024-08-29T09:59:31Z 2024-08-29T08:00:59Z 2024-08-29 22:43:09 +0000
      database-testing-3650270-15172962-ci 2024-08-29T09:59:31Z 2024-08-29T08:44:58Z 2024-08-29 22:43:09 +0000

      Job artifacts

      Database migrations (on the ci database)

      Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

      Migration Type Total runtime Result DB size change
      20240827114849 - AddProjectIdToPCiPipelineVariables Regular 6.5 s :white_check_mark: +0.00 B
      20240827114958 - IndexPCiPipelineVariablesOnProjectId Post deploy 8.3 s :white_check_mark: +0.00 B
      20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint Post deploy 7.9 s :white_check_mark: +8.00 KiB [note]
      20240828074159 - QueueBackfillPCiPipelineVariablesProjectId Post deploy 7.6 s :white_check_mark: +0.00 B
      Runtime Histogram for all migrations
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 23
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827114849 - AddProjectIdToPCiPipelineVariables

      • Type: Regular
      • Duration: 6.5 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      1 1.2 ms 1.2 ms 1.2 ms 0 
      ALTER TABLE "p_ci_pipeline_variables" ADD "project_id" bigint
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for AddProjectIdToPCiPipelineVariables
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 3
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827114958 - IndexPCiPipelineVariablesOnProjectId

      • Type: Post deploy
      • Duration: 8.3 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      1 5.9 ms 5.9 ms 5.9 ms 1 
      SELECT "postgres_partitioned_tables".*
        FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
        LIMIT $3
      1 3.8 ms 3.8 ms 3.8 ms 2 
      SELECT "postgres_partitions".*
        FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
        ORDER BY "postgres_partitions"."name" ASC
      2 1.5 ms 1.4 ms 0.7 ms 2 
      INSERT INTO "postgres_async_indexes" ("created_at", "updated_at", "name", "definition", "table_name") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for IndexPCiPipelineVariablesOnProjectId
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 6
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint

      • Type: Post deploy
      • Duration: 7.9 s
      • Database size change: +8.00 KiB [note]
      Calls Total Time Max Time Mean Time Rows Query
      1 6.5 ms 6.5 ms 6.5 ms 1 
      SELECT "postgres_partitioned_tables".*
        FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
        LIMIT $3
      1 3.7 ms 3.7 ms 3.7 ms 2 
      SELECT "postgres_partitions".*
        FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
        ORDER BY "postgres_partitions"."name" ASC
      1 0.7 ms 0.7 ms 0.7 ms 0 
      ALTER TABLE public.ci_pipeline_variables ADD CONSTRAINT check_aa63343bdf CHECK ( project_id IS NOT NULL ) NOT VALID
      1 0.6 ms 0.6 ms 0.6 ms 0 
      ALTER TABLE gitlab_partitions_dynamic.ci_pipeline_variables_102 ADD CONSTRAINT check_5969fa62a9 CHECK ( project_id IS NOT NULL ) NOT VALID
      2 0.3 ms 0.2 ms 0.2 ms 2 
      SELECT c.is_nullable
        FROM information_schema.columns c
        WHERE c.table_schema = $1 AND c.table_name = $2 AND c.column_name = $3
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for AddPCiPipelineVariablesProjectIdNullConstraint
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 8
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Migration: 20240828074159 - QueueBackfillPCiPipelineVariablesProjectId

      • Type: Post deploy
      • Duration: 7.6 s
      • Database size change: +0.00 B
      Calls Total Time Max Time Mean Time Rows Query
      1 4.0 ms 4.0 ms 4.0 ms 1 
      SELECT MAX("id")
        FROM "p_ci_pipeline_variables"
      1 3.4 ms 3.4 ms 3.4 ms 1 
      INSERT INTO "batched_background_migrations" ("created_at", "updated_at", "max_value", "batch_size", "sub_batch_size", "interval", "status", "job_class_name", "table_name", "column_name", "job_arguments", "total_tuple_count", "started_at", "gitlab_schema", "queued_migration_version") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) RETURNING "id"
      1 1.8 ms 1.8 ms 1.8 ms 0 
      SELECT $1 AS one
        FROM "batched_background_migrations"  WHERE "batched_background_migrations"."job_arguments" = $2 AND "batched_background_migrations"."job_class_name" = $3 AND "batched_background_migrations"."table_name" = $4 AND "batched_background_migrations"."column_name" = $5
        LIMIT $6
      1 0.1 ms 0.1 ms 0.1 ms 0 
      SELECT $1 AS one
        FROM "batched_background_migrations"  WHERE "batched_background_migrations"."job_class_name" = $2 AND "batched_background_migrations"."table_name" = $3 AND "batched_background_migrations"."column_name" = $4 AND (job_arguments = $5) AND "batched_background_migrations"."gitlab_schema" = $6
        LIMIT $7
      2 0.0 ms 0.0 ms 0.0 ms 2 
      SELECT pg_backend_pid()
      Histogram for QueueBackfillPCiPipelineVariablesProjectId
      Query Runtime Count
      0 seconds - 0.01 seconds 0
      0.01 seconds - 0.1 seconds 0
      0.1 seconds - 1 second 6
      1 second - 5 seconds 0
      5 seconds - 15 seconds 0
      15 seconds - 5 minutes 0
      5 minutes + 0

      Background Migration: BackfillPCiPipelineVariablesProjectId

      Sampled 17 batches. Estimated Time to complete: 1 month, 6 days, 9 hours, 26 minutes, and 54 seconds
      • Interval: 120s
      • Total tuple count: 1988877800
      • Max batch size: 0
      • Estimated seconds to complete: 3182160s
      • Estimated number of batches: 26518
      • Average batch time: 108.24s
      • Batch size: 75000
      • N. of batches sampled: 17
      • N. of failed batches: 0

      Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.

      Consider changing max_batch_size and interval if this estimate is unacceptable.

      Calls Total Time Max Time Mean Time Rows Query
      5083 186073.9 ms 394.0 ms 36.6 ms 1270750f 
      UPDATE p_ci_pipeline_variables
        SET project_id = p_ci_pipelines.project_id
        FROM p_ci_pipelines
        WHERE p_ci_pipelines.id = p_ci_pipeline_variables.pipeline_id AND p_ci_pipelines.partition_id = p_ci_pipeline_variables.partition_id AND p_ci_pipeline_variables.id IN (
          SELECT p_ci_pipeline_variables.id
          FROM p_ci_pipeline_variables
          WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3 AND p_ci_pipeline_variables.id < $4
        )
      17 603.4 ms 84.8 ms 35.5 ms 4231 
      UPDATE p_ci_pipeline_variables
        SET project_id = p_ci_pipelines.project_id
        FROM p_ci_pipelines
        WHERE p_ci_pipelines.id = p_ci_pipeline_variables.pipeline_id AND p_ci_pipelines.partition_id = p_ci_pipeline_variables.partition_id AND p_ci_pipeline_variables.id IN (
          SELECT p_ci_pipeline_variables.id
          FROM p_ci_pipeline_variables
          WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3
        )
      17 30.3 ms 13.5 ms 1.8 ms 17 
      UPDATE batched_background_migration_jobs
        SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
        WHERE batched_background_migration_jobs.id = $5
      34 13.7 ms 8.4 ms 0.4 ms 34 
      INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
      17 13.0 ms 4.0 ms 0.8 ms 17 
      UPDATE batched_background_migration_jobs
        SET updated_at = $1, started_at = $2, status = $3, attempts = $4
        WHERE batched_background_migration_jobs.id = $5
      5100 1144.5 ms 2.1 ms 0.2 ms 5083 
      SELECT p_ci_pipeline_variables.id
        FROM p_ci_pipeline_variables
        WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3
        ORDER BY p_ci_pipeline_variables.id ASC
        LIMIT $4
        OFFSET $5
      34 2.7 ms 0.2 ms 0.1 ms 34 
      SELECT batched_background_migration_jobs.*
        FROM batched_background_migration_jobs
        WHERE batched_background_migration_jobs.id = $1
        LIMIT $2
      17 1.0 ms 0.1 ms 0.1 ms 17 
      SELECT sum(batched_background_migration_jobs.batch_size)
        FROM batched_background_migration_jobs
        WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)
      17 0.7 ms 0.0 ms 0.0 ms 17 
      SELECT batched_background_migrations.*
        FROM batched_background_migrations
        WHERE batched_background_migrations.id = $1
        LIMIT $2
      17 0.6 ms 0.0 ms 0.0 ms 17 
      SELECT p_ci_pipeline_variables.id
        FROM p_ci_pipeline_variables
        WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL
        ORDER BY p_ci_pipeline_variables.id ASC
        LIMIT $3
      Histogram of batch runtimes for BackfillPCiPipelineVariablesProjectId
      Batch Runtime Count
      0 seconds - 10 seconds 0
      10 seconds - 1 minute 0
      1 minute - 2 minutes 16
      2 minutes - 3 minutes 1
      3 minutes - 5 minutes 0
      5 minutes + 0
      Histogram across all sampled batches of BackfillPCiPipelineVariablesProjectId
      Query Runtime Count
      0 seconds - 0.1 seconds 0
      0.1 seconds - 0.5 seconds 10351
      0.5 seconds - 1 second 2
      1 second - 2 seconds 0
      2 seconds - 5 seconds 0
      5 seconds + 0

      Other information

      Other migrations pending on GitLab.com
      Migration Type Total runtime Result DB size change
      20240820133200 - AddPagesSettingToApplicationSettings Regular 7.3 s :white_check_mark: +0.00 B
      20240822080620 - AddExpiresAtToPagesDeployments Regular 6.5 s :white_check_mark: +0.00 B
      20240823114755 - AddIndexOnPagesDeploymentExpiresAt Regular 8.0 s :white_check_mark: +8.00 KiB [note]
      20240825000001 - CreateWorkspacesAgentConfigs Regular 8.0 s :white_check_mark: +56.00 KiB
      20240825000002 - AddWorkspacesAgentConfigsProjectsFk Regular 7.5 s :white_check_mark: +0.00 B
      20240825000003 - AddWorkspacesAgentConfigsClusterAgentsFk Regular 7.4 s :white_check_mark: +0.00 B
      20240825000004 - AddWorkspacesAgentConfigsConstraints Regular 8.5 s :white_check_mark: +0.00 B
      20240825000005 - MigrateRemoteDevelopmentAgentConfigsToWorkspacesAgentConfigs Regular 6.2 s :white_check_mark: +0.00 B
      20240815132712 - RemoveProjectsVulnerabilityMergeRequestLinksProjectIdFk Post deploy 7.2 s :white_check_mark: +0.00 B
      20240815132940 - RemoveMergeRequestsVulnerabilityMergeRequestLinksMergeRequestIdFk Post deploy 7.1 s :white_check_mark: +0.00 B
      Clone details
      Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
      database-testing-3650270-15172962-main 2024-08-29T09:59:31Z 2024-08-29T08:00:59Z 2024-08-29 22:43:09 +0000
      database-testing-3650270-15172962-ci 2024-08-29T09:59:31Z 2024-08-29T08:44:58Z 2024-08-29 22:43:09 +0000

      Job artifacts

      Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

      Edited by ****
  • A deleted user added backend database databasereview pending labels

    added backend database databasereview pending labels

  • Ghost User
    Ghost User @ghost1 ·
    Contributor
    4 Warnings
    :warning: This MR changes code in ee/, but its Changelog commit is missing the EE: true trailer. Consider adding it to your Changelog commits.
    :warning: 2626861f: The commit subject must start with a capital letter. For more information, take a look at our Commit message guidelines.
    :warning: There were no new or modified feature flag YAML files detected in this MR.

    If the changes here are already controlled under an existing feature flag, please add
    the feature flagexists. Otherwise, if you think the changes here don't need
    to be under a feature flag, please add the label feature flagskipped, and
    add a short comment about why we skipped the feature flag.

    For guidance on when to use a feature flag, please see the documentation.
    :warning: The master pipeline status page reported failures in

    If these jobs fail in your merge request with the same errors, then they are not caused by your changes.
    Please check for any on-going incidents in the incident issue tracker or in the #master-broken Slack channel.

    Reviewer roulette

    Category Reviewer Maintainer
    backend @rcobb profile link current availability (UTC-7, 9 hours behind author) @kerrizor profile link current availability (UTC-7, 9 hours behind author)
    database @tianwenchen profile link current availability (UTC+10, 8 hours ahead of author) @ahegyi profile link current availability (UTC+2, same timezone as author)
    ~"Verify" Reviewer review is optional for ~"Verify" @pedropombeiro profile link current availability (UTC+2, same timezone as author)

    Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • A deleted user added Data WarehouseImpact Check label

    added Data WarehouseImpact Check label

  • Max Orefice added 1 commit

    added 1 commit

    • 6c1817e3 - Add and backfill project_id for p_ci_pipeline_variables

    Compare with previous version

  • Max Orefice added 1 commit

    added 1 commit

    • 53300206 - Add and backfill project_id for p_ci_pipeline_variables

    Compare with previous version

  • Max Orefice
  • Max Orefice added pipeline:skip-check-migrations label

    added pipeline:skip-check-migrations label

  • Max Orefice
  • Max Orefice added 1 commit

    added 1 commit

    • 84983b2a - Add and backfill project_id for p_ci_pipeline_variables

    Compare with previous version

  • **** added database-testing-automation label

    added database-testing-automation label

  • Max Orefice added 1 commit

    added 1 commit

    • d36b1872 - Add and backfill project_id for p_ci_pipeline_variables

    Compare with previous version

  • Max Orefice added 268 commits

    added 268 commits

    Compare with previous version

  • Max Orefice added workflowin review label and removed workflowready for development label

    added workflowin review label and removed workflowready for development label

  • Max Orefice removed pipeline:skip-check-migrations label

    removed pipeline:skip-check-migrations label

  • Max Orefice added 1 commit

    added 1 commit

    • 27ac6831 - Add and backfill project_id for p_ci_pipeline_variables

    Compare with previous version

  • Max Orefice added 1 commit

    added 1 commit

    • 1a13110d - apply raj patch to fix shared spec

    Compare with previous version

  • Max Orefice changed the description

    changed the description

  • Max Orefice requested review from @tianwenchen

    requested review from @tianwenchen

  • Max Orefice marked this merge request as ready

    marked this merge request as ready

  • Pedro Pombeiro
  • Pedro Pombeiro
  • Pedro Pombeiro mentioned in commit dead8599

    mentioned in commit dead8599

  • Pedro Pombeiro mentioned in merge request !164386 (merged)

    mentioned in merge request !164386 (merged)

  • Pedro Pombeiro
  • Pedro Pombeiro mentioned in commit 788a28c6

    mentioned in commit 788a28c6

  • Pedro Pombeiro mentioned in commit 1cff968d

    mentioned in commit 1cff968d

  • Max Orefice mentioned in issue #463244 (closed)

    mentioned in issue #463244 (closed)

  • Tianwen Chen
  • Tianwen Chen
  • Tianwen Chen
  • Pedro Pombeiro mentioned in commit 92104853

    mentioned in commit 92104853

  • Max Orefice added 1 commit

    added 1 commit

    Compare with previous version

  • Max Orefice added 1 commit

    added 1 commit

    Compare with previous version

  • Tianwen Chen approved this merge request

    approved this merge request

  • 🤖 GitLab Bot 🤖 added pipeline:mr-approved label

    added pipeline:mr-approved label

  • 🤖 GitLab Bot 🤖 added pipelinetier-2 label and removed pipelinetier-1 label

    added pipelinetier-2 label and removed pipelinetier-1 label

  • 🤖 GitLab Bot 🤖
    🤖 GitLab Bot 🤖 @gitlab-bot ·
    Maintainer

    Before you set this MR to auto-merge

    This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.

    Before you set this MR to auto-merge, please check the following:

    • You are the last maintainer of this merge request
    • The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
    • This pipeline is recent enough (created in the last 8 hours)

    If all the criteria above apply, please set auto-merge for this merge request.

    See pipeline tiers and merging a merge request for more details.

  • Tianwen Chen added databasereviewed pipelinetier-1 labels and removed databasereview pending pipelinetier-2 labels

    added databasereviewed pipelinetier-1 labels and removed databasereview pending pipelinetier-2 labels

  • Tianwen Chen removed review request for @tianwenchen

    removed review request for @tianwenchen

  • Justin Stark
    Justin Stark @jjstark ·
    Developer

    additive schema change; no impact to data warehouse

  • Justin Stark added Data WarehouseNot Impacted label and removed Data WarehouseImpact Check label

    added Data WarehouseNot Impacted label and removed Data WarehouseImpact Check label

  • Pedro Pombeiro mentioned in commit a788b6a9

    mentioned in commit a788b6a9

  • Max Orefice added 454 commits

    added 454 commits

    Compare with previous version

    Toggle commit list
  • Pedro Pombeiro mentioned in commit 479c87a5

    mentioned in commit 479c87a5

  • Pedro Pombeiro mentioned in commit 9d410e25

    mentioned in commit 9d410e25

  • Max Orefice added 210 commits

    added 210 commits

    Compare with previous version

  • Max Orefice reset approvals from @tianwenchen by pushing to the branch

    reset approvals from @tianwenchen by pushing to the branch

  • Pedro Pombeiro mentioned in commit b61d4365

    mentioned in commit b61d4365

  • Pedro Pombeiro
  • ****
    **** @project_278964_bot_a1e04ca047b466ba2f6534548fe23088 ·
    Maintainer

    Database migrations (on the main database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20240827114849 - AddProjectIdToPCiPipelineVariables Regular 5.2 s :white_check_mark: +0.00 B
    20240827114958 - IndexPCiPipelineVariablesOnProjectId Post deploy 6.4 s :white_check_mark: +0.00 B
    20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint Post deploy 5.6 s :white_check_mark: +8.00 KiB [note]
    20240828074159 - QueueBackfillPCiPipelineVariablesProjectId Post deploy 4.2 s :white_check_mark: +0.00 B
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 6
    0.1 seconds - 1 second 14
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827114849 - AddProjectIdToPCiPipelineVariables

    • Type: Regular
    • Duration: 5.2 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 20.6 ms 20.6 ms 20.6 ms 0 
    ALTER TABLE "p_ci_pipeline_variables" ADD "project_id" bigint
    1 0.1 ms 0.1 ms 0.1 ms 1 
    SELECT "feature_gates"."key", "feature_gates"."value"  FROM "feature_gates"  WHERE "feature_gates"."feature_key" = $1
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddProjectIdToPCiPipelineVariables
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 1
    0.1 seconds - 1 second 3
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827114958 - IndexPCiPipelineVariablesOnProjectId

    • Type: Post deploy
    • Duration: 6.4 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    2 14.7 ms 14.6 ms 7.4 ms 2 
    INSERT INTO "postgres_async_indexes" ("created_at", "updated_at", "name", "definition", "table_name") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
    1 5.1 ms 5.1 ms 5.1 ms 2 
    SELECT "postgres_partitions".*
      FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
      ORDER BY "postgres_partitions"."name" ASC
    1 0.4 ms 0.4 ms 0.4 ms 1 
    SELECT "postgres_partitioned_tables".*
      FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
      LIMIT $3
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for IndexPCiPipelineVariablesOnProjectId
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 1
    0.1 seconds - 1 second 5
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint

    • Type: Post deploy
    • Duration: 5.6 s
    • Database size change: +8.00 KiB [note]
    Calls Total Time Max Time Mean Time Rows Query
    1 9.7 ms 9.7 ms 9.7 ms 0 
    ALTER TABLE public.ci_pipeline_variables ADD CONSTRAINT check_6e932dbabf CHECK ( project_id IS NOT NULL ) NOT VALID
    1 4.5 ms 4.5 ms 4.5 ms 2 
    SELECT "postgres_partitions".*
      FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
      ORDER BY "postgres_partitions"."name" ASC
    1 2.0 ms 2.0 ms 2.0 ms 0 
    ALTER TABLE gitlab_partitions_dynamic.ci_pipeline_variables_102 ADD CONSTRAINT check_6e932dbabf CHECK ( project_id IS NOT NULL ) NOT VALID
    2 1.8 ms 1.0 ms 0.9 ms 2 
    SELECT c.is_nullable
      FROM information_schema.columns c
      WHERE c.table_schema = $1 AND c.table_name = $2 AND c.column_name = $3
    1 0.3 ms 0.3 ms 0.3 ms 1 
    SELECT "postgres_partitioned_tables".*
      FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
      LIMIT $3
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddPCiPipelineVariablesProjectIdNullConstraint
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 6
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240828074159 - QueueBackfillPCiPipelineVariablesProjectId

    • Type: Post deploy
    • Duration: 4.2 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for QueueBackfillPCiPipelineVariablesProjectId
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20240828100019 - ReAddProjectsSecurityTrainingsProjectIdFk Post deploy 4.4 s :warning: +0.00 B
    20240902022900 - DropIndexEventsOnGroupIdPartial Post deploy 6.4 s :white_check_mark: -44.29 MiB
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-3665155-15224124-main 2024-09-03T12:39:31Z 2024-09-03T11:40:08Z 2024-09-04 01:21:15 +0000
    database-testing-3665155-15224124-ci 2024-09-03T12:39:31Z 2024-09-03T08:46:12Z 2024-09-04 01:21:15 +0000

    Job artifacts

    Database migrations (on the ci database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20240827114849 - AddProjectIdToPCiPipelineVariables Regular 7.0 s :white_check_mark: +8.00 KiB [note]
    20240827114958 - IndexPCiPipelineVariablesOnProjectId Post deploy 8.3 s :white_check_mark: +0.00 B
    20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint Post deploy 7.6 s :white_check_mark: +16.00 KiB
    20240828074159 - QueueBackfillPCiPipelineVariablesProjectId Post deploy 7.4 s :white_check_mark: +0.00 B
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 7
    0.1 seconds - 1 second 16
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827114849 - AddProjectIdToPCiPipelineVariables

    • Type: Regular
    • Duration: 7.0 s
    • Database size change: +8.00 KiB [note]
    Calls Total Time Max Time Mean Time Rows Query
    1 6.0 ms 6.0 ms 6.0 ms 0 
    ALTER TABLE "p_ci_pipeline_variables" ADD "project_id" bigint
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddProjectIdToPCiPipelineVariables
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 1
    0.1 seconds - 1 second 2
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827114958 - IndexPCiPipelineVariablesOnProjectId

    • Type: Post deploy
    • Duration: 8.3 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 6.4 ms 6.4 ms 6.4 ms 1 
    SELECT "postgres_partitioned_tables".*
      FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
      LIMIT $3
    2 6.0 ms 5.9 ms 3.0 ms 2 
    INSERT INTO "postgres_async_indexes" ("created_at", "updated_at", "name", "definition", "table_name") VALUES ($1, $2, $3, $4, $5) RETURNING "id"
    1 3.7 ms 3.7 ms 3.7 ms 2 
    SELECT "postgres_partitions".*
      FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
      ORDER BY "postgres_partitions"."name" ASC
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for IndexPCiPipelineVariablesOnProjectId
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 4
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240827115247 - AddPCiPipelineVariablesProjectIdNullConstraint

    • Type: Post deploy
    • Duration: 7.6 s
    • Database size change: +16.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    1 6.8 ms 6.8 ms 6.8 ms 1 
    SELECT "postgres_partitioned_tables".*
      FROM "postgres_partitioned_tables"  WHERE (identifier = concat(current_schema(), $1, $2))
      LIMIT $3
    1 3.6 ms 3.6 ms 3.6 ms 2 
    SELECT "postgres_partitions".*
      FROM "postgres_partitions"  WHERE "postgres_partitions"."parent_identifier" = $1
      ORDER BY "postgres_partitions"."name" ASC
    1 2.5 ms 2.5 ms 2.5 ms 0 
    ALTER TABLE gitlab_partitions_dynamic.ci_pipeline_variables_102 ADD CONSTRAINT check_6e932dbabf CHECK ( project_id IS NOT NULL ) NOT VALID
    1 1.4 ms 1.4 ms 1.4 ms 0 
    ALTER TABLE public.ci_pipeline_variables ADD CONSTRAINT check_6e932dbabf CHECK ( project_id IS NOT NULL ) NOT VALID
    2 1.3 ms 0.7 ms 0.7 ms 2 
    SELECT c.is_nullable
      FROM information_schema.columns c
      WHERE c.table_schema = $1 AND c.table_name = $2 AND c.column_name = $3
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for AddPCiPipelineVariablesProjectIdNullConstraint
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 6
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20240828074159 - QueueBackfillPCiPipelineVariablesProjectId

    • Type: Post deploy
    • Duration: 7.4 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 32.6 ms 32.6 ms 32.6 ms 1 
    SELECT MAX("id")
      FROM "p_ci_pipeline_variables"
    1 6.3 ms 6.3 ms 6.3 ms 1 
    INSERT INTO "batched_background_migrations" ("created_at", "updated_at", "max_value", "batch_size", "sub_batch_size", "interval", "status", "job_class_name", "table_name", "column_name", "job_arguments", "total_tuple_count", "started_at", "gitlab_schema", "queued_migration_version") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15) RETURNING "id"
    1 1.7 ms 1.7 ms 1.7 ms 0 
    SELECT $1 AS one
      FROM "batched_background_migrations"  WHERE "batched_background_migrations"."job_arguments" = $2 AND "batched_background_migrations"."job_class_name" = $3 AND "batched_background_migrations"."table_name" = $4 AND "batched_background_migrations"."column_name" = $5
      LIMIT $6
    1 1.3 ms 1.3 ms 1.3 ms 0 
    SELECT $1 AS one
      FROM "batched_background_migrations"  WHERE "batched_background_migrations"."job_class_name" = $2 AND "batched_background_migrations"."table_name" = $3 AND "batched_background_migrations"."column_name" = $4 AND (job_arguments = $5) AND "batched_background_migrations"."gitlab_schema" = $6
      LIMIT $7
    2 0.0 ms 0.0 ms 0.0 ms 2 
    SELECT pg_backend_pid()
    Histogram for QueueBackfillPCiPipelineVariablesProjectId
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 2
    0.1 seconds - 1 second 4
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Background Migration: BackfillPCiPipelineVariablesProjectId

    Sampled 17 batches. Estimated Time to complete: 1 month, 6 days, 15 hours, 18 minutes, and 54 seconds
    • Interval: 120s
    • Total tuple count: 2002064800
    • Max batch size: 0
    • Estimated seconds to complete: 3203280s
    • Estimated number of batches: 26694
    • Average batch time: 107.89s
    • Batch size: 75000
    • N. of batches sampled: 17
    • N. of failed batches: 0

    Time estimation is conservative and based on sampling production data in a test environment. It represents the max time that migration could take. The actual time may differ from this estimation.

    Consider changing max_batch_size and interval if this estimate is unacceptable.

    Calls Total Time Max Time Mean Time Rows Query
    5083 213410.8 ms 456.9 ms 42.0 ms 1270750f 
    UPDATE p_ci_pipeline_variables
      SET project_id = p_ci_pipelines.project_id
      FROM p_ci_pipelines
      WHERE p_ci_pipelines.id = p_ci_pipeline_variables.pipeline_id AND p_ci_pipelines.partition_id = p_ci_pipeline_variables.partition_id AND p_ci_pipeline_variables.id IN (
        SELECT p_ci_pipeline_variables.id
        FROM p_ci_pipeline_variables
        WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3 AND p_ci_pipeline_variables.id < $4
      )
    17 605.4 ms 125.7 ms 35.6 ms 4206 
    UPDATE p_ci_pipeline_variables
      SET project_id = p_ci_pipelines.project_id
      FROM p_ci_pipelines
      WHERE p_ci_pipelines.id = p_ci_pipeline_variables.pipeline_id AND p_ci_pipelines.partition_id = p_ci_pipeline_variables.partition_id AND p_ci_pipeline_variables.id IN (
        SELECT p_ci_pipeline_variables.id
        FROM p_ci_pipeline_variables
        WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3
      )
    5100 1199.0 ms 15.7 ms 0.2 ms 5083 
    SELECT p_ci_pipeline_variables.id
      FROM p_ci_pipeline_variables
      WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL AND p_ci_pipeline_variables.id >= $3
      ORDER BY p_ci_pipeline_variables.id ASC
      LIMIT $4
      OFFSET $5
    17 21.0 ms 10.5 ms 1.2 ms 17 
    UPDATE batched_background_migration_jobs
      SET updated_at = $1, finished_at = $2, status = $3, metrics = $4
      WHERE batched_background_migration_jobs.id = $5
    17 15.1 ms 9.6 ms 0.9 ms 17 
    UPDATE batched_background_migration_jobs
      SET updated_at = $1, started_at = $2, status = $3, attempts = $4
      WHERE batched_background_migration_jobs.id = $5
    34 13.7 ms 7.5 ms 0.4 ms 34 
    INSERT INTO batched_background_migration_job_transition_logs (batched_background_migration_job_id, created_at, updated_at, previous_status, next_status) VALUES ($1, $2, $3, $4, $5) RETURNING id
    34 2.7 ms 0.2 ms 0.1 ms 34 
    SELECT batched_background_migration_jobs.*
      FROM batched_background_migration_jobs
      WHERE batched_background_migration_jobs.id = $1
      LIMIT $2
    17 0.9 ms 0.1 ms 0.1 ms 17 
    SELECT sum(batched_background_migration_jobs.batch_size)
      FROM batched_background_migration_jobs
      WHERE batched_background_migration_jobs.batched_background_migration_id = $1 AND batched_background_migration_jobs.status IN ($2)
    17 0.7 ms 0.1 ms 0.0 ms 17 
    SELECT p_ci_pipeline_variables.id
      FROM p_ci_pipeline_variables
      WHERE p_ci_pipeline_variables.id BETWEEN $1 AND $2 AND p_ci_pipeline_variables.project_id IS NULL
      ORDER BY p_ci_pipeline_variables.id ASC
      LIMIT $3
    17 0.7 ms 0.1 ms 0.0 ms 17 
    SELECT batched_background_migrations.*
      FROM batched_background_migrations
      WHERE batched_background_migrations.id = $1
      LIMIT $2
    Histogram of batch runtimes for BackfillPCiPipelineVariablesProjectId
    Batch Runtime Count
    0 seconds - 10 seconds 0
    10 seconds - 1 minute 0
    1 minute - 2 minutes 16
    2 minutes - 3 minutes 1
    3 minutes - 5 minutes 0
    5 minutes + 0
    Histogram across all sampled batches of BackfillPCiPipelineVariablesProjectId
    Query Runtime Count
    0 seconds - 0.1 seconds 1528
    0.1 seconds - 0.5 seconds 8821
    0.5 seconds - 1 second 4
    1 second - 2 seconds 0
    2 seconds - 5 seconds 0
    5 seconds + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20240828100019 - ReAddProjectsSecurityTrainingsProjectIdFk Post deploy 6.6 s :white_check_mark: +0.00 B
    20240902022900 - DropIndexEventsOnGroupIdPartial Post deploy 8.4 s :white_check_mark: -8.00 KiB
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-3665155-15224124-main 2024-09-03T12:39:31Z 2024-09-03T11:40:08Z 2024-09-04 01:21:15 +0000
    database-testing-3665155-15224124-ci 2024-09-03T12:39:31Z 2024-09-03T08:46:12Z 2024-09-04 01:21:15 +0000

    Job artifacts

    Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

  • Max Orefice requested review from @mbobin

    requested review from @mbobin

  • Pedro Pombeiro mentioned in commit f8951dc9

    mentioned in commit f8951dc9

  • Marius Bobin added pipelinetier-3 label and removed pipelinetier-1 label

    added pipelinetier-3 label and removed pipelinetier-1 label

  • Pedro Pombeiro mentioned in commit 89cb64d6

    mentioned in commit 89cb64d6

  • Ghost User
    Ghost User @ghost1 ·
    Contributor

    :tools: Generated by gitlab_quality-test_tooling.

    :snail: Slow tests detected in this merge request. These slow tests might be related to this merge request's changes.

    Click to expand
    Job File Name Duration Expected duration
    #7740538845 spec/lib/gitlab/database/decomposition/migrate_spec.rb#L103 Gitlab::Database::Decomposition::Migrate#process! when the checks pass copies main database to ci database 29.36 s < 27.12 s
  • A deleted user added rspec:slow test detected label

    added rspec:slow test detected label

  • Ghost User
    Ghost User @ghost1 ·
    Contributor

    E2E Test Result Summary

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for 2626861f

    expand test summary 
    +------------------------------------------------------------------+
|                          suites summary                          |
+-------------+--------+--------+---------+-------+-------+--------+
|             | passed | failed | skipped | flaky | total | result |
+-------------+--------+--------+---------+-------+-------+--------+
| Plan        | 72     | 0      | 1       | 0     | 73    | ✅     |
| Create      | 128    | 0      | 16      | 0     | 144   | ✅     |
| Govern      | 72     | 0      | 0       | 0     | 72    | ✅     |
| Monitor     | 8      | 0      | 0       | 0     | 8     | ✅     |
| Verify      | 44     | 0      | 2       | 0     | 46    | ✅     |
| Package     | 19     | 0      | 13      | 0     | 32    | ✅     |
| Fulfillment | 2      | 0      | 0       | 0     | 2     | ✅     |
| Data Stores | 31     | 0      | 1       | 0     | 32    | ✅     |
| Release     | 5      | 0      | 0       | 0     | 5     | ✅     |
| Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
| Secure      | 3      | 0      | 0       | 0     | 3     | ✅     |
| Manage      | 1      | 0      | 1       | 0     | 2     | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
| Total       | 387    | 0      | 34      | 0     | 421   | ✅     |
+-------------+--------+--------+---------+-------+-------+--------+
  • Pedro Pombeiro
  • Marius Bobin requested changes

    requested changes

  • Marius Bobin
  • Marius Bobin
  • Max Orefice added 290 commits

    added 290 commits

    Compare with previous version

  • Max Orefice requested review from @mbobin

    requested review from @mbobin

  • Ghost User
  • Marius Bobin approved this merge request

    approved this merge request

  • Marius Bobin added databaseapproved label and removed databasereviewed label

    added databaseapproved label and removed databasereviewed label

  • Marius Bobin resolved all threads

    resolved all threads

  • Marius Bobin enabled automatic add to merge train when checks pass

    enabled automatic add to merge train when checks pass

  • Max Orefice mentioned in merge request !165181 (merged)

    mentioned in merge request !165181 (merged)

  • Marius Bobin added this merge request to the merge train at position 3

    added this merge request to the merge train at position 3

  • Marius Bobin merged

    merged

  • 🤖 GitLab Bot 🤖
    🤖 GitLab Bot 🤖 @gitlab-bot ·
    Maintainer

    Hello @morefice :wave:

    The database team is looking for ways to improve the database review process and we would love your help!

    If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

    @gitlab-org/database-team

    And someone will be by shortly!

    Thanks for your help! :heart:

    This message was generated automatically. You're welcome to improve it.

  • Marius Bobin mentioned in commit 5e27395b

    mentioned in commit 5e27395b

  • Krasimir Angelov
    Krasimir Angelov @krasio started a thread on the diff
    • db/post_migrate/20240827115247_add_p_ci_pipeline_variables_project_id_null_constraint.rb 0 → 100644
    1 # frozen_string_literal: true
    2
    3 class AddPCiPipelineVariablesProjectIdNullConstraint < Gitlab::Database::Migration[2.2]
    4 disable_ddl_transaction!
    5 milestone '17.4'
    6
    7 TABLE_NAME = :p_ci_pipeline_variables
    8 COLUMN_NAME = :project_id
    9 CONSTRAINT_NAME = :check_6e932dbabf
    10
    11 def up
    12 Gitlab::Database::PostgresPartitionedTable.each_partition(TABLE_NAME) do |partition|
    13 add_not_null_constraint(partition.identifier, COLUMN_NAME, constraint_name: CONSTRAINT_NAME, validate: false)
  • Krasimir Angelov
    Krasimir Angelov @krasio started a thread on the diff
    • app/models/ci/pipeline_variable.rb
    6 6 include Ci::HasVariable
    7 7 include Ci::RawVariable
    8 8
    9 before_validation :ensure_project_id, on: :create
  • GitLab Release Tools Bot added workflowstaging-canary label and removed workflowin review label

    added workflowstaging-canary label and removed workflowin review label

  • GitLab Release Tools Bot added workflowcanary label and removed workflowstaging-canary label

    added workflowcanary label and removed workflowstaging-canary label

  • GitLab Release Tools Bot added workflowstaging label and removed workflowcanary label

    added workflowstaging label and removed workflowcanary label

  • GitLab Release Tools Bot added workflowproduction label and removed workflowstaging label

    added workflowproduction label and removed workflowstaging label

  • Max Orefice mentioned in merge request !165306 (merged)

    mentioned in merge request !165306 (merged)

  • GitLab Release Tools Bot added workflowpost-deploy-db-staging label and removed workflowproduction label

    added workflowpost-deploy-db-staging label and removed workflowproduction label

  • Mark Nuzzo mentioned in epic gitlab-org#14749

    mentioned in epic gitlab-org#14749

  • service-epic-status-automation mentioned in epic &14749

    mentioned in epic &14749

  • service-epic-status-automation mentioned in epic &12383

    mentioned in epic &12383

  • GitLab Release Tools Bot added releasedcandidate label

    added releasedcandidate label

  • Kubitus Bot mentioned in merge request kubitus-project/kubitus-installer!3354 (merged)

    mentioned in merge request kubitus-project/kubitus-installer!3354 (merged)

  • GitLab Release Tools Bot added releasedpublished label and removed releasedcandidate label

    added releasedpublished label and removed releasedcandidate label

  • Max Orefice mentioned in merge request !171427 (merged)

    mentioned in merge request !171427 (merged)

  • Jayakrishnan Mallissery mentioned in commit 6536770a

    mentioned in commit 6536770a

  • Jayakrishnan Mallissery mentioned in merge request !180066 (merged)

    mentioned in merge request !180066 (merged)

  • Jayakrishnan Mallissery mentioned in commit 0bfb183a

    mentioned in commit 0bfb183a

  • Jayakrishnan Mallissery mentioned in commit a71140c6

    mentioned in commit a71140c6

  • Jayakrishnan Mallissery mentioned in merge request !181106 (merged)

    mentioned in merge request !181106 (merged)

  • Jayakrishnan Mallissery mentioned in commit cab03e1b

    mentioned in commit cab03e1b

  • Jayakrishnan Mallissery mentioned in commit 9d730d00

    mentioned in commit 9d730d00

  • Jayakrishnan Mallissery mentioned in commit 863cfcbd

    mentioned in commit 863cfcbd

  • Jayakrishnan Mallissery mentioned in commit 7c0ddaab

    mentioned in commit 7c0ddaab

  • Jayakrishnan Mallissery mentioned in commit dd67b659

    mentioned in commit dd67b659

  • Jayakrishnan Mallissery mentioned in commit 475d8290

    mentioned in commit 475d8290

  • Jayakrishnan Mallissery mentioned in commit 0c634f09

    mentioned in commit 0c634f09

  • Jayakrishnan Mallissery mentioned in commit b2c07bcb

    mentioned in commit b2c07bcb

  • Jayakrishnan Mallissery mentioned in commit 0337ecbd

    mentioned in commit 0337ecbd

    • Please register or sign in to reply