Skip to content
Snippets Groups Projects

Finalize conversion for FKs targeting p_ci_builds - 2 self-managed

Merged Marius Bobin requested to merge 414396-finalize-fk-conversion-4 into master

What does this MR do and why?

Related to #414396 (closed)

Completes the FK swap for ci_job_artifacts, ci_running_builds and ci_job_variables for self-managed by reintroducing the migrations from !124186 (merged) without the should_run? guards.

:warning: It depends on !124186 (merged) being successfully executed in production.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marius Bobin

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Marius Bobin changed milestone to %16.2

    changed milestone to %16.2

  • assigned to @mbobin

  • Contributor
    2 Warnings
    :warning: New migrations added but db/structure.sql wasn't updated

    Usually, when adding new migrations, db/structure.sql should be
    updated too (unless the migration isn't changing the DB schema
    and isn't the most recent one).

    :warning: You've made some app changes, but didn't add any tests.
    That's OK as long as you're refactoring existing code,
    but please consider adding any of the maintenancepipelines, maintenancerefactor, maintenanceworkflow, documentation, QA labels.

    Reviewer roulette

    Changes that require review have been detected!

    Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

    Category Reviewer Maintainer
    database Charlie Ablett current availability (@cablett) (UTC+12, 9 hours ahead of @mbobin) Tiger Watson current availability (@tigerwnz) (UTC+10, 7 hours ahead of @mbobin)
    ~"migration" No reviewer available No maintainer available

    To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

    To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

    Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Marius Bobin marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

    marked the checklist item I have evaluated the MR acceptance checklist for this MR. as completed

  • mentioned in issue #414396 (closed)

  • Contributor

    Allure report

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :heavy_minus_sign: test report for 805829c7

    expand test summary
    +------------------------------------------------------------+
    |                       suites summary                       |
    +-------+--------+--------+---------+-------+-------+--------+
    |       | passed | failed | skipped | flaky | total | result |
    +-------+--------+--------+---------+-------+-------+--------+
    +-------+--------+--------+---------+-------+-------+--------+
    | Total | 0      | 0      | 0       | 0     | 0     | ➖     |
    +-------+--------+--------+---------+-------+-------+--------+
  • Marius Bobin changed the description

    changed the description

  • Contributor

    Database migrations (on the main database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20230621102941 - ReplaceOldFkCiJobArtifactsToBuildsV2 Post deploy 1.9 s :white_check_mark: +0.00 B
    20230621103000 - ReplaceOldFkCiRunningBuildsToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230621103043 - ReplaceOldFkCiJobVariablesToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 9
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621102941 - ReplaceOldFkCiJobArtifactsToBuildsV2

    • Type: Post deploy
    • Duration: 1.9 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 4.3 ms 4.3 ms 4.3 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiJobArtifactsToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621103000 - ReplaceOldFkCiRunningBuildsToBuildsV2

    • Type: Post deploy
    • Duration: 1.8 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 2.8 ms 2.8 ms 2.8 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiRunningBuildsToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621103043 - ReplaceOldFkCiJobVariablesToBuildsV2

    • Type: Post deploy
    • Duration: 1.8 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 3.1 ms 3.1 ms 3.1 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiJobVariablesToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20230528203340 - DropMessageFromVulnerabilityOccurrences Post deploy 2.4 s :warning: +0.00 B
    20230619072744 - ScheduleIndexEventsOnProjectIdAndIdDescOnMergedActionForRemoval Post deploy 3.2 s :white_check_mark: +0.00 B
    20230619081412 - AddIndexToCiTriggersToken Post deploy 2.3 s :white_check_mark: +8.00 KiB
    20230619134106 - PrepareIndexForVulnerabilityReadsOnCommonProjectFilters Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072854 - ReplaceOldFkCiBuildPendingStatesToBuildsV2 Post deploy 2.4 s :warning: +0.00 B
    20230620072856 - ReplaceOldFkCiBuildTraceChunksToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072900 - ReplaceOldFkCiUnitTestFailuresToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072905 - ReplaceOldFkCiSourcesPipelinesToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072908 - ReplaceOldFkCiResourcesToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072911 - ReplaceOldFkCiBuildReportResultsToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072915 - ReplaceOldFkCiBuildNeedsToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072917 - ReplaceOldFkCiBuildsRunnerSessionToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072920 - ReplaceOldFkCiPendingBuildsToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620072925 - ReplaceOldFkCiBuildTraceMetadataToBuildsV2 Post deploy 1.8 s :white_check_mark: +0.00 B
    20230620105919 - ReplaceOldFkCiJobArtifactsToBuilds Post deploy 2.5 s :white_check_mark: +0.00 B
    20230620110004 - ReplaceOldFkCiRunningBuildsToBuilds Post deploy 2.4 s :white_check_mark: +0.00 B
    20230620110021 - ReplaceOldFkCiJobVariablesToBuilds Post deploy 2.4 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2043738-10394558-main 2023-06-21T12:06:40Z 2023-06-21T08:13:21Z 2023-06-22 00:13:05 +0000
    database-testing-2043738-10394558-ci 2023-06-21T12:06:40Z 2023-06-21T08:47:47Z 2023-06-22 00:13:05 +0000

    Job artifacts

    Database migrations (on the ci database)

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20230621102941 - ReplaceOldFkCiJobArtifactsToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230621103000 - ReplaceOldFkCiRunningBuildsToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230621103043 - ReplaceOldFkCiJobVariablesToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 9
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621102941 - ReplaceOldFkCiJobArtifactsToBuildsV2

    • Type: Post deploy
    • Duration: 2.7 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 3.4 ms 3.4 ms 3.4 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiJobArtifactsToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621103000 - ReplaceOldFkCiRunningBuildsToBuildsV2

    • Type: Post deploy
    • Duration: 2.7 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 5.1 ms 5.1 ms 5.1 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiRunningBuildsToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Migration: 20230621103043 - ReplaceOldFkCiJobVariablesToBuildsV2

    • Type: Post deploy
    • Duration: 2.7 s
    • Database size change: +0.00 B
    Calls Total Time Max Time Mean Time Rows Query
    1 2.5 ms 2.5 ms 2.5 ms 1
    SELECT $1 AS one
    FROM "postgres_foreign_keys" WHERE "postgres_foreign_keys"."constrained_table_name" = $2 AND "postgres_foreign_keys"."referenced_table_name" = $3 AND "postgres_foreign_keys"."name" = $4
    LIMIT $5
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for ReplaceOldFkCiJobVariablesToBuildsV2
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 0
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20230528203340 - DropMessageFromVulnerabilityOccurrences Post deploy 2.9 s :white_check_mark: +0.00 B
    20230619072744 - ScheduleIndexEventsOnProjectIdAndIdDescOnMergedActionForRemoval Post deploy 3.6 s :white_check_mark: +0.00 B
    20230619081412 - AddIndexToCiTriggersToken Post deploy 8.3 s :warning: +11.00 MiB
    20230619134106 - PrepareIndexForVulnerabilityReadsOnCommonProjectFilters Post deploy 3.7 s :white_check_mark: +0.00 B
    20230620072854 - ReplaceOldFkCiBuildPendingStatesToBuildsV2 Post deploy 3.6 s :warning: +0.00 B
    20230620072856 - ReplaceOldFkCiBuildTraceChunksToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072900 - ReplaceOldFkCiUnitTestFailuresToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072905 - ReplaceOldFkCiSourcesPipelinesToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072908 - ReplaceOldFkCiResourcesToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072911 - ReplaceOldFkCiBuildReportResultsToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072915 - ReplaceOldFkCiBuildNeedsToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072917 - ReplaceOldFkCiBuildsRunnerSessionToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072920 - ReplaceOldFkCiPendingBuildsToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620072925 - ReplaceOldFkCiBuildTraceMetadataToBuildsV2 Post deploy 2.7 s :white_check_mark: +0.00 B
    20230620105919 - ReplaceOldFkCiJobArtifactsToBuilds Post deploy 3.4 s :white_check_mark: +0.00 B
    20230620110004 - ReplaceOldFkCiRunningBuildsToBuilds Post deploy 3.4 s :white_check_mark: +0.00 B
    20230620110021 - ReplaceOldFkCiJobVariablesToBuilds Post deploy 3.3 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2043738-10394558-main 2023-06-21T12:06:40Z 2023-06-21T08:13:21Z 2023-06-22 00:13:05 +0000
    database-testing-2043738-10394558-ci 2023-06-21T12:06:40Z 2023-06-21T08:47:47Z 2023-06-22 00:13:05 +0000

    Job artifacts


    Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

  • Marius Bobin mentioned in epic &7522

    mentioned in epic &7522

  • Author Maintainer

    Adding pipeline:skip-undercoverage because the migration is not covered by tests since it was already executed.

  • Marius Bobin requested review from @morefice

    requested review from @morefice

  • Max Orefice approved this merge request

    approved this merge request

  • :wave: @morefice, thanks for approving this merge request.

    This is the first time the merge request is approved. To ensure full test coverage, a new pipeline will be started shortly.

    For more info, please refer to the following links:

  • Max Orefice requested review from @a_akgun

    requested review from @a_akgun

  • Max Orefice removed review request for @morefice

    removed review request for @morefice

  • added databasereviewed label and removed databasereview pending label

  • added workflowin review label and removed workflowin dev label

  • Alper Akgun removed review request for @a_akgun

    removed review request for @a_akgun

  • Max Orefice requested review from @pshutsin

    requested review from @pshutsin

  • Pavel Shutsin
  • Pavel Shutsin approved this merge request

    approved this merge request

  • Pavel Shutsin removed review request for @pshutsin

    removed review request for @pshutsin

  • Marius Bobin added 657 commits

    added 657 commits

    Compare with previous version

  • Marius Bobin requested review from @pshutsin

    requested review from @pshutsin

  • Marius Bobin added 821 commits

    added 821 commits

    Compare with previous version

  • added databaseapproved label and removed databasereviewed label

  • Pavel Shutsin resolved all threads

    resolved all threads

  • Pavel Shutsin removed review request for @pshutsin

    removed review request for @pshutsin

  • Pavel Shutsin enabled an automatic merge when the pipeline for 49764546 succeeds

    enabled an automatic merge when the pipeline for 49764546 succeeds

  • merged

  • Hello @mbobin :wave:

    The database team is looking for ways to improve the database review process and we would love your help!

    If you'd be open to someone on the database team reaching out to you for a chat, or if you'd like to leave some feedback asynchronously, just post a reply to this comment mentioning:

    @gitlab-org/database-team

    And someone will be by shortly!

    Thanks for your help! :heart:

    This message was generated automatically. You're welcome to improve it.

  • Pavel Shutsin mentioned in commit 0649487c

    mentioned in commit 0649487c

  • added workflowstaging label and removed workflowcanary label

  • Mark Florian mentioned in merge request !125126 (merged)

    mentioned in merge request !125126 (merged)

  • mentioned in merge request mbobin/headway!1 (closed)

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading