Skip to content
Snippets Groups Projects

Cleanup for converted ci_stages.pipeline_id conversion

Merged Tianwen Chen requested to merge 422816-cleanup into master

What does this MR do and why?

Cleanup for converted ci_stages.pipeline_id conversion

This is to step to remove the trigger and old integer column See https://docs.gitlab.com/ee/development/database/avoiding_downtime_in_migrations.html#remove-the-trigger-and-old-integer-columns-release-n--2

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #422816 (closed)

Edited by Tianwen Chen

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Contributor
    1 Warning
    :warning: There were no new or modified feature flag YAML files detected in this MR.

    If the changes here are already controlled under an existing feature flag, please add
    the feature flagexists. Otherwise, if you think the changes here don't need
    to be under a feature flag, please add the label feature flagskipped, and
    add a short comment about why we skipped the feature flag.

    For guidance on when to use a feature flag, please see the documentation.

    Reviewer roulette

    Changes that require review have been detected!

    Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

    Category Reviewer Maintainer
    backend @tyleramos profile link current availability (UTC-5, 16 hours behind author) @mksionek profile link current availability (UTC+1, 10 hours behind author)
    database @dskim_gitlab profile link current availability (UTC+11, same timezone as author) @l.rosa profile link current availability (UTC-3, 14 hours behind author)

    Please check reviewer's status!

    • available Reviewer is available!
    • unavailable Reviewer is unavailable!

    Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

    To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

    Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • A deleted user added Data WarehouseImpact Check label
  • Tianwen Chen added 167 commits

    added 167 commits

    Compare with previous version

  • A deleted user added backend label

    added backend label

  • Contributor

    Allure report

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for c7c87abd

    expand test summary
    +-----------------------------------------------------------------------+
    |                            suites summary                             |
    +------------------+--------+--------+---------+-------+-------+--------+
    |                  | passed | failed | skipped | flaky | total | result |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Verify           | 31     | 0      | 0       | 0     | 31    | ✅     |
    | Data Stores      | 23     | 0      | 0       | 0     | 23    | ✅     |
    | Monitor          | 8      | 0      | 0       | 0     | 8     | ✅     |
    | Create           | 52     | 0      | 9       | 0     | 61    | ✅     |
    | Manage           | 0      | 0      | 1       | 0     | 1     | ➖     |
    | Govern           | 57     | 0      | 0       | 0     | 57    | ✅     |
    | Plan             | 55     | 0      | 0       | 0     | 55    | ✅     |
    | Framework sanity | 0      | 0      | 1       | 0     | 1     | ➖     |
    | Package          | 0      | 0      | 1       | 0     | 1     | ➖     |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Total            | 226    | 0      | 12      | 0     | 238   | ✅     |
    +------------------+--------+--------+---------+-------+-------+--------+

    e2e-package-and-test: :white_check_mark: test report for c7c87abd

    expand test summary
    +-------------------------------------------------------------+
    |                       suites summary                        |
    +--------+--------+--------+---------+-------+-------+--------+
    |        | passed | failed | skipped | flaky | total | result |
    +--------+--------+--------+---------+-------+-------+--------+
    | Verify | 138    | 0      | 27      | 0     | 165   | ✅     |
    | Create | 131    | 0      | 12      | 0     | 143   | ✅     |
    +--------+--------+--------+---------+-------+-------+--------+
    | Total  | 269    | 0      | 39      | 0     | 308   | ✅     |
    +--------+--------+--------+---------+-------+-------+--------+
  • Contributor

    Database migrations (on the main database)

    1 Warnings
    :warning: 20231120070345 - CleanupCiStagesPipelineIdBigint had a query that exceeded timing
    guidelines
    . Run time should not exceed 100ms, but it was 100.84ms. Please consider possible options
    to improve the query performance.
    ALTER TABLE "ci_stages" DROP COLUMN
    "pipeline_id_convert_to_bigint"

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20231120070345 - CleanupCiStagesPipelineIdBigint Post deploy 2.2 s :warning: -32.00 KiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 3
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    :warning: Migration: 20231120070345 - CleanupCiStagesPipelineIdBigint

    • Type: Post deploy
    • Duration: 2.2 s
    • Database size change: -32.00 KiB
    Calls Total Time Max Time Mean Time Rows Query
    1 100.8 ms 100.8 ms 100.8 ms 0
    ALTER TABLE "ci_stages" DROP COLUMN "pipeline_id_convert_to_bigint"
    1 16.5 ms 16.5 ms 16.5 ms 0
    LOCK TABLE ci_pipelines, ci_stages IN ACCESS EXCLUSIVE MODE
    1 9.4 ms 9.4 ms 9.4 ms 0
    DROP TRIGGER IF EXISTS trigger_07bc3c48f407 ON "ci_stages"
    1 3.8 ms 3.8 ms 3.8 ms 0
    DROP FUNCTION IF EXISTS trigger_07bc3c48f407()
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for CleanupCiStagesPipelineIdBigint
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 3
    0.1 seconds - 1 second 3
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20231109124031 - EnsureIdUniquenessForPCiBuildsV5 Post deploy 3.1 s :white_check_mark: +0.00 B
    20231114053330 - CleanupCiPipelinesAutoCanceledByIdBigint Post deploy 2.9 s :white_check_mark: -8.00 KiB
    20231115104943 - RemoveServiceAccessTokensCategoryColumn Post deploy 1.7 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2548311-11970317-main 2023-11-21T03:47:22Z 2023-11-21T01:18:55Z 2023-11-21 15:53:08 +0000
    database-testing-2548311-11970317-ci 2023-11-21T03:47:22Z 2023-11-21T00:47:05Z 2023-11-21 15:53:08 +0000

    Job artifacts

    Database migrations (on the ci database)

    1 Warnings
    :warning: 20231120070345 - CleanupCiStagesPipelineIdBigint had a query that exceeded timing
    guidelines
    . Run time should not exceed 100ms, but it was 100.26ms. Please consider possible options
    to improve the query performance.
    ALTER TABLE "ci_stages" DROP COLUMN
    "pipeline_id_convert_to_bigint"

    Migrations included in this change have been executed on gitlab.com data for testing purposes. For details, please see the migration testing pipeline (limited access).

    Migration Type Total runtime Result DB size change
    20231120070345 - CleanupCiStagesPipelineIdBigint Post deploy 52.8 s :warning: -160.15 GiB
    Runtime Histogram for all migrations
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 5
    0.1 seconds - 1 second 1
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    :warning: Migration: 20231120070345 - CleanupCiStagesPipelineIdBigint

    • Type: Post deploy
    • Duration: 52.8 s
    • Database size change: -160.15 GiB
    Calls Total Time Max Time Mean Time Rows Query
    1 100.3 ms 100.3 ms 100.3 ms 0
    ALTER TABLE "ci_stages" DROP COLUMN "pipeline_id_convert_to_bigint"
    1 16.0 ms 16.0 ms 16.0 ms 0
    DROP TRIGGER IF EXISTS trigger_07bc3c48f407 ON "ci_stages"
    1 3.3 ms 3.3 ms 3.3 ms 0
    LOCK TABLE ci_pipelines, ci_stages IN ACCESS EXCLUSIVE MODE
    1 0.2 ms 0.2 ms 0.2 ms 0
    DROP FUNCTION IF EXISTS trigger_07bc3c48f407()
    2 0.0 ms 0.0 ms 0.0 ms 2
    SELECT pg_backend_pid()
    Histogram for CleanupCiStagesPipelineIdBigint
    Query Runtime Count
    0 seconds - 0.01 seconds 0
    0.01 seconds - 0.1 seconds 5
    0.1 seconds - 1 second 1
    1 second - 5 seconds 0
    5 seconds - 15 seconds 0
    15 seconds - 5 minutes 0
    5 minutes + 0

    Other information

    Other migrations pending on GitLab.com
    Migration Type Total runtime Result DB size change
    20231109124031 - EnsureIdUniquenessForPCiBuildsV5 Post deploy 3.6 s :white_check_mark: +0.00 B
    20231114053330 - CleanupCiPipelinesAutoCanceledByIdBigint Post deploy 3.3 s :warning: -6.69 GiB
    20231115104943 - RemoveServiceAccessTokensCategoryColumn Post deploy 2.6 s :white_check_mark: +0.00 B
    Clone details
    Clone ID Clone Created At Clone Data Timestamp Expected Removal Time
    database-testing-2548311-11970317-main 2023-11-21T03:47:22Z 2023-11-21T01:18:55Z 2023-11-21 15:53:08 +0000
    database-testing-2548311-11970317-ci 2023-11-21T03:47:22Z 2023-11-21T00:47:05Z 2023-11-21 15:53:08 +0000

    Job artifacts


    Brought to you by gitlab-org/database-team/gitlab-com-database-testing. Epic

  • Tianwen Chen changed the description

    changed the description

  • Tianwen Chen
  • Tianwen Chen requested review from @eurie

    requested review from @eurie

  • Tianwen Chen marked this merge request as ready

    marked this merge request as ready

  • Ethan Urie
  • added databasereviewed label and removed databasereview pending label

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading