Skip to content

Finish CI FKs conversion and create new partitions

Marius Bobin requested to merge 414396-complete-ci-builds-partitioning into master

What does this MR do and why?

!126122 (merged) failed to get the locks on ci_builds and p_ci_builds_metadata to complete the FK conversion. I think we have a better chance of getting the necessary locks during the weekend, since we have lower traffic.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

marius@rocket-sled ~/W/g/gitlab (414396-complete-ci-builds-partitioning)> GITLAB_SIMULATE_SAAS=1 pgai use -o ci bin/rails runner "Feature.enable(:complete_p_ci_builds_partitioning); Gitlab::Database::CiBuildsPartitioning.new(logger: Gitlab::JsonLogger.new(STDOUT)).execute"
{"severity":"INFO","time":"2023-07-17T12:20:33.939Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:35.141Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:35.507Z","message":"Foreign key successfully renamed","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:20:36.038Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:36.780Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:37.142Z","message":"Partition for p_ci_builds successfully created","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:20:37.672Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:37.874Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:38.234Z","message":"Partition for p_ci_job_annotations successfully created","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:20:38.768Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Lock timeout is set","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:38.969Z","method":"with_lock_retries","class":"Gitlab::Database::CiBuildsPartitioning","message":"Migration finished","current_iteration":1,"lock_timeout_in_ms":10000}
{"severity":"INFO","time":"2023-07-17T12:20:39.326Z","message":"Partition for p_ci_runner_machine_builds successfully created","class":"Gitlab::Database::CiBuildsPartitioning"}

Subsequent executions are skipped:

marius@rocket-sled ~/W/g/gitlab (414396-complete-ci-builds-partitioning)> GITLAB_SIMULATE_SAAS=1 pgai use -o ci bin/rails runner "Feature.enable(:complete_p_ci_builds_partitioning); Gitlab::Database::CiBuildsPartitioning.new(logger: Gitlab::JsonLogger.new(STDOUT)).execute"
{"severity":"INFO","time":"2023-07-17T12:21:32.625Z","message":"Foreign key already renamed, nothing to do","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:21:32.801Z","message":"p_ci_builds partition exists, nothing to do","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:21:32.977Z","message":"p_ci_job_annotations partition exists, nothing to do","class":"Gitlab::Database::CiBuildsPartitioning"}
{"severity":"INFO","time":"2023-07-17T12:21:33.153Z","message":"p_ci_runner_machine_builds partition exists, nothing to do","class":"Gitlab::Database::CiBuildsPartitioning"}

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #414396 (closed)

Edited by Marius Bobin

Merge request reports