Improve performance while exporting CI pipelines
Related to: #391593 (closed)
What does this MR do and why?
CI pipeline statuses can't be preloaded correctly because statuses
is a polymorphic association, and each "real" association requires different associations to be preloaded.
For instance, CommitStatus
is the base class, and the child classes are Ci::Build
, Ci::Bridge
, and GenericCommitStatus
. And GenericCommitStatus
doesn't have the associations metadata
, which prevents us from preloading this association with the statuses
This change updates Import/Export to export the "real" associations instead of the polymorphic association statuses
.
See the example below of how the ci_pipelines were exported before and after:
ci_pipelines.ndjson format before
{
"ref": "ref",
"sha": "602c025ab40150333aa06fa8971446c057156e09",
...
"stages": [
{
"project_id": 1,
...
"statuses": [
{
"type": "Ci::Build"
},
{
"type": "Ci::Bridges"
},
{
"type": "GenericCommitStatus"
}
]
}
}
ci_pipelines.ndjson format after
{
"ref": "ref",
"sha": "602c025ab40150333aa06fa8971446c057156e09",
...
"stages": [
{
"project_id": 1,
...
"builds": [
{
}
],
"bridges": [
{
}
],
"generic_commit_statuses": [
{
}
]
}
}
And to still import old export files that use the statuses
association, the settings import_only_tree
was added to the import_export.yml
. The settings let the import process consider the statuses
association as valid, while during the export process, the settings is ignored.
This change also updates BulkImport aka GitLab migration, to export and import the new associations.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
It's necessary to perform 4 tests
Import/Export
- Import an export file with the old association
statuses
and check if the CI Pipelines were imported correctly - Import an export file with the new associations
builds
,bridges
, andgeneric_commit_statuses
and check if the CI Pipelines were imported correctly
GitLab Migration
- Import an export file with the old association
statuses
and check if the CI Pipelines were imported correctly - Import an export file with the new associations
builds
,bridges
, andgeneric_commit_statuses
and check if the CI Pipelines were imported correctly
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.