Skip to content

Reduce amount of locks needed for GitHub importer

Andreas Brandl requested to merge ab-54270-github-iid into master

What does this MR do?

Context: https://gitlab.com/gitlab-org/gitlab-ce/issues/54270

This MR removes iid tracking from the GitHub importer and flushes iid records for a project after an import has finished. This removes the need to lock the iid record while inserting e.g. merge_requests in a highly concurrent manner.

Flushing iid records can be done at any point in time. It is safe because the records are going to be recreated when another model instance is created. The records are needed because the provide a more fine-granular locking mechanic.

The MR re-schedules a data migration that fixes any offending iid records coming from the GitHub importer. We've executed this migration before.

What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/54270

Flushing records after-import and scheduling the migration closes https://gitlab.com/gitlab-org/gitlab-ce/issues/51817.

Does this MR meet the acceptance criteria?

Edited by Yorick Peterse

Merge request reports