RepositoryImportWorker issues long database transactions

On GitLab.com, RepositoryImportWorker has been identified to issue long database transactions. Based on the sampling of pg_stat_activity, we have seen transaction timings go up to about 7 minutes.

This issue is to track investigation and proposed steps to change the design, so that we reduce the length of individual database transactions related to this worker.

See https://gitlab.com/gitlab-org/gitlab/-/issues/342843 for more statistics.

@lmcandrew: Similar issue #328714 (closed). That issue was closed when we opened #338638. However, I believe it makes sense to keep this issue open to track the infradev risk.

Current status (2022-03-14)

With the MR merged, we're currently rolling out the feature flag.

In the past few hours, the flag has been enabled for staging and production! 🎉