RepositoryImportWorker issues long database transactions
On GitLab.com, `RepositoryImportWorker` has been identified to issue long database transactions. Based on the [sampling of pg_stat_activity](https://log.gprd.gitlab.net/app/visualize#/edit/a59d8d30-2b40-11ec-a012-eb2e5674cacf?_g=(filters%3A!()%2CrefreshInterval%3A(pause%3A!t%2Cvalue%3A0)%2Ctime%3A(from%3Anow-7d%2Cto%3Anow))), we have seen transaction timings go up to about 7 minutes. This issue is to track investigation and proposed steps to change the design, so that we reduce the length of individual database transactions related to this worker. See https://gitlab.com/gitlab-org/gitlab/-/issues/342843 for more statistics. @lmcandrew: Similar issue https://gitlab.com/gitlab-org/gitlab/-/issues/328714. That issue was closed when we opened https://gitlab.com/gitlab-org/gitlab/-/issues/338638. However, I believe it makes sense to keep this issue open to track the ~infradev risk. ## Current status (2022-03-14) With the MR merged, we're currently rolling out the [feature flag](https://gitlab.com/gitlab-org/gitlab/-/issues/354497). In the past few hours, the flag has been enabled for staging and production! :tada:
issue