Skip to content

Increase the limit of batched background migrations executed in parallel

Krasimir Angelov requested to merge 372316-parallel-bbm-increase-limit into master

What does this MR do and why?

With #372316 (closed) we implemented parallel execution of batched background migrations.

The feature flag batched_migrations_parallel_execution has been enabled for a few weeks now - #372316 (comment 1251597598), and it works as expected. Example - #384883 (comment 1264113258).

With the existing self-throttling checks in place (!84555 (merged) & !85196 (merged)), I think we can safely increase the limit of migrations executing in parallel from 2 to 4. The next step will be to make it easily configurable.

Screenshots or screen recordings

Here is an example as of time of writing - https://log.gprd.gitlab.net/goto/f12b0500-a828-11ed-85ed-e7557b0a598c. We can see that there are multiple migrations being executed at the same time (for both main and ci databases, though json.stage is not showing this correctly). There are more than 2 migrations (the current limit) as some of them are being put on hold when there is a signal, and another one is picked up meanwhile - https://log.gprd.gitlab.net/goto/5827eb10-a829-11ed-85ed-e7557b0a598c.

image

image

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #372316 (closed)

Edited by Krasimir Angelov

Merge request reports