On-demand enablement of export_reduce_relation_batch_size feature flag
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
There is export_reduce_relation_batch_size feature flag related to Adjust the batch size of export of relations with numerous nested associations. Rollout issue.
The feature flag should be used to reduce the batch size of export SQL statements for projects that are timing out during export.
Enabling the feature flag for the specific affected project may fix the issue. Enabling the feature flag might make the export slower, so we don’t intend to enable it globally.
There is no intention of making the feature flag globally available therefore it should be enabled for projects that the export fails.
From Adjust the batch size of export of relations wi... (#441426 - closed) description
Related to https://gitlab.com/gitlab-org/manage/import/import-export-performance/-/issues/4+. Root cause and solution described in https://gitlab.com/gitlab-org/manage/import/import-export-performance/-/issues/4#note_1766921962.
On exporting project, customer got error PG::QueryCanceled: ERROR: canceling statement due to statement timeout
The problem could be caused by the number of other SQLs executed on the same connection.
Reducing the BATCH_SIZE from 100 to a lower number would solve the problem as less SQLs would be executed.
Next steps
What we could do is to mention the error and feature flag in the known issues or troubleshooting section of file-based export docs. On SM users could in need enable it themselves, on .com they could request turning it on for them. The downside of the FF turned on is that DT could get slower as reducing the batch size in Direct Transfer may cause the destination instance to make more HTTP requests to download the exported batch files.
List of projects the FF was enabled
Find the list in an internal note.