Improve JSON performance of Import/Export for large projects
Large projects with thousands of issues and MRs, will export a considerable amount of data to a JSON file (especially pipelines and the serialised st_diff
)
Originally raised in issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/26363 (Large export killed after the RSS threshold reached by the memory killer)
I suggest we:
Ignore build traces by default- Force use of libraries such as
oj
forto_json
active record method.
The latter may require some testing, as we may have to implement our own version in batches to reduce high memory use.