Skip to content

[Feature flag] Enable :project_import_ndjson feature flag on staging and measure results

What

Since we have merged our NDJSON work on master (&2734)

We should enable project_import_ndjson feature flag on staging, for 'import_export_test' group, and run project import rake task in order to test, and measure its performance.

We should test import with feature flag enabled for export in legacy formant and export in NDJSON format.

We should compare memory consumption when importing big and small projects.

Roll Out Steps

  1. Enable project_import_ndjson feature flag
  2. Run again import rake task
bundle exec rake "gitlab:import_export:import[user_name, 'import_export_test', 'big-project-ndjson-import', /path/to/ndjson_export_big.tar.gz, true]"
  1. Run import rake task for the small export - ndjson_export_small.tar.gz from #215004 (closed)
bundle exec rake "gitlab:import_export:import[user_name, 'import_export_test', 'small-project-ndjson-import', /path/to/ndjson_export_small.tar.gz, true]"
  1. Run import rake task for the legacy export - legacy_export_big.tar.gz from #215004 (closed)
bundle exec rake "gitlab:import_export:import[user_name, 'import_export_test', 'big-project-ndjson-from-legacy-import', /path/to/legacy_export_big.tar.gz, true]"
  1. Compare measurement results from 2, 3 and 4. and add them as a comment to this issue
  2. Test importing of ndjson_export_small.tar.gz and legacy_export_small.tar.gz from #215004 (closed) from UI
  3. Disable feature flag on staging

Owners

  • Team: Memory team
  • Most appropriate slack channel to reach out to: #g_memory
  • Best individual to reach out to: @nmilojevic1, @qzhaogitlab

Expectations

We don't expect a big improvement in execution time during import, but we should have constant memory consumption.

Edited by Nikola Milojevic