Reduce project import memory usage

In !16900 (merged)(for issue #27700 (closed)), it introduces daemon memory killer, which allows reasonable Sidekiq memory balloon caused by some high memory usage worker jobs(such as project import).

• if the project import finished soon(within balloon time setting), it is allowed
• if the project import takes long time, it allows to tell memory killer the Worker's estimated memory usage by setting sidekiq_options memory_killer_memory_growth_kb in worker, and memory killer will whitelist the job.

This would solve some user's concerns and allow project import succeed(without triggering Sidekiq daemon memory killer). However, such whitelist should be temporary measurement since memory is limited resource. So it is a technical debt: project import should reduce its memory usage.

There are some related work already happened or WIP, which may help(or resolve) this issue.

• #18857 (closed)
• #27070 (closed)