Optimise trace handling for big traces
What does this MR do?
This greatly optimizes handling of build traces and logic behind the handling of build traces in Runner. This should effectively allow a runner to send traces of 100MB to GitLab freely.
I reworked the current networking code for PatchTrace to simplify its implementation,
and make it less dependent on other code. This results in great code reduction.
I also reworked the update loop to have a very simple logic for update of trace and general update. We do not re-send trace, as this is redundant operation now. We rather perform incremental patching of trace, in chunks.
The PUT is used only to update the general state of CI job.
This also have serious impact on Memory of GitLab. We limit size of chunk, and we limit the runtime memory on Rails side. We also do not re-send trace as this is redundant operation.
Why was this MR needed?
Make GitLab Runner capable of running AOSP build that can generate 100MB of traces.
Does this MR meet the acceptance criteria?
- Documentation created/updated
- Added tests for this feature/bug
-
In case of conflicts with
master- branch was rebased