Backend: Gaps from backend in job log traces

Summary

Collapsible sections are not being closed by the backend, also certain lines that belong to collapsible sections are returned way after a section was closed

Steps to reproduce

This seems to be reproducible on failing jobs with collapsible sections

Example Project

We have the following CNG-mirror project on GitLab.com https://gitlab.com/gitlab-org/build/CNG-mirror/-/jobs/1624962348

What is the current bug behavior?

With the feature flag `infinitely_collapsible_sections` ON

The collapsible sections start but do not end on the backend, causing the frontend not to render the inner contents of the job log

With the feature flag `infinitely_collapsible_sections` OFF

The order of the job log lines is incorrect

Further details of the bug to follow:

Currently the backend (on failing logs) starts collapsible sections without ending them

(Skipping job log lines for clarity)

{
  offset: 19421,
  content: [
    {
      text: '$ if [ "${UBI_PIPELINE}" = "true" ]; then',
      style: 'term-fg-l-green term-bold',
    },
  ],
  section: 'section-script-step-6',
  section_header: true,
},

Which is this section

If you look in the trace.json endpoint in the network tab, you will see that there's no object in the array that contains section_duration: 00:XX for the section: section-script-step-6 which marks the end of a collapsed section

There's another issue which is the end of the job log, with the feature flag turned off we show the following error line

The object looks like this

{
  offset: 33253,
  content: [
    {
      text: 'ERROR: Job failed: exit code 1',
      style: 'term-fg-l-red term-bold',
    },
  ],
  section: 'section-script-step-34',
},

But this is returned at the end which raises the question, Should this be added as part of the section: 'section-script-step-34' or be added in the end as a normal line? But because the section: 'section-script-step-6' was not closed this means the error line should be collapsible but due to this being the final line I'm under the impression that this shouldn't be able to be collapsed.

What is the expected correct behavior?

Section ends should be provided so the frontend can collapse them, lines that do not belong to a collapsible section should not have a section property.

Output of checks

This bug happens in GitLab.com

Possible fixes