streamline FF_SCRIPT_SECTIONS

What does this MR do?

This MR includes 3 improvements to FF_SCRIPT_SECTIONS:

1. Fix gaps in job trace logs(gitlab#341769 (closed)) caused by mismatch between the start and end of script section.

Before(raw job log, mismatched section start and end because of control flow in user script):

After(raw job log):

2. Add support for collapsed multi-line command.

Before(collapsed multi-line command can not be browsed):

After(can be browsed, collapsed by default):

3. Let script section play well with custom collapsible sections defined by users

Before(auto-generated script section causes parsing error):

After(avoid generating script section when there's a custom collapsible section):

Why was this MR needed?

Script section is introduced in !3051 (merged) (Kudos!) to enable CI users to see how long different steps of their script take, which is a long-desired feature tracked by #4835 (closed) .

However, current implementation has flaws aforementioned.

What's the best way to test this MR?

Use following .gitlab-ci.yml:

image: ubuntu

variables:
  # Format of the auto-deploy tag for auto-deploy builds.
  # https://gitlab.com/gitlab-org/release/docs/blob/master/general/deploy/auto-deploy.md#auto-deploy-tagging
  AUTO_DEPLOY_TAG_REGEX: '^[0-9]+\.[0-9]+\.[0-9]+\+[a-z0-9]{7,}$'
  FF_SCRIPT_SECTIONS: '1'
  FF_USE_NEW_BASH_EVAL_STRATEGY: '1'

collapsible_job_single:
  script:
    - echo -e "\e[0Ksection_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section"
    - echo 'this line should be hidden when collapsed'
    - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K"

collapsible_job_multiple:
  script:
    - echo -e "\e[0Ksection_start:`date +%s`:my_first_section\r\e[0KHeader of the 1st collapsible section"
    - echo 'this line should be hidden when collapsed'
    - echo -e "\e[0Ksection_start:`date +%s`:second_section\r\e[0KHeader of the 2nd collapsible section"
    - echo 'this line should be hidden when collapsed'
    - echo -e "\e[0Ksection_end:`date +%s`:second_section\r\e[0K"
    - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K"

error_case:
  script:
    - |
      if [[ $CI_COMMIT_TAG =~ $AUTO_DEPLOY_TAG_REGEX ]]; then
        echo "Setting COMPILE_ASSETS env variable for auto-deploy"
        export COMPILE_ASSETS=true
      fi
    - mkdir -p artifacts/images artifacts/container_versions
    - echo "source build-scripts/build.sh"
    - if [[ "${CI_COMMIT_TAG}" == *-ubi8 || "${CI_COMMIT_REF_NAME}" == *-ubi8 ]]; then
    -   export UBI_PIPELINE="true"
    - fi
    - if [ "${UBI_PIPELINE}" = "true" ]; then
    -   export DOCKERFILE_EXT='.ubi8'
    -   export IMAGE_TAG_EXT='-ubi8'
    -   use_assets
    - fi
    - if [ -z "${UBI_PIPELINE}" ]; then
    -   sleep 5
    -   bogus_command
    - fi
    - export DATE=$(date)

normal_case:
  script:
    - |
      if [[ $CI_COMMIT_TAG =~ $AUTO_DEPLOY_TAG_REGEX ]]; then
        echo "Setting COMPILE_ASSETS env variable for auto-deploy"
        export COMPILE_ASSETS=true
      fi
    - mkdir -p artifacts/images artifacts/container_versions
    - echo "source build-scripts/build.sh"
    - if [[ "${CI_COMMIT_TAG}" == *-ubi8 || "${CI_COMMIT_REF_NAME}" == *-ubi8 ]]; then
    -   export UBI_PIPELINE="true"
    - fi
    - if [ "${UBI_PIPELINE}" = "true" ]; then
    -   export DOCKERFILE_EXT='.ubi8'
    -   export IMAGE_TAG_EXT='-ubi8'
    -   use_assets
    - fi
    - if [ -z "${UBI_PIPELINE}" ]; then
    -   sleep 5
    - fi
    - export DATE=$(date)

FF_SCRIPT_SECTIONS kind of depends on FF_USE_NEW_BASH_EVAL_STRATEGY to utilize bash trap and the trap is useful when there's a user script error like "command not found". !2818 (comment 527753747) is a somewhat relevant discussion.

What are the relevant issue numbers?

Closes gitlab#341769 (closed) .

Unblock gitlab#335297 (closed) in the context of gitlab#337408 (comment 700748867) .