Provide an easy way to parse job logs for warnings

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Release notes

Problem to solve

A developer wants to know if there are warnings when their code gets compiled, and potentially get a notifications when new warnings are introduced, in order to end up with better code.

A person writing the CI jobs wants to be able to easily add support for extracting compiler hints and warnings.

Intended users

• Sasha (Software Developer)
• Devon (DevOps Engineer)
• Simone (Software Engineer in Test)

User experience goal

Proposal

It would be nice to have an easier syntax to allow extracting warnings (ideally with a regexp) without the need to explicitly pipe to files and upload artefacts. My proposal is suboptimal, but something along the lines of the following:

.compile_step: &compile_step
  stage: build
  script:
    - make
  warnings:
    - pattern: "warning: "
    - warning: True

Further details

At the moment this is possible, but utterly clumsy to do:

• The build output first needs to be redirected to a file (while ideally still printed to stdout, so one needs non-trivial syntax for piping output, esp. on Windows).
• The file then needs to be explicitly stored as an artefact, adding unnecessary clutter to the list of files that users actually want to download. (If one wants to download a zip with the program just compiled, they get this file as a collateral damage as well.)
• A special step is needed to parse the output. If we want to run three types of builds (Release, Debug, ...) for two architectures (32-bit, 64-bit) and check for compiler hints (less severe warnings) and warnings, we need 12 extra steps. That is both ugly to write and ugly to visualise.
• Unless a DAG is used, you never know which artefact will be downloaded, so each artefact from each of the different builds needs a different name.
• When using different names for the artefacts (from different build types) along with GIT_STRATEGY: none, no files are ever removed from the folder where one is grepping for warnings, and artefacts from other unrelated jobs are just kept sitting there, so cat build_output*.txt (in the hope that you might be spared the need to specify the exact names of the artefacts) would simply use files from random other pipelines. (This is worth reporting as a separate bug, if one doesn't exist yet.)

Here's an example from a convoluted yaml file to achieve that simple goal.

.compile_step: &compile_step
  stage: build
  script:
    - msbuild {here-goes-the-full-compile-line} | tee    "$env:BUILD_OUTPUT_FILE"
    - msbuild {here-goes-another-build-command} | tee -a "$env:BUILD_OUTPUT_FILE"
  artifacts:
    paths:
      - "$env:BUILD_OUTPUT_FILE"
      - "$env:BIN_DIR/software.exe"
    expire_in: 1 week

.check_for_hints_or_warnings: &check_for_hints_or_warnings
  stage: test
  script:
    # find hints or warnings from the build output
    - $warnings = cat "$env:BUILD_OUTPUT_FILE" | Select-String -Pattern $env:SEARCH_STRING | Select-Object -ExpandProperty Line
    # remove the arbitrary path prefix from filenames and sort them
    - $warnings = $warnings -replace ".*some\\prefix\\","" | Sort-Object | Get-Unique
    # display hints or warnings
    - $warnings
    # exit with an error if there were any warnings left
    - if ($warnings) { exit 1 }

# 12 times like this ...

check-hints_release32:
  needs: ["compile-release32"]
  variables:
    GIT_STRATEGY: none
    SEARCH_STRING: "Hint warning H"
    BUILD_OUTPUT_FILE: build_output_R32.txt
  <<: *check_for_hints_or_warnings

check-warnings_debug32:
  needs: ["compile-debug32"]
  variables:
    GIT_STRATEGY: none
    SEARCH_STRING: "warning W"
    BUILD_OUTPUT_FILE: build_output_D32.txt
  <<: *check_for_hints_or_warnings

Now, compare this to the way it's done in BuildBot where warnings just work out of the box, with an option to define a custom pattern, taking just one extra line of code:

f.addStep(steps.Compile(command=["make", "test"],
                        warningPattern="^Warning: ")) # optional, to specify a custom pattern (defaults work)

and some other simple configuration options like warnOnWarnings=True that will color the step orange if warnings are found, or mark the process as failed and make it red on flunkOnWarnings=True. By default the warnings are still collected separately and easy to display / extract, but the step would remain green unless requested.

This is how that looks (pretty clean and easy to navigate):

Permissions and Security

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

@thaoyeager