Missing documentation for gl-code-quality-report.json

In https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html is described how to implement code quality checks. This documentation is very minimal.

Existing Docs

Basically the documentation states that:

  • There should be a job named code_quality.
  • This job should output gl-code-quality-report.json as an artifact.

Furthermore it is linking to an example using Code Climate.

Real Use Case

I am developing a web app. This means I use JavaScript, and of course ESLint for linting. I have an .eslintrc.js and .eslintignore to configure ESLint, and a yarn.lock to lock my dependencies. This is great, as this is all understood by all common tooling: The ESLint CLI, VSCode integration, Atom integration, etc., and of course a convenient eslint npm script has been made.

The GitLab job looks like this:

eslint:
  stage: test
  image: node:10
  script:
    - yarn --frozen-lockfile
    - yarn eslint

What isn’t integrated, is GitLab code quality.

Code Quality Report Format

Reverse engineering code quality reports on gitlab-org/gitlab-ce>, it appears the following JSON structure is expected, based on an entry in one of the coverage reports in this project:

[
  {
    // List of strings? Enums? Unused?
    "categories": ["Complexity"],
    // The linter rule name?
    "check_name": "method_complexity",
    "content": {
      // The full body of the file. Unused?
      "body": "# Cognitive Complexity\nCognitive Complexity is a measure of how difficult a unit of code is to intuitively understand. Unlike Cyclomatic Complexity, which determines how difficult your code will be to test, Cognitive Complexity tells you how difficult your code will be to read and comprehend.\n\n### A method's cognitive complexity is based on a few simple rules:\n* Code is not considered more complex when it uses shorthand that the language provides for collapsing multiple statements into one\n* Code is considered more complex for each \"break in the linear flow of the code\"\n* Code is considered more complex when \"flow breaking structures are nested\"\n\n### Further reading\n* [Cognitive Complexity docs](https://docs.codeclimate.com/v1.0/docs/cognitive-complexity)\n* [Cognitive Complexity: A new way of measuring understandability](https://www.sonarsource.com/docs/CognitiveComplexity.pdf)\n"
    },
    // The linter message, displayed in the code quality report.
    "description": "Function `simulateDrag` has a Cognitive Complexity of 9 (exceeds 5 allowed). Consider refactoring.",
    // Code Climate internals? Unused?
    "fingerprint": "a995a617b0ce0599b6d0a5649a308a8e",
    "location": {
      // Shown in the code quality report
      "path": "app/assets/javascripts/test_utils/simulate_drag.js",
      // Shown in the code quality report
      "lines": {
        "begin": 75,
        "end": 137
      }
    },
    // No clue
    "other_locations": [],
    // No clue
    "remediation_points": 550000,
    // A string, probably an enum. Is it used?
    "severity": "minor",
    // No clue
    "type": "issue",
    // This is probably where it should say fill in `eslint`, `stylelint`, `prettier`, etc., right?
    "engine_name": "structure"
  }
]

Some fields are alien to be, but I’m also missing some data I know exists in ESLint reports.

  • Begin and start columns.
  • A link to the violated rule.

I would love to create a GitLab CI reporter for ESLint. For this I need to know what to output exactly.

Edited by Remco Haszing