Create logic for security report schema validation

Why are we doing this work

Relevant links

• Enforce JSON schema validation for generic security report
• Previous discussion on the topic

Non-functional requirements

• Documentation:
• Feature flag:
• Performance:
• Testing:

Behaviour

This is how the security report schema to be used is determined.

Scenario: report contains malformed JSON

• GIVEN a validatable artifact
  • WHEN the file cannot be parsed as valid JSON
  • THEN the security report is not ingested
  • AND the error information is stored

Scenario: report contains wellformed JSON

• GIVEN a validatable artifact
  • WHEN the content of the report artifact does not comply with the corresponding report schema
  • THEN the security report is not ingested
  • AND the error information is stored

Scenario: report contains wellformed JSON

• GIVEN a validatable artifact
  • WHEN the content of the report artifact complies with the corresponding report schema
  • THEN the security report is ingested

Implementation plan

• Copy the report schemas from https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/tree/master/dist to Gitlab

• Create a service class to validate the Artifact content with the JSONSchemer gem

• Update the Gitlab::Ci::Parsers::Security::Common to accept a new attribute called validate default to false. If this attribute is true, validate the JSON before doing anything on it and add the error information to report entity by using Gitlab::Ci::Reports::Security::Report#error=.

• As we need to validate the artifact content just once, extend the interface of the Ci::JobArtifact#security_report method with a flag called validate. And pass this to fabricate! method.

  def security_report(validate: false)
    ...
    ::Gitlab::Ci::Parsers.fabricate!(file_type, blob, report, validate: validate).parse!
    ...
  end

• Update the Security::StoreGroupedScansService to pass validate argument as true for the jobs subject to validation(see #284062 (closed)).

• Update the Security::StoreScanService to set info by using the error attribute of the report. Store the error message in the following format: { errors: ['', '', '', ...] }.