Refactor Security Report Schemas to use common and typed object definitions

Note

Many of the concepts of this MR have been merged into the Restructure Security Reports proposal.

The MR Draft: Adds generic security report schema demonstrates the generic security report concept, including many of the changes from this MR (specifically the typed details field).

Summary

This MR refactors the security report schema definitions to use common and potentially typed object definitions.

From @cam_swords !32 (comment 365288675)

• Ensure that you link to the new test in test/test.sh
• Check the version is not already released in verify-versions.sh
• Edit the release utils script to include a link to the fuzz schema
• I'd also recommend creating a new minor major version in the Changelog, for example 3.0.0. You will need to update the Secure Report Format to the same version.

Architectural Support

Reminder: 72-hour SLA

Scope checklist

• Does not involve architectural decisions
• Is after-the-fact
• Is not already covered by architecture guidelines/handbook
• Has a broad impact within #secure
• Is a new unit of work
• Is strictly #secure
• Could not come to an agreement (escalation)
• Involves architectural decisions

== in-scope

Reviewed by

• @d0c-s4vage
• @cam_swords
• @theoretick
• @fcatteau
• [-] @markrian (outside my expertise)

Description

Copied from !32 (diffs, comment 374541130)

The refactoring in this MR is centered around putting common definitions into a single file (common.json) so that they would not have to be redefined in each of the specific report schemas. This makes sense to me because it would prevent drift between redefined field types and make maintenance and extendability simpler.

For example, the location field in the secret detection schema and the location field in the sast schema are an example of nearly-identical definitions where drift could occur.

The largest change has to do with the location field using a oneOf to allow multiple location types. I think this will help with extendability in the future. For example, suppose a DAST tool is able to also detect a file location of a vulnerability. Having the DAST schema use the generic location definition instead of the specific endpoint_location would allow this.

The SAST schema also uses the generic location definition. Currently SAST tools operate on source code. However, SAST tools also exist that operate on compiled binaries. Locations of identified vulnerabilities within compiled code would use the module_location type. SAST tools could also potentially identify the endpoint that would be used to reach identified vulnerabilities.

Additional location types could be added in the future that would be applicable to multiple scanners. DAST, IAST, and SAST tools (or other types?) could each detect configuration vulnerabilities in infrastructure. This could be identified as file location types in the infrastructure-as-code contained in the project, or they could also be a new location type that identifies nodes or services within a network.

Main Concern

I'm expecting the main concern of this MR to be that additional, optional fields are added to definitions that were previously simpler and that only included the fields required for that specific report schema.

I think the benefits of having common definitions outweighs the downside of having unused, optional fields.

/cc @ypats @stkerr @sethgitlab