Skip to content

Save `finding_data` for security findings

Mehmet Emin INAC requested to merge 376213_save_finding_data into master

What does this MR do and why?

With this MR, we will start ingesting all the data(except the remediations) for report findings into the database. This way, we will no longer need to parse all the JSON report artifacts for each HTTP request to generate the response of a collection of vulnerability findings.

Remediations are not ingested because they are base64 encoded git diffs and we don't know how big a remediation can be, nor we can truncate it as truncation breaks the patch, therefore, we just store their byte offsets for each vulnerability they are associated with and later read the JSON report artifact partially to get the data.

If this byte offset approach succeeds, we can later store only the byte offsets of vulnerability objects and follow the same approach as remediations to reduce the amount of data we store in the database(basically a platform-agnostic object file storage which stores the indices on PG).

The oj-introspect gem repository can be found here: https://github.com/meinac/oj-introspect

Related to Save the data of report finding in Security::St... (#376213 - closed).

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Mehmet Emin INAC

Merge request reports