Add artifact size QA check for Analyzers

Problem to solve

We don't track the size of the generated artifacts and could drastically increase it without being aware of that.

Intended users

devopssecure team members

Proposal

To make sure we don't increase these reports size unintentionally after an update, we need to track them and ensure they stay within a given threshold.

  • add an expected size + accepted variation in the QA projects
  • add an artifact size comparison logic in the QA jobs
  • make the QA fail when size increase by more than X percent or X Kb. To be determined
  • apply to the existing tests projects

Documentation

This should be added as part of our test projects documentation: https://gitlab.com/gitlab-org/security-products/tests/common#security-products-test-projects

Testing

Try to make the QA fail by generating a report artifact that gets over the threshold.

What does success look like, and how can we measure that?

QA Pipeline fails when a report size increases more than the allowed variation.

Links / references

Edited Nov 15, 2019 by Olivier Gonzalez
Assignee Loading
Time tracking Loading