Add artifact size QA check for Analyzers
Problem to solve
We don't track the size of the generated artifacts and could drastically increase it without being aware of that.
Intended users
devopssecure team members
Proposal
To make sure we don't increase these reports size unintentionally after an update, we need to track them and ensure they stay within a given threshold.
-
add an expected size + accepted variation in the QA projects -
add an artifact size comparison logic in the QA jobs -
make the QA fail when size increase by more than X
percent orX
Kb. To be determined -
apply to the existing tests projects
Documentation
This should be added as part of our test projects documentation: https://gitlab.com/gitlab-org/security-products/tests/common#security-products-test-projects
Testing
Try to make the QA fail by generating a report artifact that gets over the threshold.
What does success look like, and how can we measure that?
QA Pipeline fails when a report size increases more than the allowed variation.
Links / references
Edited by Olivier Gonzalez