Resolve "Add a test to check if files should be regenerated"

Example run https://gitlab.com/gitlab-org/quality/triage-ops/-/jobs/837398769

Based on changes from !679 (merged) (already discarded in this merge request)

Action items

• (If applicable) Add documentation to the handbook pages for Triage Operations =>
• (If applicable) Identify the affected groups and how to communicate to them:
  • /cc @person_or_group =>
  • Relevant Slack channels =>
  • Week-in-review

Closes #606 (closed)