Always use frozen fixtures from actual data from www-gitlab-com to run tests
Proposal
Use fixed fixtures to run tests, but the fixed fixtures should use the actual data from www-gitlab-com
, and we should periodically update the fixtures to be up-to-date.
Idea from #1175 (comment 1200009215)
After we did this, we can consider adding a new job in www-gitlab-com
which can trigger a pipeline in triage-ops
to attempt to update the fixtures and see if it will break anything.
Previous description
For the tests #comment_for_intelligent_stage_and_group_labels_inference
we're attempting to use fixtures for more consistent results, but most of the time we're actually not really using those fixtures, because whenever WwwGitLabCom.groups
is called before setting the stubs from other tests, it'll memoize the actual data in instance variables and they're not reset.
This caused flaky tests: #697 (closed) because RSpec will shuffle the ordering of tests. Sometimes it's using the actual data, and sometimes it's using the stubbed fixtures.
We should make it consistently use the fixtures for the specific tests, or even across the whole test suite.