Consider failing tests if a test attempts to connect to a host other than localhost
Context
Follow-up issue for gitlab-org/quality/engineering-productivity/master-broken-incidents#412 (closed).
In gitlab-org/quality/engineering-productivity/master-broken-incidents#412 (closed)'s root cause analysis, it was found that spec/features/projects_spec.rb
was testing CSS selectors on GitLab.com. Because of the version with the new CSS selector was not deployed at that time, the spec passed in the merge request that introduced a new CSS selector.
@stanhu
opened an MR (!107292 (merged)) to avoid testing the CSS selector against GitLab.com.
In addition, @mikegreiling
suggested the following at gitlab-org/quality/engineering-productivity/master-broken-incidents#412 (comment 1213135141):
Apart from integrations involving third parties, is there any legitimate reason for our tests to be reaching out to anything other than
locahost
? As a sanity check, should we find a way to ensure that our specs have no access to the outside internet at all? And to perhaps trigger an automatic fail state if it is attempted?Perhaps we can configure headless Chrome to not have access to WAN, or set up some sort of reverse firewall?
Goal
We should consider failing tests if a test reaches out to a host other than localhost
.