visual_gitlab_integration fails to detect inconsistencies

With !1258 (merged) merged we fixed the order of including CSS, which seems to reproduce the real state of components. A lot of our stories now render differently with and without Include GitLab CSS bundle checkbox, for example https://gitlab-org.gitlab.io/gitlab-ui/?path=/story/base-alert--default

Since alert component is marked as followsDesignSystem: true - we should have 🔴 broken master, but this does not happen, which means we do not capture these differences properly.

I've made a quick hack to beforeScreenshot and added

  return page.addStyleTag({ url: 'https://gitlab-org.gitlab.io/gitlab/application.css' });

and immediately got 72 broken snapshots, which sounds reasonable for me.

I do not like this approach since it relies on requesting a network resource multiple times, which definitely the thing we do not want on CI. Additionally that makes our build non-reproducible when our master is updated. As a proper approach, I think of downloading CSS and including it from local file multiple times and having it as artifact.

So, the strategy for fixing this issue is:

  • Add step of downloading CSS to the local file and including it in beforeScreenshot
  • Removing followsDesignSystem from all broken components, so we have 💚 green pipeline again
  • Creating follow-up issues for these components

@markrian in slack message (internal) mentioned that this approach is not 100% perfect

Since GitLab's CSS includes GitLab UI's component styles, we'd need to be careful to bundle the version of the file from GitLab which consumes the current GitLab UI version... Which is a dependency cycle 😬

Theoretically we can add manual job to main repository and trigger it just to build CSS there and grab artifact, but this will greatly increase complexity of our build pipeline

Edited by Illya Klymov