Broken master due to not running the `*-as-if-foss` jobs in MRs anymore

  1. gitlab-org/gitlab#219030 (closed) => gitlab-org/gitlab!32564 (merged): System test.
  2. gitlab-org/gitlab#219073 (closed): System and controller tests.
  3. gitlab-org/gitlab#219169 (closed) => gitlab-org/gitlab!32346 (merged): Model test.
  4. gitlab-org/gitlab#219193 (closed) => gitlab-org/gitlab!32341 (merged): Lib test. This was broken because a new file was added that was in .gitignore, causing the merge to gitlab-foss process to miss this file. *-as-if-foss wouldn't have caught this.
  5. gitlab-org/gitlab#219231 (closed) => gitlab-org/gitlab!31803 (merged): System test.
  6. gitlab-org/gitlab#219313 (closed) => gitlab-org/gitlab!32641 (merged): Unit test
  7. gitlab-org/gitlab#219401 (closed) => gitlab-org/gitlab!33041 (merged): Unit test
  8. gitlab-org/gitlab#219396 (closed) => originally flaky and then discovered to be an invalid test with gitlab-org/gitlab!33306 (merged): Integration test
  9. gitlab-org/gitlab#220117 (closed) - gitlab-org/gitlab!32562 (merged) Unit test (pipeline was 17 hours old so rspec foss-impact didn't run!)

Corrective Actions

  1. From the team's agenda - tracked in gitlab-org/gitlab#219261 (closed):

    - Albert: instead of running tests in foss mode, can we do a static check on dependencies from foss to ee and run this check on MRs? Today’s failure was due to a code in FOSS that uses a constant defined in EE
- Jen-Shin: I don’t think static check is enough, because Ruby is too dynamic, but maybe it can catch 80% of issues? I don’t know if we can leverage any existing tools though. My feeling is that it’s not worth the effort to build it.
- Remy: We could start the application in *as-if-foss mode and maybe that would catch constants that aren’t available as well.

  2. https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/520#note_349943246 - tracked in gitlab-org/gitlab#219262 (closed):

    We could also try to run as-if-foss specs for the modified files, e.g. app/models/user.rb => spec/models/user_spec.rb (same for controllers, views, helpers, lib etc.).

  3. https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/520#note_349939838 - tracked in gitlab-org/gitlab#219263 (closed):

    we could also add a suggestion in danger if the changes are all in ee/ and spec/

Edited by Rémy Coutable