Split the MySQL/PostgreSQL tests in their own stage

Description

I'm currently trying to get tests across all the application to correctly pass, which leads to a turnaround time of ~80-100 minutes for the whole test suite on GitLab.com.

I strongly believe that we need to test for both databases, but could we do it sequentially instead of in parallel?. Here are our current test stages:

Build
 package-qa
 review-docs-deploy
Prepare
 retrieve-tests-metadata
 setup-test-env
Test
 codequality
 docs lint
 downtime_check
 gitlab:assets:compile
 gitlab_git_test
 karma
 qa:internal
 static-analysis

 # mysql related
 migration:path-mysql
 db:seed_fu-mysql
 db:migrate:reset-mysql
 db:rollback-mysql
 rspec-mysql 26
 spinach-mysql 4
 
 # pg related 
 migration:path-pg
 db:check-schema-pg
 db:migrate:reset-pg
 db:rollback-pg
 db:seed_fu-pg
 rspec-pg 26
 rspec-pg geo
 spinach-pg 4
Post-test
 coverage
 flaky-examples-check
 lint:javascript:report
Post-cleanup
 review-docs-cleanup

As every task is retried 3 times, having a code related failure will end up with 6 failed jobs. In the end, we need to fix problems in either MySQL or PostgreSQL anyways, so we could test one then the other. I would suggest PostgreSQL first as it is part of our Omnibus package.

Proposal

Create new build stages:

Test-pg (Test-pg could be part of the main Test stage too)
  # pg related 
  migration:path-pg
  db:check-schema-pg
  db:migrate:reset-pg
  db:rollback-pg
  db:seed_fu-pg
  rspec-pg 26
  rspec-pg geo
  spinach-pg 4
Test-mysql
  # mysql related
  migration:path-mysql
  db:seed_fu-mysql
  db:migrate:reset-mysql
  db:rollback-mysql
  rspec-mysql 26
  spinach-mysql 4

This way it will be easier to use the the -/jobs/:id navigator to navigate (no dupplicate per stage).

Links / references

/cc @rymai @ayufan

@rymai Is there any architectural problems that could cause this not to work (build artifacts?)

@ayufan Do you think this would affect the cycle time a lot for a green build?

Edited Jan 10, 2018 by Micaël Bergeron
Assignee Loading
Time tracking Loading