Split the MySQL/PostgreSQL tests in their own stage
Description
I'm currently trying to get tests across all the application to correctly pass, which leads to a turnaround time of ~80-100 minutes for the whole test suite on GitLab.com.
I strongly believe that we need to test for both databases, but could we do it sequentially instead of in parallel?. Here are our current test stages:
Build
package-qa
review-docs-deploy
Prepare
retrieve-tests-metadata
setup-test-env
Test
codequality
docs lint
downtime_check
gitlab:assets:compile
gitlab_git_test
karma
qa:internal
static-analysis
# mysql related
migration:path-mysql
db:seed_fu-mysql
db:migrate:reset-mysql
db:rollback-mysql
rspec-mysql 26
spinach-mysql 4
# pg related
migration:path-pg
db:check-schema-pg
db:migrate:reset-pg
db:rollback-pg
db:seed_fu-pg
rspec-pg 26
rspec-pg geo
spinach-pg 4
Post-test
coverage
flaky-examples-check
lint:javascript:report
Post-cleanup
review-docs-cleanup
As every task is retried 3 times, having a code related failure will end up with 6 failed jobs. In the end, we need to fix problems in either MySQL or PostgreSQL anyways, so we could test one then the other. I would suggest PostgreSQL first as it is part of our Omnibus package.
Proposal
Create new build stages:
Test-pg (Test-pg could be part of the main Test stage too)
# pg related
migration:path-pg
db:check-schema-pg
db:migrate:reset-pg
db:rollback-pg
db:seed_fu-pg
rspec-pg 26
rspec-pg geo
spinach-pg 4
Test-mysql
# mysql related
migration:path-mysql
db:seed_fu-mysql
db:migrate:reset-mysql
db:rollback-mysql
rspec-mysql 26
spinach-mysql 4
This way it will be easier to use the the -/jobs/:id
navigator to navigate (no dupplicate per stage).
Links / references
@rymai Is there any architectural problems that could cause this not to work (build artifacts?)
@ayufan Do you think this would affect the cycle time a lot for a green build?