Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab FOSS
GitLab FOSS
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 0
    • Merge Requests 0
  • Requirements
    • Requirements
    • List
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • GitLab.org
  • GitLab FOSSGitLab FOSS
  • Issues
  • #63509

Closed
Open
Opened Jan 12, 2017 by Reb@xyzzy🥓Developer

Document automatic test splitting

Problem to solve

We have the parallel keyword which gives control and flexibility to set up parallel tests (or run any job in a parallel fashion), but this requires a lot of setup from a developer and sometimes the split logic just get duplicated. There are open source solutions like https://github.com/renderedtext/test-boosters/ that take this a step further by also separating the test configuration into multiple files, handling that part of the configuration for you.

Intended users

Sasha (Software Developer)

Further details

Test Boosters is an open source project for helping make test parallelization easy. Test Boosters supports RSpec, Cucumber, Minitest, ExUnit, and Go.

On the GitLab side, we recently added the parallel keyword (https://docs.gitlab.com/ee/ci/yaml/#parallel) which works as follows:

test:
  parallel: 5

We can build on top of this by showing people how to use test boosters with this feature.

Proposal

For the MVC we will add instructions to the official documentation on how to use test boosters with the parallel keyword to achieve better, easier results. The syntax would be something like the following:

test:
  parallel: 5
  script:
    - gem install semaphore_test_boosters
    - rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL

This would create parallel 5 jobs, each running rspec_booster --job n/5 (where n is the job index)

Permissions and Security

This should not impact the GitLab security model apart from what's already implemented in the parallel keyword.

Documentation

  • We should add documentation at the parallel keyword, and also include an example in the quick start guide.
  • We should add a ci.yml template that demonstrates how to do it
  • We should write a blog post about it (this is a task for the PM)

What does success look like, and how can we measure that?

Success is helping our users set up their parallel tests even easier than before. We can measure # of users who are installing the booster gem as part of their configuration.

Links / references

  • https://gitlab.com/gitlab-org/gitlab-ce/issues/3819#note_12287609
  • test-boosters: https://github.com/renderedtext/test-boosters/
  • Setup guide for test-boosters on Semaphore: https://semaphoreci.com/docs/setting-up-boosters.html
  • As discussed during Cancun CI / CD brainstorming session
  • CircleCI uses a bespoke CLI for it: https://circleci.com/docs/2.0/parallelism-faster-jobs/
Edited Jul 24, 2019 by Jason Yavorska
Assignee
Assign to
12.2
Milestone
12.2 (Past due)
Assign milestone
Time tracking
None
Due date
None
Reference: gitlab-org/gitlab-foss#63509