Improve Jest spec sharding/parallelisation/sequencing in CI

As discussed in !150196 (comment 1880916097), we currently distribute Jest tests to parallel jobs in a round-robin fashion. This means that related test files are spread across separate jobs, and unrelated test files are run in the same job.

This probably worsens the resource usage of Jest jobs, both in terms of memory (Jest/V8 leaks) and compute (transpiling common source files).

Proposals

  • Distribute sequentially, e.g., something like listOfSpecs.slice(n, n + specsPerJob) rather than using index % this.ciNodeTotal === this.ciNodeIndex - 1.
  • Handle paths such that ee/analytics/devops_reports and ~/analytics/devops_reports are run in the same job.
Edited by Mark Florian