Automatically split list of tests in parallel
Problem to solve
Currently, we have the parallel which is great and gives control and flexibility to set up parallel tests but this requires a lot of setup from a developer and sometimes the split logic just get duplicated.
Knapsack does something similar and it's possible (I'm not going to say otherwise before I don't have any data on this) others are doing the same thing.
- The user creates a $JOB_NAME.txt inside of the
- When the
parallelkeyword is present for a job we check
.gitlab/$JOB_NAME.txtand split it automatically
- Each line will be a test we have to found out what is the best mean to send each test of pkg define in the line to the user, we can do this via env variables or something similar
Convention of the txt file
- It doesn't have to plain text, we can do json if we want
- Each line will be a test/package or something that the user will run
Permissions and Security
Giving an example on how to set up something like this for different language/testing frameworks
What does success look like, and how can we measure that?
Developers just have to set up the
parallel keyword and define the command that will run inside of a loop to run the tests.