Create performance and load testbed for GitLab Runner

👋 Outcome: Youtube, Presentaiton 👋

Problem

As part of &2502 we want to have some kind of benchmarks and load testing tool to validate any solution that we want to explore to replace docker-machine.

Why

We need a way to validate that the new platform/system/solution that we come up with and scale better or the same as we do in docker-machine. Without having some kind of metrics/numbers to validate if we are going in the right direction we might end up coming up with a solution that doesn't scale for customers.

What should the testbed do

• Create CPU intensive jobs
• Create Memory intensive jobs
• Create I/O intensive jobs
• Build real-world big applications and run their tests. For example, a nodejs, ruby, Go, C program
• Able to create a large number of jobs to stress test the runner fleet.
• Provide easy metrics/dashboards to monitor everything.