Add request concurrency test coverage

After deploying a faulty commit to the kubernetes cluster it became clear that the request concurrency was not working correctly. We were reaching request_concurrency limit with only one running job. This was never caught in unit or integration tests.