We are hitting gitlab pipeline creation rate limits
Every kernel-ark MR pipeline currently triggers 23 downstream pipelines.
Gitlab.com has a rate limit of 25 pipeline creations per minute per user:
Pipeline creation requests for a project, user, or commit: 25 requests per minute
With rc1 week we find half a dozen or more automatic new config MRs opened every day. The daily script which creates these does its work all at once and all the MRs are opened within seconds of each other. This means after the first new config MR is opened, any following now always have all their downstream pipelines blocked because of the rate limit. It requires manual intervention to restart them, and we can only restart 1 MR's pipelines per minute :p.
I expect we're not going to convince gitlab.com to increase their rate limit so we need to find another solution. Can we reduce the number of downstream pipelines? Should we add sleep 60 to the script after it creates each config MR? Something else?