Reduce resource contention on picking CI builds during `jobs/request` [RUN ALL RSPEC] [RUN AS-IF-FOSS]

Merged Kamil Trzciński | OoO 2022.07.23 till 2022.08.20 requested to merge ci_temporary_lock into master

What does this MR do?

Based on !55194 (merged). It will automatically be retargeted on master upon merge.

Introduce a temporary lock on a resource allowing us to quickly skip build from the queue instead of performing an expensive database optimistic lock status transition

This should reduce a rate of runners competing for the same resource, but still allowing them to recover right after.

Related to:

I wonder, how impact it will have on Redis calls. It should be pretty cheap.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Grzegorz Bizon