Backport 17.0 - Introduce lock-free rescheduling for duplicate job

What does this MR do and why?

This MR introduces a concurrency fix for rescheduling deduplicated jobs. This backports the fix to 17.0.

Details on bug behaviour: #471239 (closed)

See the thread https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/team/-/issues/5372#note_2003023102 for context on customer impact.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
• The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
• This MR has a severity label assigned (if applicable).
• Set the milestone of the merge request to match the target backport branch version.
• This MR has been approved by a maintainer (only one approval is required).
• Ensure the e2e:package-and-test-ee job has either succeeded or been approved by a Software Engineer in Test.

Note to the merge request author and maintainer

If you have questions about the patch release process, please:

• Refer to the patch release runbook for engineers and maintainers for guidance.
• Ask questions on the #releases Slack channel (internal only).