Fix triggering multiple children pipeline with the same artifact

What does this MR do?

This MR aims to fix the bug for issue #212373 (closed).

Note: it is behind a feature flag ci_new_artifact_file_reader (#249588 (closed))

.gitlab-ci.yml

stages:
  - generate
  - run

generate-config:
  stage: generate
  script:
    - ./generate-configs.sh
    - cat config-a.yml
    - cat config-b.yml
  artifacts:
    paths:
      - config-a.yml
      - config-b.yml

trigger-a:
  stage: run
  trigger:
    include:
      - artifact: config-a.yml
        job: generate-config

trigger-b:
  stage: run
  trigger:
    include:
      - artifact: config-b.yml
        job: generate-config

generate-configs.sh

#!/bin/bash

cat > config-a.yml <<HEREDOC
service-a-1:
  script:
    - echo 'job 1 for service A'
service-a-2:
  script:
    - echo 'job 2 for service A'
HEREDOC

cat > config-b.yml <<HEREDOC
service-b-1:
  script:
    - echo 'job 1 for service B'
service-b-2:
  script:
    - echo 'job 2 for service B'
HEREDOC

Result:

• "trigger-a" tries to reach the artifact file of "generate-config" job.
• It gets the file with an exclusive lock.
• Then opens zip and reads the content of the file.
• Sometimes, this may take time.
• While "trigger-a" does that, "trigger-b" also tries to reach the artifact file. However, it can not get the lock and raises ExclusiveLeaseTaken. (Sentry)

---

In this MR, we are adding a retry mechanism to avoid this error.

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team