<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "type::bug" label: - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab/issues?label_name%5B%5D=type::bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary <!-- Summarize the bug encountered concisely. --> There is a [test case](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/browser_ui/4_verify/pipeline/pipeline_with_image_pull_policy_spec.rb#L94) where we set in runner configuration only to allow `never` policy with `allowed_pull_policies = [never]` . Then we would try to apply `pull_policy: [always]` in CI configuration with expectation that the job would fail with indication that this policy is not allowed. Most of the time the job would have no problem trying to pull `runner-helper` image.  But sometimes it would failed at trying to pull `runner-helper` image due to the policy restriction  ### Steps to reproduce <!-- Describe how one can reproduce the issue - this is very important. Please use an ordered list. --> Since this is a transient behavior, the chance of reproducing is low. 1. Register a runner (might need to enable [FF](https://gitlab.com/gitlab-org/gitlab/-/issues/363186) `:ci_docker_image_pull_policy` if testing happens before the flag is removed) 1. Update runner's configuration to only allow `never` policy in `/etc/gitlab-runner/config.toml` 2. Create CI yml with `pull_policy: [always]` 3. Visit the pipeline and test job (the job should fail) and observe the error message in job log. ### Example Project <!-- If possible, please create an example project here on GitLab.com that exhibits the problematic behavior, and link to it here in the bug report. If you are using an older version of GitLab, this will also determine whether the bug is fixed in a more recent version. --> ### What is the current *bug* behavior? <!-- Describe what actually happens. --> Sometimes the job had no problem pulling `runner-helper` image, sometimes the pulling attempt was restricted. ### What is the expected *correct* behavior? <!-- Describe what you should see instead. --> I'm not sure, one scenario needs to be decided and should happens consistently - on going discussion https://gitlab.slack.com/archives/CBQ76ND6W/p1661415704071729 ### Output of checks <!-- If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com --> #### Results of GitLab environment info <!-- Input any relevant GitLab environment information if needed. --> <details> <summary>Expand for output related to GitLab environment info</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`) </pre> </details> #### Results of GitLab application Check <!-- Input any relevant GitLab application check information if needed. --> <details> <summary>Expand for output related to the GitLab application check</summary> <pre> (For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:check SANITIZE=true`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`) (we will only investigate if the tests are passing) </pre> </details> ### Possible fixes <!-- If you can, link to the line of code that might be responsible for the problem. -->