[E2E] Add sleep interval for run_git retries

What does this MR do and why?

[E2E] Add sleep interval for run_git retries

We often encounter situations where failures where git pushes via the CLI fail due to what appear to be intermittent or flaky issues At the moment we don't have any backoff time to retry these errors so instead, add a default 10 second wait prior to retrying to give a better opportunity for a retry to be successful.

As an example https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/12155880 we can see a failure here where all 3 attempts are made within a 2 second window. The added sleep_interval here should help alleviate against this type of failure.

[Dec 08 2023 08:11:55 UTC (QA Tests)] DEBUG -- Git: pwd=[/tmp/d20231208-27-7haj0h], command=[HOME="/tmp/qa-netrc-credentials/27" git push https://qa-user-7bc52c2f3ddd4c5f@staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git new_branch_6414ddbb4b81fcc5:new_branch_6414ddbb4b81fcc5 2>&1]
[Dec 08 2023 08:11:55 UTC (QA Tests)] DEBUG -- Git: output=[remote: You are not allowed to push code to this project.
fatal: unable to access 'https://staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git/': The requested URL returned error: 403], exitstatus=[128]
[Dec 08 2023 08:11:55 UTC (QA Tests)] DEBUG -- Retrying action with: max_attempts: 3; sleep_interval: 0; raise_on_failure: false; retry_on_exception: false
[Dec 08 2023 08:11:55 UTC (QA Tests)] DEBUG -- Attempt number 2
[Dec 08 2023 08:11:55 UTC (QA Tests)] DEBUG -- Git: pwd=[/tmp/d20231208-27-7haj0h], command=[HOME="/tmp/qa-netrc-credentials/27" git push https://qa-user-7bc52c2f3ddd4c5f@staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git new_branch_6414ddbb4b81fcc5:new_branch_6414ddbb4b81fcc5 2>&1]
[Dec 08 2023 08:11:56 UTC (QA Tests)] DEBUG -- Git: output=[remote: You are not allowed to push code to this project.
fatal: unable to access 'https://staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git/': The requested URL returned error: 403], exitstatus=[128]
[Dec 08 2023 08:11:56 UTC (QA Tests)] DEBUG -- Attempt number 3
[Dec 08 2023 08:11:56 UTC (QA Tests)] DEBUG -- Git: pwd=[/tmp/d20231208-27-7haj0h], command=[HOME="/tmp/qa-netrc-credentials/27" git push https://qa-user-7bc52c2f3ddd4c5f@staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git new_branch_6414ddbb4b81fcc5:new_branch_6414ddbb4b81fcc5 2>&1]
[Dec 08 2023 08:11:56 UTC (QA Tests)] DEBUG -- Git: output=[remote: You are not allowed to push code to this project.
fatal: unable to access 'https://staging.gitlab.com/gitlab-qa-sandbox-group-6/parent-group-to-test-user-access-79b0320977100b37/sub-group-to-test-user-access-de1eaeec4ae2b7b3/sub-groupd-project-to-test-user-access-f113bcc510c545d5.git/': The requested URL returned error: 403], exitstatus=[128]
[Dec 08 2023 08:11:56 UTC (QA Tests)] DEBUG -- ended retry

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.