Fix flakey time-sensitive remove deploy key spec

What does this MR do?

Fix flakey time-sensitive remove deploy key spec

Found while working on https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6051, failing test: https://gitlab.com/gitlab-org/gitlab-ee/-/jobs/74649300

Are there points in the code the reviewer needs to double check?

Does this MR meet the acceptance criteria?

• Changelog entry added, if necessary
  • Not a user-facing change
• Tests added for this feature/bug
• Conform by the code review guidelines
  • Has been reviewed by a Backend maintainer
• Conform by the merge request performance guides
• Conform by the style guides
• If you have multiple commits, please combine them into a few logically organized commits by squashing them
• Internationalization required/considered
• End-to-end tests pass (package-and-qa manual pipeline job)

What are the relevant issue numbers?