Possible race condition causing QA test failures | Error 500 response to GET /groups/:id because Project#ci_default_git_depth delegated to ci_cd_settings.default_git_depth, but ci_cd_settings is nil

Summary

If we make a GET request to the /groups/:id endpoint while a project is being deleted in a background process, the API can return error 500 with an error starting with Project#ci_default_git_depth delegated to ci_cd_settings.default_git_depth, but ci_cd_settings is nil

Steps to reproduce

  1. Sent a GET request for a group at the same time as ProjectDestroyWorker is deleting a project in that group that had CI/CD settings.

This could be difficult to reproduce manually. I've only reproduced it in CI in QA tests. For example, the failures in this job: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/1293779900

QA failure details

Full description

Create push after setting the file size limit via admin/application_settings push successful when the file size is under the limit

File path

./qa/specs/features/browser_ui/3_create/repository/push_over_http_file_size_spec.rb

Stack trace

Failure/Error:
  @project = Resource::Project.fabricate_via_api! do |p|
    p.name = 'project-test-push-limit'
    p.initialize_with_readme = true
  end

QA::Resource::ApiFabricator::InternalServerError:
  Failed to GET http://gitlab-ee-9ed174fb.test/api/v4/groups/gitlab-qa-sandbox-group?private_token=[****] - (500): `{"message":"500 Internal Server Error"}`.

First happened in https://gitlab.com/gitlab-org/gitlab-qa-mirror/-/jobs/1028364509.

Edited by Mark Lapierre