Fix a number of race conditions that can occur during namespace deletion

There are two problems in the current implementation:

1. If a project is marked for deletion via the pending_delete flag and then the namespace was quickly deleted, it's possible that the namespace skips over that project and leaves that project in an orphaned state.

2. Before namespace deletion, the namespace attempts to clean up all the relevant storage paths. However, if all projects have been removed synchronously, then the namespace will not be able to clean anything. To prevent this, we should load the paths to be deleted before actually destroying projects.

The specs were missing this second case due to a permission issue that caused project removal never to happen.

@MrChrisW @dblessing FYI.

assigned to @DouweM

merged

Thanks, @DouweM. Let me know where you think this change should land: 8.16, 8.17, or 9.0? I put 8.16 since this may be affecting a customer, but open to scheduling it somewhere else.

changed milestone to %8.17

On second thought, moving to 8.17 since it is only 6 days away. Going to ping for approval from @DouweM, @jameslopez, and @dzaporozhets. :)

I'm okay with this, seems small enough and it's a bug.

Picked into 8-17-stable

removed ~149423 label

I reverted this in 8.17-stable. It won't work without https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5803 which is scheduled for 9.0

cc @stanhu @DouweM