Assume runner registration tokens are not allowed by default

NOTE: 📢 This MR is a prerequisite for !150562 (merged), which needs to be merged on the 6th, so that it is deployed during the third window (2024-05-06 09:00 UTC to 2024-05-08 22:00 UTC) to perform a batched background migration

What does this MR do and why?

This MR changes tests that were dependent on allow_runner_registration_token being true. This is done by setting the appropriate value in namespace_setting. For simplicity, whenever we need to test with a different value, we make the change at the instance level, since the setting is hierarchical in nature.

This is needed in anticipation for the change of the default column value in !150562 (merged).

I also changed a check for register_project_runners in favor of create_runner, since that functionality should work independently of whether runner registration tokens exist or not.

🚨 NOTE: This MR is probably best reviewed commit-by-commit 🚨

Part of #413956 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

n/a

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

This is mostly a test setup change (except for the policy changes).