Capture running builds on non-shared runners

What does this MR do and why?

This MR expands the usage of Ci::RunningBuild.upsert_shared_runner_build! to capture all running builds, not only the ones running on shared runners. This will be needed for #437849 (closed).

I've added a FF to derisk .com deployment. Once we confirm that no adverse effects occurred from creating/destroying the additional Ci::RunningBuild instances, we can remove this FF (patch in rollout issue).

Looking at the production database, we're creating 1.1M records on a typical day. With this change, that will increase to almost 5M.

Changelog: changed

Closes #437846 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

n/a

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

The tests already cover the changes performed. However, this can be tested by following the steps below:

1. Enable the FF in the GDK Rails console:

   Feature.enable(:add_all_ci_running_builds)
   Feature.enable(:remove_all_ci_running_builds)

2. Select a project in your GDK.

3. Create and register a project runner in that project with the tags: shell, gdk.

4. Edit the .gitlab-ci.yml to include the following content:

   default:
     tags:
       - shell
       - gdk
   
   build1:
     stage: build
     script:
       - echo "Do your build here"
       - sleep 300

5. Commit and ensure a pipeline is run.

6. On the GDK Rails console, confirm that the Ci::RunningBuild was created:

   > Ci::RunningBuild.where.not(runner_type: :instance_type).exists?
     Ci::RunningBuild Exists? (0.4ms)  SELECT 1 AS one FROM "ci_running_builds" WHERE "ci_running_builds"."runner_type" != 1 LIMIT 1 /*application:console,db_config_name:ci,console_hostname:gitlab-macbookpro.work.pombei.ro,console_username:pedropombeiro,line:(pry):12:in `__pry__'*/
   => true

added Category:Fleet Visibility devopsverify grouprunner sectionci labels

assigned to @pedropombeiro

changed the description

changed milestone to %16.11

added GitLab Ultimate backend direction documentation featureaddition typefeature labels

	3 Warnings
	c344f6f8: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines.
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.
	The master pipeline status page reported failures in setup-test-env-fips setup-test-env If these jobs fail in your merge request with the same errors, then they are not caused by your changes. Please check for any on-going incidents in the incident issue tracker or in the #master-broken Slack channel.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@janis (UTC+2, same timezone as author)	@mhamda (UTC+2, same timezone as author)
database	@bauerdominic (UTC+2, same timezone as author)	@mattkasa (UTC-7, 9 hours behind author)
~"Verify"	Reviewer review is optional for ~"Verify"	@grzesiek (UTC+2, same timezone as author)

Please check reviewer's status!

• Reviewer is available!
• Reviewer is unavailable!

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

added 1 commit

• b5568201 - Capture running builds on non-shared runners

Compare with previous version

removed GitLab Ultimate label

added 108 commits

• b5568201...ac9d4605 - 106 commits from branch master
• b7ee549b - Capture running builds on non-shared runners
• f190bfd3 - Add add_all_ci_pending_builds feature flag

Compare with previous version

added feature flag feature flagexists labels

changed the description

added 1 commit

• 79f0c7ac - Add add_all_ci_pending_builds feature flag

Compare with previous version

Hey @dbiryukov , mind doing the backend initial review?

requested review from @dbiryukov

Hey @bauerdominic , even though this MR doesn't introduce new database queries, I'd like to get your opinion on the increased pressure due to the volume of new record additions/deletes

requested review from @bauerdominic

added database label

changed the description

added databasereview pending label