Enable pipeline creation limits

In gitlab-foss#51401 (closed) we added support for job_activity_limit as a limitation for the pipeline creation. The job_activity_limit will drop a pipeline if the number of jobs in alive pipelines exceeds the specified limit.

Similar limits we have support for are pipeline_size_limit and pipeline_activity_limit.

Problem

Although the basic implementation for these limits is in place we haven't enabled them yet. In order to do that we would need to run the following in a production Rails console, given that they can only be set at DB level:

# iterate that for each plan
Plan::ALL_HOSTED_PLANS.each do |plan_name|
  Plan.find_by_name(plan_name).update!(active_jobs_limit: 1000)
end

Or set specific (e.g. progressive) limits per plan.

However, there are a couple of issues to be addressed in order to enable these limits on production:

• it's currently not possible to set limits to free plan because it does not exist as a record in the DB and in the code a nil plan is considered free.
• we have not tracking/logging on when a limit is hit and drops a pipeline. Only SELECT COUNT(*) FROM ci_pipelines WHERE failure_reason = 22;

Proposal

• Add logging when a pipeline is dropped because of an exceeded limit (use Prometheus metrics if possible) !19155 (merged)
• Force free plan to be a record in the DB and default to it when namespace.actual_plan is nil !19033 (merged)
• Seed a Plan.create!(name: 'free', title: 'Free') record on production !19033 (merged)
• Extract limits out of namespaces table and into a dedicated plan_limits table !19438 (merged)
• Enable active_jobs_limit: 500, especially those in the lower plans: E.g. free, bronze, silver https://gitlab.com/gitlab-com/gl-infra/production/issues/1106