Allow instance level runners to be restricted to certain plan types
This is a dummy proof of concept for how we could allow to restrict some of the runners to certain plan types.
With this change we will be able to define a list of plans that a specific instance runner will support. When defined, only projects belonging to namespace with matching plan will be able to execute jobs on such runner. By default plans are not defined and any project may use the runner.
All other "filtering" methods - like tag matching, "protected" etc. - are still respected as they were.
As this is a proof of concept, the MR currently doesn't add any way of editing the allowed_plans
field. Neither by API nor UI. For our GitLab.com tests (which is the main target of this feature as multiple plans per instance is a SaaS thing only) we will define them through the Rails console.
Database query plans
Query for namespace through project
https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/9512/commands/33759
SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id",
"namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description",
"namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level",
"namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error",
"namespaces"."ldap_sync_last_update_at",
"namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at",
"namespaces"."description_html", "namespaces"."lfs_enabled",
"namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit",
"namespaces"."require_two_factor_authentication",
"namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version",
"namespaces"."project_creation_level", "namespaces"."runners_token",
"namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted",
"namespaces"."custom_project_templates_group_id",
"namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit",
"namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level",
"namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size",
"namespaces"."max_artifacts_size",
"namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap",
"namespaces"."max_personal_access_token_lifetime",
"namespaces"."push_rule_id", "namespaces"."shared_runners_enabled",
"namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids"
FROM "namespaces"
INNER JOIN "projects" ON "namespaces"."id" = "projects"."namespace_id"
WHERE "projects"."id" = 20087204
LIMIT 1
Limit (cost=1.13..7.16 rows=1 width=363) (actual time=1.037..1.038 rows=1 loops=1)
Buffers: shared hit=2 read=9 dirtied=2
I/O Timings: read=0.947 write=0.000
-> Nested Loop (cost=1.13..7.16 rows=1 width=363) (actual time=1.035..1.036 rows=1 loops=1)
Buffers: shared hit=2 read=9 dirtied=2
I/O Timings: read=0.947 write=0.000
-> Index Scan using idx_projects_on_repository_storage_last_repository_updated_at on public.projects (cost=0.56..3.58 rows=1 width=4) (actual time=0.624..0.625 rows=1 loops=1)
Index Cond: (projects.id = 20087204)
Buffers: shared read=6 dirtied=2
I/O Timings: read=0.566 write=0.000
-> Index Scan using namespaces_pkey on public.namespaces (cost=0.56..3.58 rows=1 width=363) (actual time=0.405..0.405 rows=1 loops=1)
Index Cond: (namespaces.id = projects.namespace_id)
Buffers: shared hit=2 read=3
I/O Timings: read=0.381 write=0.000
Fetch ancestor namespaces
https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/9512/commands/33760
WITH RECURSIVE "base_and_ancestors" AS ((
SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id",
"namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description",
"namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level",
"namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error",
"namespaces"."ldap_sync_last_update_at",
"namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at",
"namespaces"."description_html", "namespaces"."lfs_enabled",
"namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit",
"namespaces"."require_two_factor_authentication",
"namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version",
"namespaces"."project_creation_level", "namespaces"."runners_token",
"namespaces"."file_template_project_id", "namespaces"."saml_discovery_token",
"namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id",
"namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit",
"namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level",
"namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size",
"namespaces"."max_artifacts_size",
"namespaces"."mentions_disabled", "namespaces"."default_branch_protection",
"namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime",
"namespaces"."push_rule_id", "namespaces"."shared_runners_enabled",
"namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids"
FROM "namespaces"
WHERE "namespaces"."type" = 'Group'
AND "namespaces"."id" = 7582552)
UNION (
SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id",
"namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description",
"namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level",
"namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error",
"namespaces"."ldap_sync_last_update_at",
"namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at",
"namespaces"."description_html", "namespaces"."lfs_enabled",
"namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit",
"namespaces"."require_two_factor_authentication",
"namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version",
"namespaces"."project_creation_level", "namespaces"."runners_token",
"namespaces"."file_template_project_id", "namespaces"."saml_discovery_token",
"namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id",
"namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit",
"namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level",
"namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size",
"namespaces"."max_artifacts_size",
"namespaces"."mentions_disabled", "namespaces"."default_branch_protection",
"namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime",
"namespaces"."push_rule_id", "namespaces"."shared_runners_enabled",
"namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids"
FROM "namespaces", "base_and_ancestors"
WHERE "namespaces"."type" = 'Group'
AND "namespaces"."id" = "base_and_ancestors"."parent_id"))
SELECT "id"
FROM "base_and_ancestors" AS "namespaces"
CTE Scan on base_and_ancestors namespaces (cost=364.31..364.53 rows=11 width=4) (actual time=0.470..0.866 rows=3 loops=1)
Buffers: shared hit=9 read=6
I/O Timings: read=0.751 write=0.000
CTE base_and_ancestors
-> Recursive Union (cost=0.56..364.31 rows=11 width=363) (actual time=0.466..0.858 rows=3 loops=1)
Buffers: shared hit=9 read=6
I/O Timings: read=0.751 write=0.000
-> Index Scan using index_namespaces_on_type_and_id on public.namespaces namespaces_1 (cost=0.56..3.58 rows=1 width=363) (actual time=0.457..0.458 rows=1 loops=1)
Index Cond: (((namespaces_1.type)::text = 'Group'::text) AND (namespaces_1.id = 7582552))
Buffers: shared hit=1 read=4
I/O Timings: read=0.417 write=0.000
-> Nested Loop (cost=0.56..36.05 rows=1 width=363) (actual time=0.124..0.124 rows=1 loops=3)
Buffers: shared hit=8 read=2
I/O Timings: read=0.333 write=0.000
-> WorkTable Scan on base_and_ancestors (cost=0.00..0.20 rows=10 width=4) (actual time=0.001..0.001 rows=1 loops=3)
I/O Timings: read=0.000 write=0.000
-> Index Scan using index_namespaces_on_type_and_id on public.namespaces namespaces_2 (cost=0.56..3.58 rows=1 width=363) (actual time=0.120..0.120 rows=1 loops=3)
Index Cond: (((namespaces_2.type)::text = 'Group'::text) AND (namespaces_2.id = base_and_ancestors.parent_id))
Buffers: shared hit=8 read=2
I/O Timings: read=0.333 write=0.000
Fetch plans for namespaces
https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/9512/commands/33761
SELECT DISTINCT "plans".*
FROM "plans"
INNER JOIN "gitlab_subscriptions" ON "gitlab_subscriptions"."hosted_plan_id" = "plans"."id"
WHERE "plans"."name" IN ('bronze', 'silver', 'premium', 'gold', 'ultimate', 'ultimate_trial', 'premium_trial', 'opensource')
AND "gitlab_subscriptions"."namespace_id" IN (7582552, 7582547, 7582535)
Unique (cost=12.63..12.66 rows=2 width=42) (actual time=0.489..0.493 rows=1 loops=1)
Buffers: shared hit=15 read=5
I/O Timings: read=0.349 write=0.000
-> Sort (cost=12.63..12.64 rows=2 width=42) (actual time=0.487..0.490 rows=1 loops=1)
Sort Key: plans.id, plans.created_at, plans.updated_at, plans.name, plans.title
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=15 read=5
I/O Timings: read=0.349 write=0.000
-> Hash Join (cost=4.75..12.62 rows=2 width=42) (actual time=0.428..0.441 rows=1 loops=1)
Hash Cond: (gitlab_subscriptions.hosted_plan_id = plans.id)
Buffers: shared hit=6 read=5
I/O Timings: read=0.349 write=0.000
-> Index Scan using index_gitlab_subscriptions_on_namespace_id on public.gitlab_subscriptions (cost=0.43..8.29 rows=3 width=4) (actual time=0.344..0.355 rows=1 loops=1)
Index Cond: (gitlab_subscriptions.namespace_id = ANY ('{7582552,7582547,7582535}'::integer[]))
Buffers: shared hit=6 read=4
I/O Timings: read=0.319 write=0.000
-> Hash (cost=4.22..4.22 rows=8 width=42) (actual time=0.068..0.069 rows=7 loops=1)
Buckets: 1024 Batches: 1 Memory Usage: 9kB
Buffers: shared read=1
I/O Timings: read=0.030 write=0.000
-> Seq Scan on public.plans (cost=0.00..4.22 rows=8 width=42) (actual time=0.046..0.058 rows=7 loops=1)
Filter: ((plans.name)::text = ANY ('{bronze,silver,premium,gold,"ul
timate",ultimate_trial,premium_trial,opensource}'::text[]))
Rows Removed by Filter: 4
Buffers: shared read=1
I/O Timings: read=0.030 write=0.000
Merge request reports
Activity
added [Deprecated] Category:Runner devopsverify grouprunner sectionops labels
assigned to @tmaczukin
@erushton @pedropombeiro This is my PoC for the idea of allowing separation of runners for specific GitLab plans.
1 Warning featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.
For more information, see:
- The Handbook page on merge request types.
- The definition of done documentation.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Jonathan Schafer ( @jschafer
) (UTC-5, 7 hours behind@tmaczukin
)Etienne Baqué ( @ebaque
) (UTC+3, 1 hour ahead of@tmaczukin
)database Matt Kasa ( @mattkasa
) (UTC+0, 2 hours behind@tmaczukin
)Mayra Cabrera ( @mayra-cabrera
) (UTC-5, 7 hours behind@tmaczukin
)~migration No reviewer available No maintainer available To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
Generated by
DangerEdited by Ghost Useradded featureenhancement + 1 deleted label
added typefeature label
Allure report
allure-report-publisher
generated test report!review-qa-reliable:
test report
review-qa-smoke: test report
package-and-qa-ff-enabled: test report
package-and-qa-ff-disabled: test reportreview-qa-blocking:
test report for 436ba944expand test summary
+-------------------------------------------------------------------+ | suites summary | +----------------------+--------+--------+---------+-------+--------+ | | passed | failed | skipped | flaky | result | +----------------------+--------+--------+---------+-------+--------+ | Plan | 41 | 0 | 1 | 41 | ❗ | | Manage | 31 | 0 | 2 | 29 | ❗ | | Create | 24 | 0 | 2 | 22 | ❗ | | Verify | 12 | 0 | 1 | 12 | ❗ | | Package | 0 | 0 | 1 | 0 | ➖ | | Version sanity check | 0 | 0 | 1 | 0 | ➖ | | Protect | 2 | 0 | 0 | 2 | ❗ | | Configure | 0 | 0 | 1 | 0 | ➖ | +----------------------+--------+--------+---------+-------+--------+ | Total | 110 | 0 | 9 | 106 | ❗ | +----------------------+--------+--------+---------+-------+--------+
test report for 436ba944expand test summary
+-------------------------------------------------------------------+ | suites summary | +----------------------+--------+--------+---------+-------+--------+ | | passed | failed | skipped | flaky | result | +----------------------+--------+--------+---------+-------+--------+ | Create | 148 | 1 | 6 | 7 | ❌ | | Verify | 33 | 0 | 8 | 5 | ❗ | | Plan | 53 | 0 | 0 | 1 | ❗ | | Secure | 16 | 0 | 3 | 2 | ❗ | | Manage | 91 | 0 | 6 | 2 | ❗ | | Version sanity check | 0 | 0 | 1 | 0 | ➖ | | Fulfillment | 2 | 0 | 10 | 0 | ✅ | | Configure | 0 | 0 | 3 | 0 | ➖ | | Release | 6 | 0 | 0 | 3 | ❗ | | Package | 0 | 0 | 3 | 0 | ➖ | | Non-devops | 2 | 0 | 0 | 0 | ✅ | | Protect | 2 | 0 | 0 | 0 | ✅ | +----------------------+--------+--------+---------+-------+--------+ | Total | 353 | 1 | 40 | 20 | ❌ | +----------------------+--------+--------+---------+-------+--------+
test report for 436ba944expand test summary
+----------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+--------+ | | passed | failed | skipped | flaky | result | +-------------+--------+--------+---------+-------+--------+ | Plan | 48 | 0 | 0 | 1 | ❗ | | Create | 122 | 1 | 3 | 6 | ❌ | | Verify | 28 | 0 | 6 | 4 | ❗ | | Manage | 66 | 0 | 6 | 1 | ❗ | | Release | 6 | 0 | 0 | 3 | ❗ | | Fulfillment | 2 | 0 | 8 | 0 | ✅ | | Secure | 11 | 0 | 3 | 2 | ❗ | | Package | 0 | 0 | 3 | 0 | ➖ | | Non-devops | 1 | 0 | 0 | 0 | ✅ | | Configure | 0 | 0 | 3 | 0 | ➖ | | Protect | 2 | 0 | 0 | 0 | ✅ | +-------------+--------+--------+---------+-------+--------+ | Total | 286 | 1 | 32 | 17 | ❌ | +-------------+--------+--------+---------+-------+--------+
Edited by Ghost User- Resolved by Tomasz Maczukin
This MR is a work-in-progress. It will still require few updates (+description) before it'll be ready for the first review
- Resolved by Tomasz Maczukin
- Resolved by Tomasz Maczukin
- Resolved by Tomasz Maczukin
- Resolved by Tomasz Maczukin
- Resolved by Tomasz Maczukin
- Resolved by Tomasz Maczukin