Backend changes for instance and group settings for foundational_agents_default_enabled

What does this MR do and why?

This change adds a new setting called "foundational agents default enabled" to control access to AI chat agents in GitLab. The setting can be configured at both the application level (for self-hosted instances) and at the top namespace level (for GitLab.com). When enabled, users get access to all available AI chat agents; when disabled, they only get access to the basic "GitLab Duo Agent". The feature includes database migrations to store the setting, updates to the admin interface to configure it, and logic to check permissions before showing different AI agents to users. This gives administrators more granular control over which AI features their users can access.

Why is this not a cascading setting

Governance checks such as this must be implemented against the root level (either namespace or application setting), so cascading properties are not useful

Migrations

Up

main: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32629
main: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: migrating
main: -- add_column(:ai_settings, :foundational_agents_default_enabled, :boolean, {:default=>true})
main:    -> 0.0032s
main: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: migrated (0.0208s)

main: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32629
ci: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32630
ci: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: migrating
ci: -- add_column(:ai_settings, :foundational_agents_default_enabled, :boolean, {:default=>true})
ci:    -> 0.0032s
ci: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: migrated (0.0087s)

ci: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32630
main: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32632
main: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: migrating ==
main: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: migrated (0.0031s)

main: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32632
ci: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32637
ci: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: migrating ==
ci: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: migrated (0.0056s)

ci: == [advisory_lock_connection] object_id: 130760, pg_backend_pid: 32637

Down

❯ bundle exec rails db:migrate:down:main VERSION=20251110220737 RAILS_ENV=development
main: == [advisory_lock_connection] object_id: 130460, pg_backend_pid: 63121
main: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: reverting
main: -- remove_column(:namespace_ai_settings, :foundational_agents_default_enabled)
main:    -> 0.0052s
main: == 20251110220737 AddFoundationalAgentsDefaultEnabledToNamespaceAiSettings: reverted (0.0249s)

main: == [advisory_lock_connection] object_id: 130460, pg_backend_pid: 63121
❯ bundle exec rails db:migrate:down:main VERSION=20251110220848 RAILS_ENV=development
main: == [advisory_lock_connection] object_id: 130460, pg_backend_pid: 63304
main: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: reverting ==
main: -- remove_column(:ai_settings, :foundational_agents_default_enabled)
main:    -> 0.0028s
main: == 20251110220848 AddFoundationalAgentsDefaultEnabledToAiSettings: reverted (0.0192s)

How to set up and validate locally

1. Setup gitlab-duo/test

2. run the following query:

curl 'http://gdk.test:8080/api/v4/groups/1000000' \
    -X 'PUT' \
    -H "Authorization: Bearer $GITLAB_TOKEN" \
    -H 'Accept: application/json, text/plain, */*' \
    -H 'Content-Type: application/json' \
    --data-raw '{"ai_settings_attributes":{"foundational_agents_default_enabled":"false"}}'

1. rails console, the following query should now be false

Group.find_by_id(1000000).foundational_agents_default_enabled

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #579734 (closed)