Persist groups configured to use an Agent

What does this MR do?

• Creates a new model to represent groups that have been configured in an Agent configuration project.
• Adds an endpoint to the internal API to allow KAS to send this information.

Process

1. A user updates their Agent configuration project, authorizing a group to use that Agent.
2. KAS sees this change, and sends the updated config to the internal API endpoint
3. The config is persisted, one record per group
4. [Future] these details will be used to determine which jobs are allowed to use the Agent CI Tunnel.

Full details can be found at https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kubernetes_ci_access.md

Notes

• There are no limits on config size, KAS has limits on total file size and enforces these before sending config to the Rails app.
• However, there is an arbitrary limit of 100 groups per agent, to keep database updates manageable. This may be adjusted in the future, but for now 100 seemed like a sensible limit.
• The validation for the JSON database column is very permissive, again because any validation will be performed by KAS prior to sending.

Indexing

There are no restrictions on which groups an agent can authorize, as long as both the agent and the group share a top level namespace. This means that in theory a user of a subgroup can guess the path of a private project in a different subgroup and add it to their agent configuration, and that group will be associated with their agent.

To prevent authorization churn as users/permissions/projects/groups are modified, permissions will instead be checked at the time of use. That is, when a CI job is created and we generate agent-related CI variables, we will not include agents that the job shouldn't be able to access. If a user were to guess the path of a private group and it was stored, there is no indication to them that this is the case - from their point of view they are just committing a file to their configuration repo.

Eventually there will be a UI to allow project/group owners to specify which agents (if any) should be usable in their pipelines.

How to setup and validate locally (strongly suggested)

Migrations

Up

== 20210809014918 AddAgentGroupAuthorizationsForeignKeys: migrating ===========
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:agent_group_authorizations)
   -> 0.0039s
-- execute("ALTER TABLE agent_group_authorizations\nADD CONSTRAINT fk_2c9f941965\nFOREIGN KEY (group_id)\nREFERENCES namespaces (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0076s
-- execute("SET statement_timeout TO 0")
   -> 0.0007s
-- execute("ALTER TABLE agent_group_authorizations VALIDATE CONSTRAINT fk_2c9f941965;")
   -> 0.0059s
-- execute("RESET statement_timeout")
   -> 0.0007s
-- transaction_open?()
   -> 0.0000s
-- foreign_keys(:agent_group_authorizations)
   -> 0.0027s
-- execute("ALTER TABLE agent_group_authorizations\nADD CONSTRAINT fk_fb70782616\nFOREIGN KEY (agent_id)\nREFERENCES cluster_agents (id)\nON DELETE CASCADE\nNOT VALID;\n")
   -> 0.0030s
-- execute("ALTER TABLE agent_group_authorizations VALIDATE CONSTRAINT fk_fb70782616;")
   -> 0.0020s
== 20210809014918 AddAgentGroupAuthorizationsForeignKeys: migrated (0.0368s) ==

Down

== 20210809014918 AddAgentGroupAuthorizationsForeignKeys: reverting ===========
-- foreign_keys(:agent_group_authorizations)
   -> 0.0109s
-- remove_foreign_key(:agent_group_authorizations, {:column=>:group_id})
   -> 0.0146s
-- foreign_keys(:agent_group_authorizations)
   -> 0.0031s
-- remove_foreign_key(:agent_group_authorizations, {:column=>:agent_id})
   -> 0.0056s
== 20210809014918 AddAgentGroupAuthorizationsForeignKeys: reverted (0.0515s) ==

== 20210809014850 CreateAgentGroupAuthorizations: reverting ===================
-- drop_table(:agent_group_authorizations, {:primary_key=>[:agent_id, :group_id]})
   -> 0.0068s
== 20210809014850 CreateAgentGroupAuthorizations: reverted (0.0111s) ==========

== 20210809014850 CreateAgentGroupAuthorizations: migrating ===================
-- create_table(:agent_group_authorizations, {:primary_key=>[:agent_id, :group_id]})
   -> 0.0203s
== 20210809014850 CreateAgentGroupAuthorizations: migrated (0.0204s) ==========

Does this MR meet the acceptance criteria?

Conformity

• I have included changelog trailers, or none are needed. (Does this MR need a changelog?)
• [-] I have added/updated documentation, or it's not needed. (Is documentation required?)
• [-] I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?)
• I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?)
• I have self-reviewed this MR per code review guidelines.
• This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines)
• I have followed the style guides.
• This change is backwards compatible across updates, or this does not apply.

Availability and Testing

• I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.)
• I have tested this MR in all supported browsers, or it's not needed.
• I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

Related to #323708 (closed)