<!-- This issue template can be used a great starting point for feature requests. The last section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. --> ## Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ## Problem to solve As a Platform Engineer, I want to set up an agent to be accessible for deployments of specific project/groups inside my company namespace, so that I won't block developers on deploying new projects. ## Intended users * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) ## User experience goal The Platform Engineer and the Application Operator should be able to use a single agent connection that is managed by the Platform Engineer and customized in a restricted way by the Application Operator. ## Proposal The current proposal is mostly for the GitOps workflow and Helm support. Sharing the access in this way for [CI/CD is not finalized yet](https://gitlab.com/gitlab-org/gitlab/-/issues/283885#note_1042893991) due to performance constraints about the lookup of available agents. From the technical discovery meeting, we have discussed the following: ### Authorization :lock: #### For manifest projects (1st iteration) ```yaml gitops: manifest_projects: - id: third-group/other-project paths: - glob: '/manifests/**/*.yaml' ``` The agent config will be able to specify `gitops.manifest_projects` that are: 1. Private. 1. Outside of their group hierarchy. 1. And even outside of their root level namespace. 1. Any combination of the above, but respecting the required licenses. For SaaS, **number 3** should be disabled. For self-managed, we're still discussing if **number 3** should be behind an instance setting, and whether it should be **CORE** or **ULTIMATE** It has been noted that **number 3** might impact FOSS projects that run on self-managed in a SaaS-like way. ##### The access request (AR) workflow process When the agent config gets parsed, it should: * Identify if the manifest project is allowed to access request (AR) given the previous notes (license, root level namespace, etc.) * if not, we haven't discussed, but I believe we simply ignore the config. A second iteration where we notify the user somehow in the UI would be good. * if yes, then an access request is created from the agent config project to the manifest project. * (1st iteration) available in ~"GitLab Premium", the access will be pending until the manifest project accepts it. This may require to be behind a setting since this might penalize PREMIUM users that simply want to auto-accept and should also be able to opt into it. * (2nd iteration) AR is auto-approved for ~"GitLab Free". ##### Performance considerations With this AR implementation for GitOps we don't expect to have the scalability concerns that we have with CI/CD Tunnel because: * when the agent requests to sync the manifests, we know exactly which project and agent we're talking about. So it's just a matter of looking up their IDs in the AR table. * we can validate access requests early in the process when we parse the config. So to check whether the project is within the expected root level namespace hierarchy, which is necessary for SaaS, and is very cheap. #### For Groups (2nd iteration) **This needs more refinement and validation**, so it will be a second iteration. But the idea so far, is that one could specify something like: ```yaml gitops: manifest_groups: - id: gitlab-org/configure project_filter: *_deployment paths: - glob: '/manifests/**/*.yaml' ``` This raises some questions: * Who approves/revokes the AR? Just the group? Every group, sub-group and projects? Just every project? * Where do we put the UI to manage ARs? * If generating ARs for every project, could this be a vector of security since one could easily create thousands of ARs by adding just one agent configuration for a root level group? Maybe not if we're respecting the root level which belongs to the company, but still. * How well does this scale? ##### For paths/globs (2nd iteration or discard it?) We've discussed that: 1. Whenever the `paths.glob` config changes for an already approved agent, the access should be revoked and a new request cycle should start, so the manifest project can decide which globs it wants to share with the agent. 1. Another approach could be to invert the responsibilities. Nowadays, the agent defines which globs the manifests are expected to be, but we could make the manifest projects specify which globs they allow the agent to use instead. 1. Although, if the manifest projects are already giving access to a certain agent project to deploy their application, why do the manifest project cares about hiding one or another folder from this agent? So maybe it's just not worth to implement path/glob authorization, and not worth to revoke the AR when the globs change. 1. Still, for flexibility purposes, the manifest project might want to choose which folders to save its manifests, but this is not a question of authorization, just another feature. Anyway, we don't think that our proposal should be blocked by this, and we can start without it. ### Alerts about authorization (2nd iteration) :bell: The discussion is about whether we should send emails to project/group owners when ARs arrive. For CORE (auto-accept), this is less of a big deal, since the expectation is that project maintainers would be talking to each other in their org, through issues, MRs or other, to guarantee that they have access, and there would be an UI to manage ARs. For Premium, the e-mails might make more sense, as the project maintainers could want to be notified since they have to take action regarding accepting/rejecting the AR. Still, we can do this on the 2nd iteration, as it does not seem to be a hard requirement for the feature. ## Implementation process ### 1st batch: Core functionality * [Store GitOps configuration](https://gitlab.com/gitlab-org/gitlab/-/issues/373132) * [GraphQL API](https://gitlab.com/gitlab-org/gitlab/-/issues/373133) * [Update access check](https://gitlab.com/gitlab-org/gitlab/-/issues/355602) We should have a production-ready, documented feature at this stage, without UI. The feature can be announced as a secondary release post. ### 2nd batch: UI * [Project settings UI](https://gitlab.com/gitlab-org/gitlab/-/issues/355607) We should continue the work at least until the 2nd batch is done. The plan is to have a primary/top feature announcement at this point and make the feature GA. ### Xth batch: Other * [Instance level cross-hierarchy setting](https://gitlab.com/gitlab-org/gitlab/-/issues/373136) ### Yth batch: Group level GitOps We need more refinement ## Further details - We can further improve this feature by implementing https://gitlab.com/gitlab-org/gitlab/-/issues/337904 in a following iteration - Not in scope for this issue is extended authorization at the group level https://gitlab.com/gitlab-org/gitlab/-/issues/354514+ ## Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [x] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members No new permission models are introduced. Both sides, parent and child, need to be configured manually without revealing details about themselves to the other side. The agents are not exposed for discoverability before enabled. Once a parent project sets the `extensible` flag, the project is listed on the child project/group's agent list view, as it happens [for CI/CD tunnels](https://gitlab.com/groups/gitlab-org/-/epics/7292). ## Documentation See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html ## Availability & Testing This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning ## What does success look like, and how can we measure that? - monthly active users around all the projects that have an agent configuration used, both child and parent projects