<!-- This issue template can be used a great starting point for feature requests. The last section "Release notes" can be used as a summary of the feature and is also required if you want to have your release post blog MR auto generated using the release post item generator: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator. The remaining sections are the backbone for every feature in GitLab. --> ### Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> Support for Monorepo Workflows through Trunk Based Development ### Problem to solve <!-- What is the user problem you are trying to solve with this issue? --> For the past 10 years, the standard workflow for using git was a branching model. This was easy to explain to new users as they could create staging areas for changes to share with others while maintaining a clean primary branch for builds and releases. With products supporting new collaborative features around this branching model we have become familiar with PR/MR type workflows that leverage hooks to enforce things like security, compliance and code reviews before a merge can take place. This works great when talking about applications that are developed that have loose coupling and can be built independently of other repositories. Today, we are seeing companies that want a model that is more conducive to an all-in-one repository that makes it easier for companies to build a single set of rules for collaboration, build and deploy capabilities. For many people, they call these monorepos as everything from source code to documentation to infrastructure definition is combined into a single repository. Because these repositories tend to be larger in size, a branching workflow does not always work because there could be 1000s of developers working on a single repository committing 10,000s changes per day across millions of lines of code in this repository. There are examples of this at large technology companies that are very public about this like Google, Microsoft, Netflix and Twitter to name a few. For these larger, monorepo style repositories an older paradigm is very commonly used called trunk based development. This model requires more ownership on the developer as they are expected to validate a lot of their changes locally before they make their changes public. To implement this style of development, most tools will create a staging area with patches that are submitted by developers. As patches are accepted, they are inserted into the trunk (main) repository through a series of merge trains that knows how to assemble the code changes in the right order. This model was very common back when we used SVN, CVS and ClearCase and in the OSS world with Bugzilla. I have seen this issue directly when working with the Spinnaker community as the product is made up of 12 different services that were forced to move to different repositories because vendors won't support single repositories larger than 5G. Because of this, it is difficult to search for issues, find the right location to make changes, create shared components, leverage coding best practices and simplified build scripts. It ends up becoming a dance that requires a lot of orchestration across multiple build scripts that leverage different packages as build dependencies creating fragmentation and risk around security and vulnerabilities. Because of this fragmentation, collaboration has been difficult causing the community to shrink which could happen just as easily inside an enterprise that is forced to do the same. This issue is not to debate the advantages and/or disadvantages of one model over the other, but to discuss options where we can accommodate both workflows while maintaining performance and scale that our customers expect. ### Proposal <!-- Use this section to explain the feature and how it will work. It can be helpful to add technical details, design proposals, and links to related epics or issues. --> There are a few ways we could technically accomplish this: - [ ] Partial Forks - https://gitlab.com/gitlab-org/gitlab/-/issues/342158 - [ ] Stacked Diffs - https://kurtisnusbaum.medium.com/stacked-diffs-keeping-phabricator-diffs-small-d9964f4dcfa6#.jd1kqorhb When a developer is working on a branch locally, they can push the diffs to staging area. This staging area would generate an MR for each "chain" of patches from a given developer. This keeps the object sizes in the repository small as you are only merging the patches once they are accepted rather than creating the objects up front in the repository to hold the changes. This also reduces the issue of people that create long running branches. This staging area could become very powerful when combined with our merge train concept. - [ ] Git Stash - Is there a way where we can use the `git stash` capability for a developer to publish a stash for review. You could use a Stash MR to review the changes and then apply the stash to repository. These are the three solutions I could come up with, but hope there are others in the community that have some additional solutions as well. ### Intended Users This solution would involve every Persona from https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) * [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor) --> ### Metrics <!-- How are you going to track uage of this feature? Think about user behavior and their interaction with the product. What indicates someone is getting value from it? Create tracking issue using the Snowplow event tracking template. See https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Snowplow%20event%20tracking.md --> <!--- Use the following resources to find the appropriate labels: - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ Consider adding related issues and epics to this issue. You can also reference the Feature Proposal Template (https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/issue_templates/Feature%20proposal%20-%20detailed.md) for additional details to consider adding to this issue. Additionally, as a data oriented organization, when your feature exits planning breakdown, consider adding the `What does success look like, and how can we measure that?` section. Other sections to consider adding: ### Intended users Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) ### User experience goal What is the single user experience workflow this problem addresses? For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>" https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ ### Further details Include use cases, benefits, goals, or any other details that will help us understand the problem better. ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [ ] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members ### Documentation See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/workflow.html * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html ### Availability & Testing This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning ### What does success look like, and how can we measure that? Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. ### What is the type of buyer? What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#three-tiers ### Is this a cross-stage feature? Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> cc:// @sarahwaldner @adawar @brianwald <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->