Support multiple, automated Iteration Cadences
<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> ## Iteration Cadences ### Problems To Solve 1. There are often times when many teams share one or more projects to track Issues. Right now, there is no way to allow teams with the flexibility to manage their own Iteration cadence in this scenario. 1. Every iteration is intended to the same duration and there is an opportunity to make managing them easier through automated scheduling and rolling over issues from one iteration to the next. 1. Currently as an MVC we implemented a way to define iterations at group level and validate that created iteration dates do not overlap with other iterations from parent groups. Whereas that gives the possibility to introduce iterations concept as an MVC it does not seem that is how iterations work and has some rigidity to it vs more flexibility that iterations would suggest. ### Practical Use Cases & End Goal **View Example:** <details> Step 1: 1. Plan:Project Management wants to have two week iterations, but Plan:Product Planning wants to have one week iterations. 1. Plan:Project Management creates an iteration cadence called `Project Management` and Plan:Product Planning creates an iteration cadence called `Product Planning`. 1. Project Management manual creates individual iterations within the `Project Management` iteration cadence. 1. Issues can then be assigned to an iteration within the `Project Management` cadence. 1. PM's [Issue Board](https://gitlab.com/groups/gitlab-org/-/boards/1235826?&label_name[]=group%3A%3Aproject%20management) can be associated to the `Project Management` cadence. 1. To track work in a current iteration across workflow states, this allows the Board to be globally scoped to the current Iteration within the `Project Management` cadence. 1. This allows Plan:Project Management to create a Board [similar to this](https://gitlab.com/groups/gitlab-org/-/boards/1910149?&label_name[]=group%3A%3Aproject%20management) but with the current + upcoming Iterations as lists instead of Milestones. Step 2: 1. Up to this point, Project Management has to manually create each iteration and add the iteration as a new list to a Board. 1. Plan:Project Management configures their cadence such that each Iteration is 2 weeks and starts on Wednesday. They select to maintain 6 upcoming iterations. They select a start date for their Cadence. 1. Once they hit save, iterations automatically appear within their cadence. When they go to their issue Board (scoped to the `Project Management` cadence) with iteration lists, the lists are synchronized to their automated schedule and show the current plus upcoming iterations. Step 3 (outside the scope of this epic): 1. Based on the issues completed within previous 4 iterations within the `Project Management` cadence, GitLab calculates the velocity of the cadence to be 15 weight per iteration. 1. Velocity lines are shown within each iteration list on the board so that teams can schedule issues across iterations according to what they will likely be able to complete within each iteration. ![Screen_Shot_2021-01-05_at_1.18.30_PM](/uploads/91e6d4d7b0004680236997076c7afb13/Screen_Shot_2021-01-05_at_1.18.30_PM.png) </details> ### Proposal ### Define an Iteration Cadence (IC) - [x] A Group may define one or more ICs. - [x] A Group may inherit 1 or more ICs from parent Groups (still in discussion) - [x] Only validate non-overlapping dates within a single cadence (a set of iterations) - [x] An Iteration Cadence is named. - [ ] An IC may be disabled. If it is, it is disabled in all subgroups automatically. - [ ] Disabling an IC removes current and future Iterations from Issues currently assigned to an Iteration within that cadence. Closed issues are not automatically updated. Disabled ICs disappear from all dropdowns / removed from Boards. - [x] A Board can be scoped to a specific IC. Until this scope is set, "scope to current Iteration" checkbox is disabled. ### Automate Iteration Cadence - [x] An IC can define a start date - [x] An IC can define the duration of a single Iteration in weeks - [x] Automated IC scheduling is a user configurable option defaulted to on for new Iterations. For existing ICs, it is defaulted to Off. - [x] A configurable option to allow automatically moving Issues from a newly closed Iteration to the next. - [x] A configurable option to specify how many future iterations to complete. Consider a range of 1-12. ### UX **View PoC:** <details> ![Iteration List](https://gitlab.com/gitlab-org/gitlab/uploads/51e5dd58002b836371e12abcf680090d/Screen_Shot_2020-12-11_at_1.15.59_PM.png) ![Configure Iteration Cadence](https://gitlab.com/gitlab-org/gitlab/uploads/8ab51a068d5a77651f23b0c5a929ae2b/Screen_Shot_2020-12-11_at_1.16.43_PM.png) ![Select Iteration From Dropdown](https://gitlab.com/gitlab-org/gitlab/uploads/37f4c4f4ad61d4fe22a07f1d3504ecfc/Screen_Shot_2020-12-11_at_1.18.16_PM.png) ![Scope Board to Iteration Cadence](https://gitlab.com/gitlab-org/gitlab/uploads/93cd3b27552ceb42b4bd9b2a3fe67c5b/Screen_Shot_2020-12-11_at_1.23.41_PM.png) </details> #### List View ![Default_list](/uploads/3017a278a3acd13b7c9212a161c6ee4b/Default_list.png) ##### Vertical Slices **MVC:** - [x] I can navigate to the Iteration Cadences list from `Issues > Iterations` in the left nav sidebar. - [x] I can click a button to navigate to the "New Cadence" form - [x] I can view a list of Iteration Cadences with their respective Iterations - [x] I can toggle between `Open` and `Done` Iterations. - [x] Open iterations are ordered by date with the first being the iteration closest to the current date. - [ ] Done iterations are ordered by date with the first being the iteration that was most recently completed. - [x] Within each cadence, I can select options to `Add` an Iteration (manual mode only), `Edit`, or `Delete`. - [x] Deleting the Iteration cadence prompts me with a double confirmation modal (use Milestones as a fast follow) - [x] Selecting `edit` takes me to the Iteration Cadence Create/Edit form - [x] Selecting `Add Iteration` (manual mode only), takes me to the single Iteration create/edit form. - [ ] I can select an option to `Edit` or `Delete` and individual Iteration (manual mode only) (**Missing From UX Comp**) - [ ] Selecting `Delete` for a single iteration (manual mode only) prompts with a double confirmation modal (use Milestones as a fast follow) - [x] Selecting `Edit` for a single iteration (manual mode only) takes me to the create/edit view for a single Iteration. **Next:** - [x] Within each iteration cadence, I can see what cadence the Iteration is running on (MANUAL or `Duration Value`) ### Iteration Cadence Create/Edit Form ![New_cadence_-_automated](/uploads/6b61de892aae3a251c0da9fb8d785baa/New_cadence_-_automated.png) ##### Vertical Slices **MVC:** - [x] I can enter the name for the Iteration cadence. (Required Field) - [x] I can enter a description using markdown for the Iteration Cadence. (Optional Field) - [x] When `Automated Scheduling` is disabled and all the required fields are defined, I can click a button to create only the cadence, create the cadence and proceed to creating a new iteration within the cadence, or cancel the creation process all together. (see [wireframes](https://whimsical.com/iteration-cadences-HRa5WUtDUrkeRJnVjv9Gqs@2Ux7TurymQRJvroNsPXE) for example) - [x] When I click the button to `create cadence only`, I am taken to the Iteration Cadence list view. - [x] The single iteration create/edit form behaves the same way as it does currently with the exception of no longer having a `name` field as that will be inherited from the iteration cadence as each iteration with the cadence will be identified via its sequential number + date range. **Next:** - [x] I can disable `Automated Scheduling` (default to enabled) - [x] I can enter the start date for the first Iteration. - [x] I can select the duration (in weeks) for each Iteration within my cadence from a drop down list with values of `1`, `2`, `3`, `4`, `5`, or `6` - [x] I can select the number of future iterations i want to always have automatically scheduled from a list with the following values -- `2`, `4`, `6`, `8`, or `10` - [x] All fields are required except `Description` when `Automated Scheduling` is enabled. When it is disabled, only `Name` is required. - [x] When `Automated Scheduling` is enabled and the required fields are defined, I can either click a button to create the cadence or to cancel the creation process. ### Other Products Doing This Well <details> **Pivotal Tracker:** ![Screen_Shot_2021-01-05_at_1.02.52_PM](/uploads/1058ab645491f6f4af75bad28b63db86/Screen_Shot_2021-01-05_at_1.02.52_PM.png) **Linear.app:** ![Screen_Shot_2021-01-05_at_1.03.16_PM](/uploads/88b42e10f4c768ef24297f2dc6351d1f/Screen_Shot_2021-01-05_at_1.03.16_PM.png) </details> ### Example Use Case <details> ```mermaid graph TD Group_A --> Subgroup_B --> Project Group_X --> Iteration_Cadence3 Subgroup_B --> Iteration_Cadence2 Group_A --> Iteration_Cadence1 Project --> IssueA Project --> IssueB Project --> IssueC Group_X .->|Group Share|Project IssueA .-> Iteration_Cadence3 IssueB .-> Iteration_Cadence2 IssueC .-> Iteration_Cadence1 ``` </details> Old proposal: <details> - Instead of creating one iteration at a time, change the UI/UX for the user to only provide the cadence of the iteration (in weeks for starters) and a start date in future. - At this step we should take care that the start date in future does not overlap with any running iteration. It can overlap with future iterations, which would mean that future iterations get overwritten(either deleted and new ones are created or start/end dates get updated according to new cadence) - Iteration dates would care about not overlapping only at the level of the group that it is being defined, rather than care about parent groups or entire hierarchy. - List iteration cadences defined at parent group levels, and opt to either use one of those or create a custom iteration cadence for this specific group, if the parent inheritance strategy allows for overwriting iteration cadence. ### Some raw mockups of the idea - Listing Iteration definitions(cadences) from different parent levels, when none of the parents enforces a specific cadence ![1-iterations-list-subgroup](/uploads/6d85753c3bfddd0d33f34850afa95870/1-iterations-list-subgroup.png) - Creating a custom iterations cadence at subgroup level, if parent does not enforce a specific cadence ![2-create-new-iterations-cadence](/uploads/7ca5fffd926e5139ba34b5a37dc928b8/2-create-new-iterations-cadence.png) - Pick to enforce or not a specific cadence to subgroups ![3-specify-inheritance-strategy](/uploads/a0604efded32e484965402d56b26d627/3-specify-inheritance-strategy.png) - Listing only iterations that are being enforced when parent group enforces its iteration cadence ![4-enforced-inheritance-strategy](/uploads/4a4ac3b5b87dfe512fcb24707733d2c0/4-enforced-inheritance-strategy.png) - A more generalized diagram from @gweaver & some takeaways - There should only ever be a choice of subscribing to your direct parent or yourself. - Syncing to parent should be on by default. - When we enable this, if a group already has iterations defined in self for future or current dates, do not enable sync by default. ![Screen_Shot_2020-11-18_at_8.54.07_AM](/uploads/e87ded4578a43601cfcf78e0c7d51f25/Screen_Shot_2020-11-18_at_8.54.07_AM.png) </details> ## Backend perspective <details> ##### DB table `iterations_cadence` | column | type | notes | | ------ | ---- | ----- | | id | integer | | | group_id | integer | The group for which this iteration cadence was defined | | duration | integer | Duration of an iteration in days, i.e. cadence | | is_enforced | boolean | Determines if this cadence is being enforce to any child groups/projects or child groups/projects can define their own cadence. Default: faslse. | | is_active | boolean | Determines if given iteration cadence is active. We need to be able to de-activate a cadence in case a top level group enforces a cadence. | | start_date | datetime | the date/time when iterations with this cadence should start | | next_run_date | datetime | the date time when next set of iterations should be created | | iterations_in_advance | integer | how many iterations in advance should we create. We can skip this column for now and just use a constant hardcoded in the code, so this is not configurable | | created_at | datetime | | | updated_at | datetime | | ##### Iterations cadence and iterations workflow considerations - create iteration cadence - iteration cadences cannot be deleted and can only be created. When new iteration cadence needs to be defined, old iteration cadence becomes inactive and new iteration cadence record is being created. This should help audit any changes in the iteration cadence. - allow creation of iteration cadence only if at the time of creation none of the ancestor groups enforces its iteration cadence - Allow to set iteration cadence `start_date` at any date(past of future) if no iterations exist in the group where it is being defined or all defined iterations start in future - Set iteration cadence `start_date` to NEXT iteration `start_date` if there is a CURRENT iteration running in the group. - Allow to set iteration cadence `start_date` to LAST iteration `due_date` if all existing iterations are in the past or to any date after LAST iteration `due_date` - Iteration cadence is provided in days - Iterations start/due dates are auto-computed from iteration_cadence#start_date + cadence - All future iteration start/due dates need to be updated based on new cadence duration & start_date. - Daily cron job to check `next_run_date` on active iteration cadences to create new iterations and update `next_run_date` for respective iteration cadences. - By default `is_enforced` is set to false. - When a higher level group enforces an iteration cadence a warning should be shown if any of the subgroups already has an iteration cadence defined, notifying the user that the change will disrupt the workflow. - In the scenario when a subgroup has an iteration cadence defined and any of its ancestors enforces a cadence, the subgroup's iteration cadence should be set to `active: false` - If a given group does not have an active iteration cadence it should lookup iterations at ancestor level - Allow to create new iterations manually with a single click, if group has active iteration cadence. Use last iteration `due_date` to compute new iteration `start_date` and iteration cadence duration to compute `due_date`. Update iteration cadence `next_run_date` accordingly. - Generate 3? iterations in advance of CURRENT iteration - Validate that iteration dates do not overlap only at the group level where the iteration is defined instead of hierarchy wide. ##### Things to think about - Investigate/discuss if a migration is need or we can introduce the iteration cadence concept on top of existing iterations implementation - a migration would consider the duration of last iteration in order to set iteration cadence duration. - determine if iterations are being used. - check if iterations are spread across multiple groups in the hierarchy. How to prevent disruption ? </details> ## Blockers to General Availability - [x] After creating a new Iteration Cadence, it does not immediately show in the list of Iteration Cadences. I have to manually refresh the page for it to show up. - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68924 - [x] Fix FE state for "couldn't find iteration cadence" after deleting an iteration cadence and editing another (https://gitlab.com/groups/gitlab-org/-/epics/5077#note_627892792) - [x] When editing an existing iteration cadence, the form fields are not populated with the current values (i.e. Title, start date, and so on are blank) -- Data is returned correctly from the server - [x] Improve empty state of iteration cadence in manual mode with no iterations yet. "Create your first Iteration" CTA maybe? - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68924 - [x] No way to visually tell if an iteration is open or done when filtering list by `All`. - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68924 - [x] Breadcrumbs do not show the name of the iteration cadence. Currently shows ID. https://gitlab.com/gitlab-org/gitlab/-/issues/342759 - [x] Update filter bar to support cadences (see https://gitlab.com/gitlab-org/gitlab/-/issues/293925/designs/Iterations_filters.png) https://gitlab.com/gitlab-org/gitlab/-/issues/293925 - [x] Proper support in Boards for Iteration Cadences in Boards -- https://gitlab.com/gitlab-org/gitlab/-/issues/323653 - [x] Search for iterations in the picker by cadence name (https://gitlab.com/groups/gitlab-org/-/epics/5077#note_624848383) https://gitlab.com/gitlab-org/gitlab/-/issues/336611 - [x] Properly handle updating updates to iteration cadence - switching from manual to auto, changing during for auto-mode, etc. (https://gitlab.com/groups/gitlab-org/-/epics/5077#note_611884765) - [x] UX polish as defined by @hollyreynolds
epic