Custom workflows - Group configuration

Motivation

• Many large companies need custom workflows as part of their issue management, as stated here: https://gitlab.com/gitlab-org/gitlab-ee/issues/1743.
• An example is that a team needs to ensure every single issue goes through exactly these stages: "To Do" > "In Dev" > "In QA" > "Code review" > "UAT" > "Ready for release" > "Post-production check".

Larger organizations also have more complex compliance needs. For those that we've talked to about "enforced workflows", it always goes back to a need to verify certain things due to compliance regulations. Several companies we've talked with base their compliance related processes on the Gates Methodology.

As one of these customers put it, they would ideally like GitLab to take what is now a hugely manual process for them -- collect all the issues that need approval, put screenshots in a slidedeck for the approval manager, have a meeting and walk through each issue one by one to get approval -- and have GitLab automate the approval process along the lines of:

1. Make some fields mandatory and mappable to certain gates/checkpoints.
2. When those fields have been filled out and they are within acceptable ranges, automatically pass the issue through the gate/checkpoint.
3. Associate this "evidence" into compliance/audit reports.

The customer sentiment was, "if a machine can help remind people to follow the process, it would be much better to let it do the nagging so I don't have to strain my interpersonal relationship with colleagues by constantly pestering them to add some piece of required meta-data to their issues"

Description

• Per group, configure the custom workflow states for issues.
• Define allowable workflow states.
• State is different from workflow states.
• GitLab issues have only two states. Open and Closed.
• When an issue is Open, it can be in one of several customized workflow states.
• When an issue is Closed, it doesn’t belong to any workflow state by definition. (I.e. its workflow state is null.)
• By default there are two customized workflow states called Ready and Development. You can change the names of these workflow states. These two workflow states come as part of any new group created.
• When an issue is created, it starts in the first workflow state (part of the Open state) by default.
• You can create any number of workflow states per group, but you can only delete down to two workflow. I.e. minimum two workflow states per group.
• GitLab won't check if you use the same name for multiple workflow states.
• When you delete a workflow state, for all issues that were in that workflow state, they just go "backward" a stage. If the first workflow state was deleted, then all issues in that workflow state just go "forward" a workflow state.
• An issue takes the workflow states from its immediate parent group.

In the group configuration, we could have something like this, to allow an admin to configure the workflow states.

Notes

• Issue lists and existing issue boards and burndowns work the same as now since there are still the same two defined states.
• There is a new group board type. This special board type has all lists predefined which are the custom stages. You can config the board as normal. But cannot add/remove lists. &424

Value stream management

• The workflow design here needs to work with VSM. See &505. So we need to be careful in designing the workflow states so it is straightforward to calculate times in a given workflow state and it is well-defined.
• In particular, there is an outstanding problem to identify cycle time versus lead time. Cycle time is the time required for an issue to be worked on a shipped. Lead time also includes the additional time before the issue is even worked on, and is first logged by a customer. So while cycle time is focused on operational excellence, lead time is even more important, because it measures responsiveness of an organization to customer feedback.
• I think we should focus on cycle time first, because it is easier to define within GitLab given or existing abstractions. Lead time is harder because for example in GitLab, issues are often open/closed/re-opened/moved and often an issue stays around for a long time before we consider implementing it, but that doesn't reflect badly that we are ignoring customers.

Groups vs projects and workflow states

• Within an issue page, you will be able to see what workflow state it is in (only if it is Open). And you can transition to another workflow state. Or simply close the issue. In particular, within an issue, you should only see one workflow state. This is the problem that Jira ran into which became very complicated. Even though an issue belongs to one project, which belongs to a group and indirectly a family tree of groups, it should only ever have a single workflow state. So we probably should allow workflow state schemes to be defined at multiple projects and groups, an issue should only ever have a single workflow state.
• I think we should start defining workflow states in a single group, and not even bring this functionality to projects at all, to avoid confusion. Teams that need this level of precision and management are typically already using groups. For teams using projects to do workflow management, we can ask them to use labels, the existing method.
• So if you have a single group, and a set of workflow states defined for that group, all issues (in projects) in that group share the same workflow states, and everything is well-defined. You can also have group issue boards that use that workflow board. See &424.
• What happens now with subgroups? There is a need to define workflow states at higher-level ancestor groups since organizations want consistency at a higher management level, and want all teams lower down that tree to use the same workflow states. This should be tackled in future iterations, but we should try to have the best design upfront before we iterate there.
• I think we should have a simple overriding mechanism: Given the following group hierarchy from oldest to youngest: Group A > Group B > Group C > Project P > Issue x. Again, issue x always has one well-defined workflow state. (Or at most one. It can have done if the workflow states are not defined.) Groups A, B, C can all have their own workflow states defined. But C overrides B, and B overrides A. For example, if only A has workflow states defined (and B and C don't), then issue X takes on A's workflow states. If B has its own workflow states, then issue X takes on B's. And if C has its own workflow states, then issue X takes on C's.
• There are at least two salient consequences of this design:
  • Given a project P, all issues in P will have the same workflow states.
  • Given a group G, different issues in G may have different workflow states. So that means if you create a workflow issue board in G (i.e. &424), not all issues in G may appear in that workflow issue board. I think this design is okay. But we should be aware of this and make a conscious designs/decisions along the way.

Feature flag

This issue and https://gitlab.com/gitlab-org/gitlab-ee/issues/8181 should be implemented and released together as a first iteration. Since they are both big issues, I (Victor) think we should do this one, release it hidden behind feature flag so that the feature is off by default. And then when https://gitlab.com/gitlab-org/gitlab-ee/issues/8181 is ready, then have feature flag be removed altogether.

Other supporting use cases

Incident Management Use Case

Incident management in and of itself can be considered a series of workflows to route critical issues that require urgent remediation to the right person. This can be achieved by a custom workflow set up to route alerts, create incidents, set severity, page different people, link to different resources, etc

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.