No-code Workflow Automation - Nov 28th, 2022 Update
Recording
Summary
- The automation UX prototypes
- Explore the architectural options (to be updated in a separate issue shortly)
- Workflow specification: https://serverlessworkflow.io/
- Event specification: https://cloudevents.io/
- Fault-tolerant execution engine https://temporal.io/
Automation UX
It's been a while since my update that covered a PoC for the workflow automation. I've been very lucky to receive some excellent feedbacks and suggestions from the folks who are experienced in the problem domain, especially on the future direction and its potential.
One particular area that clearly needs more love is the UX to define the workflow, as also discussed here. On one hand, the yaml based UX, as demonstrated in the PoC, offers limited discoverability due to the inherited learning curve to understand and use the DSL. To be frank, business users are less likely to read through documentation like this. On the other hand, the yaml syntax, from 5 years ago, is getting outdated and needs some work to realign with CI/CD pipeline (more on this topic shortly).
So the question becomes: what kind of UX can we offer that is both intuitive and extensible while abstracting away the DSL for business users?
Compare the two approaches
There are essentially two types of UX for building workflows: flow chart and web form.
The flowchart-based solutions typically provide a canvas for drag-and-drop components and making connections. Products in this category tend to focus more on the reusability of the components, data flow, branching and looping logic. You may find this construct closely resembles the core concepts in computer programming, therefore they are more popular among technical users. To highlight a few:
- https://www.workato.com/
- https://tray.io/
- https://n8n.io/
- https://nodered.org/
- https://docs.aws.amazon.com/step-functions/latest/dg/workflow-studio.html
In comparison, the web-form based solutions usually take away the flow control and leave the users with a much more guided experience, such as a step-by-step wizard to collect configuration info. General users who are comfortable with web forms can find themselves at home. Some notable products:
- https://zapier.com/
- https://www.airtable.com/product/automations
- https://www.atlassian.com/software/jira/features/automationhttps://ifttt.com/
- https://support.apple.com/en-au/guide/shortcuts/welcome/ios
- https://notion-automations.com/
It's not hard to imagine GitLab's strong developer user base may benefit directly from the flow chart experience, especially for authoring CI/CD pipelines. Not surprisingly, we already have some work here:
Both are great improvements over the hand-rolled yaml, however, they also left some room for imagination compared to the more graphical builder such as https://harness.io/. So I decided to work on a prototype as an alternative. (See demo in the next update together with the serverless workflow spec
I quickly learnt that the flow chart could be overkill for building the majority of the automations simply structured with trigger, conditions and actions. This could also be why Zapier and Jira Automation chose the guided form wizard experience instead of asking users to program with flowcharts. So I decided to build another prototype to explore and try to answer the questions such as:
- How viable is the form wizard experience to support first-time users to define workflow automation within a few clicks?
- How extensible can we design the FE solution to support additional work items and rules in the future?
- How can we decouple the UI from the underlying DSLs to allow independent evolution of the yaml syntax?
- How much existing capability can we reuse to optimise the time-to-market in order to kick-off the iteration?
The technical design
There are two specs to describe the attributes and UI components used in conditions and actions. It means we can simply append the spec to support additional fields with built-in or custom components.
A snippet from conditionsSpec.json
date: [
{
name: "attribute",
type: String,
component: "select",
values: ["created_at", "updated_at", "merged_at", "authored_date", "committed_date"],
},
{
name: "condition",
type: String,
component: "select",
values: ["older_than", "newer_than"],
},
{
name: "interval",
type: Number,
component: "input"
},
{
name: "interval_type",
type: String,
component: "select",
values: ["days", "weeks", "months", "years"],
},
],
milestone: [
{
name: "value",
type: String,
component: "select",
values: ["none", "any", "upcoming", "started"],
allow_custom: true,
}
],
state: [
{
name: "value",
type: String,
component: "select",
values: ["closed", "opened", "locked", "merged"],
}
],
JSON schema generated form, e.g. react-jsonschema-form has been considered but it's hard to justify the customization effort for GitLab's design system.
I've also borrowed some implementation ideas from @janis's pipeline wizard while deciding to build the prototype separately due to the need for the nested fields and a rather different UX. However, given the similarity between the two problems, I do believe we can converge on the solutions.
A yaml builder is responsible for serializing the workflow object into the gitlab-triage yaml policy that gets committed to the project. This component decouples the domain model from the representation, allowing new DSL to be introduced in the future.
I kept the workflow execution the same as the previous PoC, i.e. a sidekiq worker wraps the gitlab-triage gem. The gem may not be the most future-proofing solution but does significantly reduce the TTM.
UX design
My conclusion
The form wizard appeared to be the easiest to get started with than the yaml or flow-chart based UX. It's not as powerful as the flow-chart, but good enough for most of the use cases (assumption alert). Please disagree with me on this by leaving a comment below.
Hiding the DSL behind the UI can buy us some time to iterate on the specification and the execution engine, more on this in the next update.
This page contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned on this page are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.