Allow checklists to have additional states: "failed", "does not apply", and "in progress"

Problem to solve

GitLab includes checklists, which look like this in markdown:

- [ ] Unchecked Item A
- [X] Checked Item B

However, there are other states that we wish to communicate in a checklist - notably, "this check has failed" and "this check does not apply".

Target audience

Developer, Maintainer, Project Manager, anyone who uses checklists

Further details

The difference between "I haven't looked at this checklist item yet" and "I have decided this checklist item has failed", or "I have decided this checklist item does not apply", are interesting. At GitLab, we tend to represent the first by "checking" the checklist item with a - character. The second, we tend to represent in either the same way, or by removing the item from the checklist. Once rendered, that looks like:

Screenshot_from_2018-12-11_16-25-44

There's also an "in progress" state we could consider.

Proposal

We could take the - [-] entry we already use in these cases and treat it as either the "does not apply" or "check has failed" case. Probably more the former than the latter.

It would be awesome to use - [x] as the "failed" case, but we already commonly use that for the "checked" case, and migration /retraining would be impossible. So perhaps we can use - [!] to denote that a check has failed.

"In progress" could be represented as - [~], although that might be a bit too visually similar to - [-]. Other suggestions welcome. Of the five total states, this seems like the least useful to me.

When we're actually using the rendered checklist, we need to be able to choose between the two new states when checking it. A quick click could toggle between "checked" "unchecked" (now "passed" and "not checked"), while a long click could give us a dropdown with all possible states to choose from.

Alternatively, each click could cycle between the possible states, but I feel that could be somewhat annoying. One for our gitlab-ce~2024184 team to consider!

The rendered checklist can now show a big green tick for passes, a big red cross for failed, or a nice bold strikethrough for "does not apply".

What does success look like, and how can we measure that?

At a glance, I should be able to determine what items were in the checklist when it was originally created; which items have been deemed inapplicable; which have passed, and which have failed (which implies further action is needed).

Here's a checklist with the suggested new states in the markdown source, so we can see how it looks now:

  • This item is TODO
  • This item has passed
  • [!] This item has failed
  • [-] This item does not apply
  • This item is in progress

After merging, we should see five distinct states here, beautifully rendered, and be able to move between states easily.

Links / references

Since checklists are usable in all Issuables and notes, I've put this between ~Plan and ~Create. I think either could pick it up, but perhaps it tends more to the former?

cc @victorwu @jramsay @smcgivern @DouweM

Edited by Nick Thomas