Draft: POC: Configurable work item types

What does this MR do and why?

This is a work in progress POC for configurable work item types which consists of two parts:

1. Move WorkItems::Type to WorkItems::SystemDefined::Type and all related classes/methods
   1. Create/list/view work items works
   2. Legacy issue view works for incidents
   3. Legacy Service Desk list works
   4. Related GraphQL queries work
   5. change type works
   6. Issue and epic boards work
   7. 🚧 custom fields
   8. 🚧 some note related errors... I likely missed a preload somewhere
   9. 🚧 some of the settings
2. Introduce custom work item types WorkItems::Custom::Type and related classes

WorkItems::SystemDefined::Type

This is the system-defined version of the WorkItems::Type model. Right now it uses a minimal set of attributes and only has the methods that were needed to make work items (list, detail, create, boards) work.

Some notable details:

def key
  name.parameterize.underscore.to_sym
end

wit.key
=> :issue

This is the base type. I chose to not implement the epic?, issue? etc. methods, so we can see in this MR which places we need to change. I changed the method calls to something along the lines of wit.key == :issue. Not because I like this more, but so we have this change recorded in the MR.

I intentionally used the name key, so I needed to change occurrences of calls to base_type. Again this is not because I like key more, it's to record these changes in the MR. With custom types the term base_type could make sense, so let's use whatever feels right.

# Could be an idea to fetch the "base type" to reduce the length of the expression.
# WorkItems::SystemDefined::Type[:issue] vs. WorkItems::SystemDefined::Type.by_base_type(:issue)
def [](key)
  return if key.nil?

  # Accept several input formats
  key = key.parameterize.underscore.to_sym unless key.is_a?(Symbol)

  all.find { |item| item.key == key }
end

This would allow us to fetch a system-defined type by calling WorkItems::SystemDefined::Type[:issue] instead of WorkItems::SystemDefined::Type.find(1) or some other method name with base_type. I think that could be a pattern for fixed items models that has some kind of additional identifier apart from the id, so one can access it in a more readable and short way.

def by_ids_ordered_by_name(ids)
  where(id: ids).sort_by { |type| type.name.downcase }
end

Scopes are not ideal right now because we cannot chain them (scope.order...). I'm unsure whether we should introduce an intermediate class (like a relation) that handles queries better. For now, let's just add class methods that do what we need.

def icon_name
  "issue-type-#{parameterized_name}"
end

Derived from the name. I think we should move the icon names to an enum or something different because icon names can change and might not be based on the name in the future. That's especially true for custom types where we should store the integer value of an enum in the database instead of the icon name.

def widgets(_resource_parent)
end

There's no point in enabled_widgets any more 👍

Loading items

In discussions with @kassio and @stefanosxan we agreed that it'd be great to use a module for the type definition of each type so it's more readable and easier to digest. We also discussed including the available widgets there and build the widget definitions items from these lists. The current implementation in the POC doesn't reflect that but has a lot of comments about what we discussed. I chose to stop here because Stefanos already started with implementing system-defined types.

This module contains a rough idea of how these type modules that build the type hash can look like.

WorkItems::SystemDefined::WidgetDefinition

This is the system-defined version of WorkItems::WidgetDefinition. Right now its structure is similar to it's legacy model version.

Some notable details:

auto_generate_ids!

Fixed items models must have an id. This enforcement is especially useful for items that are used in database associations. That's not true for widget definitions. We don't really care about the id of the widget definition. To make the definition of items easier, I added the option to auto generate ids for items, so they don't need to be defined any more. This makes dynamic item definitions very readable.

We're planning to load widgets from the list of available widgets on the type. We moved the symbol list out of the widget definitions scope because it felt natural that "all things types" are in one place. So in combination with auto_generate_ids! we can build create the items on app load dynamically.

Issue

I added a getter and setter to Issue for work_item_type and chose not to use the belongs_to_fixed_items association because we'll need to resolve system-defined and custom types anyway.

The getter/setter implementation contains a lot of comments about thoughts we went through and lacks of resolving an actual custom type using the custom_type_id, but I believe can figure that out on the go.

Custom types

This POC contains the initial set of table and model to make converted types work and give a preview of how custom types can work. I intentionally chose to stop here because the places we'd need to do changes were notable and I think we can use that time better on the actual implementation. Most notably we need to add the second custom_type_id to tables around lifecycles and custom fields.

The GraphQL API supports both GlobalIds already through a shared interface module and custom types use the legacy GID when it's a converted type.

When we convert a system-defined type to a custom type we create a custom type and set the converted_from_system_defined_type_identifier to the id of the system-defined type. This way we can look up which custom type to use as a replacement for a system type.

For MVC1 widgets cannot be changed. That means we can always delegate available widgets and hierarchy to the system-defined type. The POC uses simple delegation to achieve this.

For completely new custom types we can use the same model. We agreed that new types will behave like issues, so we can use issue as the base type for custom types and delegate widgets and hierarchy to the system-defined issue type.

In the future customers will be able to customize widgets and hierarchy. We can copy available widgets and hierarchy to a custom version for MVC2 and 3 when they perform the first modification action.

WorkItems::TypeProvider

Because a namespace can now have both system-defined and custom types we need a central place where we resolve the list of types based on the context (root namespace/organization). In the POC I chose to go with the namespace only. We already have the app/models/work_items/custom/type.rb which we could use as this place. This class acts like a funnel that starts with all types and then discards types based on feature flag and license and project/group level context. So it gives you the allowed types for a context. The logic here is already complex, so for the sake of the POC I chose to create a new class TypeProvider which simply fetches the valid types for a context, which can then be fed into the filter or directly be used to resolve a type based on an id.

I also tried to illustrate ideas on how we can cache the types list using a lookup table, using query cache, request store or a more permanent cache. This is crucial because we cannot rely on preloading associations now. It's a mix of system-defined and custom types now.

So whatever the end solution will look like, we should try to keep it in one place in my mind.

Legacy DB types model

I chose to keep the current model so I don't break more things than needed. Not all endpoints consume the system-defined types not but that's okay I guess.

What this POC doesn't do!

1. It's not a full implementation.
2. It's a sneak peak into the solution space
3. It doesn't implement custom types fully
4. You cannot create items of new types
5. It doesn't change the frontend
6. The code is not optimized or aims to be performant

Examples

Here're some examples of how this actually looks like. Not all endpoints work, but I focused on the namespaceWorkItemTypes and namespaceWorkItem(s)queries. Also the boards queries etc. work.

Namespace work item types

Using the query with system defined types returns the same result as before. You can add a custom type via the console that for example overrides the issue base type.

::WorkItems::Custom::Type.create!(namespace: Group.find(33), name: "Feature", plural_name: "Features", converted_from_system_defined_type_identifier: 1)

Now when you query work item types you'll notice that issues aren't available any more and instead there's an item with the same global id on it and the name "Feature":

This is also true for fetching a single work item or a list of work items:

Real custom types are also included in the work item types query and you can also see that they have the same list of widgets and hierarchy than issues and use the custom global id (instead of the legacy one that the converted uses):

References

1. [POC] Configurable Work item types (#580260 - closed)

Screenshots or screen recordings

Before	After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.