Draft Problems to solve for AI in everything we do
As we are building AI in everything we do there are certain problems to solve that could be applied across the entire experience. Unlike most "UIs", and APIs" that are generally geared toward a specific task, AI's ability to understand a wide variety of inputs and produce a wide variety of output makes it unique to any other technology. If we can extract the fundamentals and do those well, leveraging the wide range of inputs and outputs, we may be able to accelerate our design practice with regard to AI to meet the demands of rapid changes by not delivering a lot of good experiences, but rather delivering a couple of great ones but that applies in many situations.
TL;DR
- Need to investigate the "root cause" problems that AI introduces to an experience so we understand the needs
- Need to envision the whole design problem so we know where we are going
- Need to tackle that design problem thematically so we can work efficiently
- Need to componentize themes and apply them broadly so we can be sustainable
LLM limitations
This is an example of the type of root cause analysis we could do. As always, focusing first on the problems tends to give you a better chance at arriving at solutions so let's first look at some of the limitations of AI.
Context
LLMs are trained on a particular data set but are not necessarily aware of information outside of that data set. This is particularly true of dynamic and "private" data like most of the GitLab data set. In order for AI to provide meaningful results, it needs to be provided with context. GitLab can overcome this limitation by providing LLMs with the relevant context for a given task.
Memory
Similar to context, LLMs lack stable memory. Depending on the application some do have some memory, but it is often unpredictable for how long and in what context. With human-like responses, this can often present itself with very confusing results. GitLab can overcome this limitation by providing a consistent and relevant memory as a form of context that can provide a much more stable and predictable outcome.
UI
In theory with enough copy-pasting, sophisticated prompting, and a way to pass in context a user can leverage AI without the need for a UI. But the inefficiency required knowledge, and the inconvenience of this degrades the experience. GitLab can overcome this limitation by integrating AI into our existing UI reducing the effort of leveraging AI.
Actions
AI output is limited to being informational unless it is given the ability to take action. This action could take the form of performing an action within another system or it could be using the generated information as input back into the AI to produce additional output. GitLab can overcome this limitation by giving AI some capability, but also protecting the user from unsafe capabilities.
Approach example
This may not be at all correct, it is incomplete and it's more just an illustration of the kinds of exercises I think we should go through, but here are some of the things we could start doing. Much of this I think we are already doing. I think we just need to formalize and document this in a digestible way.
Steps we could take to see an overall view of the UI problem to solve:
- Identify at a high level the designs we will need (example below but I also see this happening naturally as the team is working)
- Have teams look through their product areas and image any place in the UI they could imagine AI showing up
- Generate a list of possibilities
- Map possibilities against current identified design patterns (what pattern could we use or do we really need a new one)
- See gaps that need to be filled
- Maintain this map to inform future feature additions and identify experience divergence
Example of what step 1 could look like - If AI is in everything, what designs do we need
We should try to see all the designs we are going to need and do them in priority but also do them as themes.
- Chatbot
- These designs seem to be well underway.
- Determine 1 solution or if we need variance, what drives the varience
- Text inputs
- We are handling the cases like issue descriptions.
- We should just start designing for accessing AI in general from any text box because we know it's probably coming.
- Actions
- Standalone button
- Integrated button
- In a menu
- In a drop-down
- Flows
- Need a standard interaction flow for triggering an AI event.
- Need a pattern for what starting in action looks like.
- Need a pattern for what just doing and action looks like.
- Need a waiting pattern/animation
- As a user
- In a conversation like issues
- As avatars
- Other places where users are shown like the members page.
- In history (commits, comments, activity)
- Do we treat it as a special thing or do we display it like a user? We should make that decision once and stick with it.
- Tech writing
- We should develop standardized patterns since AI is very good at filling in patterns correctly.
- We should have a go-to place for all AI possible patterns for your messages and a list of actual error messages.
- Research
- We can focus our research on the interaction patterns and know that we have those points nailed down so when we have to apply new patterns we already have parts of the testing in place.
Examples of how we could start looking into solutions to the root cause problems:
Memory
- We need to figure out what kind of memory gives the best results. Is it forever is it short-term what is the optimal length of time?
- Is it task specific and if so what are the variables that change it?
- Really what we're asking is what are the problems we need to solve in regards to memory.
Context
- Product design is probably in the best position to inform what is the best context to give because we are the most aligned with the needs of a user in a given job.
- This is probably the least repeatable task we need to do, but if we map what context was provided for what actions we might be able to spot trends and predict future asks and find ways to improve across all situations.
Actions
- What are we willing to let AI do and what are we not willing to let it do?
- Do we prefer a prefill pattern or an actually do pattern or both and why.
- Similar to the UI flows, what is the interaction pattern in and out canceled and errored ect.
Why the ML do we need all this
Last week my spell checker Chrome plugin was a spell checker, this week it is a full-featured generative AI writing assistant (no this was written by a human). AI has changed the world, our industry, and our company, and our design practice needs to change with it. "We need to win" and "AI in everything we do" are tall orders. We won't win without a great experience in this space. Everything we do is a lot and the window to succeed is very short.
We should look at what has not gone well in the past and do things the way we would have done knowing what we know now if we knew them from the beginning. If we focus only on designs to meet our current feature development, we will always be behind the curve. We can't just focus on what is in front of us, we need to look where we are going. Design needs to lead development. If development happens rapidly the only way to be out in front is to predict the UI that you're going to need and have it designed already. We need a build a practice now that can meet the pace of the new world.
We can focus our efforts on places that will have the most impact like research on the interaction patterns and know that we have those points nailed down so when we have to apply new patterns we already have parts of the testing in place. We can separate the ux (interaction design, visual design) from the product design, front-loading with UX and setting us up for the product design that will likely follow.
Let's be willing to be wrong but if we componentize these things much like updating the last set of buttons we had the confidence to do it across all of them, then on the day that we need to fix some of these things we just fix it once and it fixes all of them.
It's challenging to collaborate across the whole product but we can collaborate on these experiences and make sure we have a shared understanding and contributions for anyone who has the capacity to spare.