Explore placement of operations dashboard icon

Problem to solve

We're getting feedback that the new operations dashboard icon is too distracting in the top nav.

Proposal

Explore whether the icon should be replaced or moved elsewhere.

Methodology

Usability testing conducted with 5 GitLab users. None of the users have used GitLab's Operations Dashboard prior to this research. Testing occurred with GitLab.com users, with the exception of User 5. User 5 completed the task in the same way as other users but was shown a mockup from GitLab documentation, in order to evaluate the icon.

Participants

You can access video recordings by clicking on the user number.

  • User 1, Entrepreneur, using GitLab for about 3 years, organization size of 1-10 people
  • User 2, Executive, using GitLab for almost 2 years, organization size of 11-100 people
  • User 3, Development Team Lead, using GitLab for about 3 years, organization size of 11-100 people
  • User 4, Software Quality Engineer (in a DevOps environment), using GitLab for about 4 years, organization size of 1,001-10,000 people
  • User 5, Site Reliability Engineer, using GitLab for about 3 years, organization size of 101-500 people

Task performance

Users were given the following task to complete, using their GitLab instance:

Imagine that you had a number of projects, and you wanted to see a summary of their health. Where do you think you would look?

Metrics: Pass/Fail, Time taken to complete

Correct Location User 1 User 2 User 3 User 4 User 5
Operations Dashboard Fail, 8 seconds Fail, 2m7s Fail, 1m17s Pass, 16 seconds Fail, 1m8s

Key insights

  • User 1: Expects to find a way to view a summary of project health in both CI/CD>Pipelines and Operations>Metrics

    • Imagines that the Operations Dashboard icon will take him to an operations page. Realizes that maybe once he has configured operations-related features in his projects, he’d be able to see data on the page. He understands that the dashboard represents health across all projects. Has no concerns about the icon placement / design.

  • User 2: Goes to group overview page because he was thinking of an overview of many projects; then when he thinks about summary of their health he tries to find info related to all the tests that they are running. He thinks he could find it near Activity or Details in that case. If he goes to one specific project, he thinks he can find it by looking at the CI/CD pipelines page to see if everything is passing. He’d expect to have information about CI/CD on that group overview page in this situation, in addition to the project level.

    • When he thinks of this concept in the context of the group overview page, he starts to struggle with the concept of what the dashboard would be mapping onto. He thinks he should be able to find the dashboard for a certain team and tries to edit the URL to go directly to the team but is surprised when he gets a 404 error. He then starts to question whether the dashboard is a dashboard of all the projects and also the other teams and groups. For his workflow, he’d like to see a dashboard related to one of his main projects as well. He thinks it makes sense to also be on the group overview page and accessible via CI/CD. He says maybe on both locations.
    • When asked about his first impression of the icon he remarks that he knows that we may not have enough space in the topnav to write “Dashboard.” He says the only thing that’s weird is that it’s the only one that is not in written text on the left side of the topnav (it doesn’t match the rest if the items). Says once you know it, it’s fine. He thinks that the icon in the header provides easier access than having CI/CD stats on the group overview page.

  • User 3: He first goes to groups and thinks that it should be possible in the CI/CD pipelines page. He thinks there should be some monitoring tools in CI/CD. Then says he’d either look in CI/CD or Operations. He clicks into Operations>Metrics and says that if he installs Prometheus, he should be able to use it. Says he always has confusion between CI/CD and Operations. He often looks for Operations-related things in CI/CD.

    • User gets an explanation of the icon and dashboard. When asked if there in a difference between what he’d see in the project level Operations> Metrics vs Operations Dashboard, he says there is. In the project level he expects everything to be more specific and detailed about the project, whereas the dashboard would be more general and act as an overview.
    • General impression of the icon: it’s a bit tiny. He has a sense of what it could mean but he’s never attempted to click on it. He thought that it may have represented things that he needed to configure, and that prevented him from clicking on it (because he didn’t want to spend time on setup).

  • User 4: Says that he would guess that the summary of health can be found by clicking the gauge in the top navigation. He guessed this location because it would be an easy location to access especially if you needed to check the health of all projects. When asked what made him look in that location he remarked that it was partly because he’d already seen it before and wondered “ooh, what’s that!” The other reason he looked there is because he generally expects the top navigation to hold things that need to be easily accessed. He states that you’d want it to be “ooh, there it is” rather than hidden behind a series of menus, since it relates to the health of all projects.

    • From his first impression of the icon he was definitely thinking it looked like it'd lead to a dashboard. The gauge made him think “what is the health? Are you running at a good pace? Are you running a bit low on capacity or capabilities?” To him it was a visual indicator. He goes on to mention that he did need to actually hover his mouse over it to get the alt text and see it really was. He thinks that potentially having Operations Dashboard written in the topnav, instead of an icon, may make it easier to understand the concept.

  • User 5: First instinct is to go to the projects switcher and then to the projects dashboard view. He then starts to think about the word "health" and says that it is indicated by the number of commits (shown on the projects dashboard view). He says this is where he’d expect to see whether a continuous integration piece passed or failed. When he heard the word “health,” he was inclined to think of build health.

    • When he took a look at the documentation for the operations dashboard, he thinks it would be perfect for his infrastructure project. He think it’s a really cool feature. He likes the quick status view but mentioned some concern from the colorblind perspective (because the green alert may look like a red error to someone with colorblindness).
    • When talking about impressions of the icon, he says that he’s not sure that he’d understand the difference between all the instance-wide icons in the topnav (operations dashboard, instance statistics, and admin area). One is showing him what looks like a speedometer which would make him think of velocity or some sort of gauge.
    • He does not think it’s bad placement for the icon, however, he is not sure that he’d be able to imagine what it is just from the icon. He says it’s clearly something high-level and something to do with measuring or checking either velocity or health, but if he had seen just the icon first (without viewing the docs for the dashboard) he’d likely click on it out of curiosity, rather than intentionally clicking on it to find an operations dashboard. He’d almost expect an operations dashboard to be represented by the icon that is currently being used for instance statistics.

Conclusions

  • Participants did not find the presence of the icon distracting or out of place.

  • Users who had no awareness of the dashboard were inclined to look for similar capabilities under CI/CD>Pipelines or Operations>Metrics. Once users fully understood that the summary of health was for all projects, the majority began to understand that placement in the sidebar would limit the scope to a specific project. With this context in mind, they were open to having the Operations Dashboard icon in the top navigation as a way to easily access the dashboard.

Next Steps

  • Consider how user feedback compares to Josh's concerns / suggestions in #119 (comment 120665600)
  • If decided that we’d like to gather more feedback on the icon, the following recommendations arose from users:
    • Keep the operations dashboard on the left side of the top navbar and change the icon into text that reads “Ops Dashboard” or “Operations Dashboard.” This recommendation was made in favor of showing consistent design on the top navbar and also eliminating guesswork. Some users may clearly expect a gauge to relate to operational health while others may not fully understand it without further context.
    • Move the icon to the right side of the top navigation so that it is consistent in design

Progress

Edited by Katherine Okpara