Skip to content

Project & Group Level Card Sorts (Contextual Navigation)

Progress

  • Enter project level contextual navigation items onto cards.
  • Enter group level contextual navigation items onto cards.
  • Write email to accompany card sorts.
  • Distribute card sort to a test sample of users.
  • Review answers and make any necessary amendments.
  • Distribute email to remaining users.
  • Purchase Amazon vouchers, conduct prize draw and contact winners.
  • Cleanse data.
  • Analyse results.
  • Write up findings.
  • Create new issues as required.
  • Update relevant issues in the CE project.

Research Questions

  • Is Charts misplaced under Repository? Would a more suitable location be within an Insights category?
  • Do users confuse the content of Graph and Charts? (A different research study is required to answer this question, however, we may be able to glean some insight from this study)
  • Is Contributors misplaced under Repository?
  • Should Labels appear in the top level navigation at a project level?
  • Should Labels appear in the top level navigation at a group level?
  • Should Milestones appear in the top level navigation at a project level?
  • Should Milestones appear in the top level navigation at a group level?
  • Should Epics appear in the top level navigation at a group level?
  • Are there differences in the way users sorted cards at a project and group level?
  • What is the optimal structure of the contextual navigation at a project and group level?

Methodology

  • Only the contextual navigation was tested.
  • 72 users participated in an open card of project level items.
  • 46 users participated in an open card of group level items.
  • Users could create as many categories as they wanted.
  • Users could also access a brief description of each piece of content if they wanted to.
  • Users were asked to order content within the categories. Where number 1 would be the item they wanted to see first within the category. The results detailed below reflect this ordering.
  • Due to the research questions being asked, I made sure users who do not use issues and users who do not use issues but have applied labels and/or milestones to merge requests were included in the data. I analysed the results from both of these user groups individually and cumulatively with the rest of the data. There were no distinct differences.

Results

Is Charts misplaced under Repository? Would a more suitable location be within an Insights category?

Yes, Charts is misplaced under Repository. Users felt Charts should appear within Statistics.

Do users confuse the content of Graph and Charts?

Inconclusive - could be solved with tree testing or usability testing. This wasn't the right kind of research study to test this question. Users could access a brief description of each piece of content if they were unsure of what it was from just its name alone. Based on the description users read, they felt Graph was better suited to the new category of Statistics rather than Repository. However, we should still be cautious if we do decide to rename Graph to something else as it may change users' feelings on which category it should reside within.

Is Contributors misplaced under Repository?

Yes, Contributors is misplaced under Repository. Users felt Contributors should appear within Statistics.

Should Labels appear in the top level navigation at a project level?

No, Labels was never categorised on its own as a top-level navigation item at a project level. 61% of users grouped Labels with other content found under the heading of Issues. The alternative suggestion of moving Labels to Merge Requests (mentioned in https://gitlab.com/gitlab-org/gitlab-ce/issues/39268) is no longer possible as Merge Requests is no longer a top-level category. I purposely ensured that we tested with users who 1) didn't use issues and 2) users who didn't use issues, but had applied labels to merge requests - the results were no different. Whilst I fully appreciate that it is very inconvenient for users to access Labels from an Issues menu if they are not using issues. We have to design for the majority of our users. If Labels becomes a top-level navigational item we risk overloading the sidebar navigation and causing confusion for the majority of our users.

Should Labels appear in the top level navigation at a group level?

No, Labels was never categorised on its own as a top-level navigation item at a group level. 55% of users grouped Labels with other content found under the heading of Issues.

Should Milestones appear in the top level navigation at a project level?

No, Milestones was never categorised on its own as a top-level navigation item at a project level. 67% of users grouped Milestones with other content found under the heading of Issues.

Should Milestones appear in the top level navigation at a group level?

No, Milestones was never categorised on its own as a top-level navigation item at a group level. 72% of users grouped Milestones with other content found under the heading of Planning.

Should Epics appear in the top level navigation at a group level?

No, Epics was never categorised on its own as a top-level navigation item at a group level. 69% of users grouped Epics with other content found under the heading of Planning.

Are there differences in the way users sorted cards at a project and group level?

Yes, see below. For the tree test, we will test the project and group contextual navigation together, rather than in isolation of one another (this wasn't possible for the card sort). This will help us to evaluate whether the differences in structure cause users confusion.

What is the optimal structure of the contextual navigation at a project and group level?

Project Level

58% participants categorised their cards as follows:

  • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Environments
  • Issues
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Repository
    • Details
    • Files
    • Commits
    • Branches
    • Tags
    • Compare
    • Locked Files
    • Snippets
    • Merge Requests
  • Statistics
    • Activity
    • Cycle Analytics
    • Contributors
    • Graph
    • Members
    • Charts

Group Level

74% participants categorised their cards as follows:

  • Project
    • Issues
    • Boards
    • Labels
    • Milestones
    • Merge Requests
    • Roadmap
    • Epics
  • Project (This is not a typo! Project was used twice.)
    • Details
    • Members
  • Analytics
    • Activity
    • Contribution

Recommended optimal structure of the contextual navigation:

Project Level

  • Project
    • Details
  • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Environments
  • Issues
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Compare
    • Locked Files
    • Snippets
    • Merge Requests
  • Statistics
    • Activity
    • Cycle Analytics
    • Contributors
    • Graph
    • Members
    • Charts
  • Wiki
  • Registry
  • Settings (if applicable)

Group Level

  • Project
    • Details
  • Issues
    • List
    • Boards
    • Labels
    • Merge Requests
  • Planning
    • Epics
    • Milestones
    • Roadmap
  • Statistics
    • Activity
    • Contribution Analytics
    • Members
  • Settings (if applicable)

There is no way to quantify from this card sort what the value of my recommendation is. Whilst I am confident that it is an improvement over the existing navigation, I cannot put a % value on it. However, if I run a tree test, I can provide this data. A tree test may also highlight further tweaks/improvements which can be made to my recommendation, especially in regards to the points below. I will be conducting a tree test as my next piece of research.

Justification

  • Moving Details to Project (Project Level) - At a project level, Details received 30 unique categorisations. 31% of people categorised Details under the heading of Project. This was actually the most frequently used categorisation. However, the reason why Details ended up within Repository is that more people frequently grouped it with other content within Repository than on its own. Project was also a weaker term for collectively describing all the content within Repository. At a group level, Project was the most frequent categorisation for Details. Subsequently, we could create some consistency between project and group level navigation. I do want to flag that some users felt Details and Files were a repetition of one another and that both pages of content were not necessarily needed - I think we should investigate this further.

  • Issues as a top-level category (Group Level) - Obviously we can't have two categories entitled Project at a group level. Despite two categories having the same heading very few users grouped Details and Members with any of the other content available within the second Project category, so users do think the content is different. Therefore, we shouldn't just combine the Project categories. 36% of people grouped the issues List, Boards, Labels and Merge Requests together. Surprisingly the strongest relationship is the issues List and Merge Requests (76% of people grouped them together). The most popular headings for this cluster of content were Actionables and Issues. I'm inclined to go with Issues as I feel this is a more universal phrase.

  • Planning as a top-level category (Group Level) - Roadmap, Milestones and Epics were grouped together by 57% of users. Plan and Planning were the most popular headings for this cluster of content. I cross-checked how many people combined the content of Planning and Issues together - 13%. Therefore, the content is stronger in two individual categories if we don't use the heading of Project.

  • Changing the heading of Analytics to Statistics (Group Level) - This is predominantly for consistency purposes between the project and group navigation. Statistics was 3 x stronger than the heading Analytics at a project level. However, the word Statistics was not used at all at a group level. So why am I suggesting it? I believe some bias has occurred at a group level. At a project level, there's more content within Statistics and therefore greater diversity in what content is called. At a group level, we have two pieces of content, one of which contains the word Analytics. I feel users have been led by this.

  • Members added to Statistics (Group Level) - Again it offers consistency between the project and group navigation. 64% of people categorised Contribution Analytics and Members together so I feel the content works well together.

  • I didn't actually test Wiki and Registry (since not all projects use them and from previous rounds of usability testing I've hardly seen them enabled) but some users noticed they were missing from the sort! Based on the feedback from users these are still best as their own categories as, like suspected, not all people use them.

Follow-up Research

  • Conduct a tree test of the existing and the recommended contextual navigation to further test and understand the value of the new hierarchy. (#43 (closed))
  • Investigate whether users are using Snippets. (#44)
  • Investigate whether Details and Files should be consolidated into one view. (#45)
Edited by Taurie Davis