### Progress * [x] Enter project level contextual navigation items onto cards. * [x] Enter group level contextual navigation items onto cards. * [x] Write email to accompany card sorts. * [x] Distribute card sort to a test sample of users. * [x] Review answers and make any necessary amendments. * [x] Distribute email to remaining users. * [x] Purchase Amazon vouchers, conduct prize draw and contact winners. * [x] Cleanse data. * [x] Analyse results. * [x] Write up findings. * [x] Create new issues as required. * [x] Update relevant issues in the CE project. --- ### Research Questions * Is `Charts ` misplaced under `Repository`? Would a more suitable location be within an `Insights` category? * Do users confuse the content of `Graph` and `Charts`? (A different research study is required to answer this question, however, we may be able to glean some insight from this study) * Is `Contributors` misplaced under `Repository`? * Should `Labels` appear in the top level navigation at a **project** level? * Should `Labels` appear in the top level navigation at a **group** level? * Should `Milestones` appear in the top level navigation at a **project** level? * Should `Milestones` appear in the top level navigation at a **group** level? * Should `Epics` appear in the top level navigation at a **group** level? * Are there differences in the way users sorted cards at a project and group level? * What is the optimal structure of the contextual navigation at a project and group level? --- ### Methodology * Only the contextual navigation was tested. * 72 users participated in an open card of **project** level items. * 46 users participated in an open card of **group** level items. * Users could create as many categories as they wanted. * Users could also access a brief description of each piece of content if they wanted to. * Users were asked to order content within the categories. Where number 1 would be the item they wanted to see first within the category. The results detailed below reflect this ordering. * Due to the research questions being asked, I made sure `users who do not use issues` and `users who do not use issues but have applied labels and/or milestones to merge requests` were included in the data. I analysed the results from both of these user groups individually and cumulatively with the rest of the data. There were no distinct differences. --- ### Results #### Is Charts misplaced under Repository? Would a more suitable location be within an Insights category? Yes, `Charts` is misplaced under `Repository`. Users felt `Charts` should appear within `Statistics`. #### Do users confuse the content of Graph and Charts? Inconclusive - could be solved with tree testing or usability testing. This wasn't the right kind of research study to test this question. Users could access a brief description of each piece of content if they were unsure of what it was from just its name alone. Based on the description users read, they felt `Graph` was better suited to the new category of `Statistics `rather than `Repository`. However, we should still be cautious if we do decide to rename `Graph` to something else as it may change users' feelings on which category it should reside within. #### Is Contributors misplaced under Repository? Yes, `Contributors` is misplaced under `Repository`. Users felt `Contributors` should appear within `Statistics`. #### Should Labels appear in the top level navigation at a project level? No, `Labels` was never categorised on its own as a top-level navigation item at a project level. 61% of users grouped `Labels` with other content found under the heading of `Issues`. The alternative suggestion of moving `Labels` to `Merge Requests` (mentioned in https://gitlab.com/gitlab-org/gitlab-ce/issues/39268) is no longer possible as `Merge Requests` is no longer a top-level category. I purposely ensured that we tested with users who 1) didn't use issues and 2) users who didn't use issues, but had applied labels to merge requests - the results were no different. Whilst I fully appreciate that it is very inconvenient for users to access `Labels` from an `Issues` menu if they are not using issues. We have to design for the majority of our users. If` Labels` becomes a top-level navigational item we risk overloading the sidebar navigation and causing confusion for the majority of our users. #### Should Labels appear in the top level navigation at a group level? No, `Labels` was never categorised on its own as a top-level navigation item at a group level. 55% of users grouped `Labels` with other content found under the heading of `Issues`. #### Should Milestones appear in the top level navigation at a project level? No, `Milestones` was never categorised on its own as a top-level navigation item at a project level. 67% of users grouped `Milestones` with other content found under the heading of `Issues`. #### Should Milestones appear in the top level navigation at a group level? No, `Milestones` was never categorised on its own as a top-level navigation item at a group level. 72% of users grouped `Milestones` with other content found under the heading of `Planning`. #### Should Epics appear in the top level navigation at a group level? No, `Epics` was never categorised on its own as a top-level navigation item at a group level. 69% of users grouped `Epics` with other content found under the heading of `Planning`. #### Are there differences in the way users sorted cards at a project and group level? Yes, see below. For the tree test, we will test the project and group contextual navigation together, rather than in isolation of one another (this wasn't possible for the card sort). This will help us to evaluate whether the differences in structure cause users confusion. #### What is the optimal structure of the contextual navigation at a project and group level? **Project Level** 58% participants categorised their cards as follows: * **CI/CD** * Pipelines * Jobs * Schedules * Environments * **Issues** * List * Boards * Labels * Service Desk * Milestones * **Repository** * Details * Files * Commits * Branches * Tags * Compare * Locked Files * Snippets * Merge Requests * **Statistics** * Activity * Cycle Analytics * Contributors * Graph * Members * Charts **Group Level** 74% participants categorised their cards as follows: * **Project** * Issues * Boards * Labels * Milestones * Merge Requests * Roadmap * Epics * **Project** *(This is not a typo! Project was used twice.)* * Details * Members * **Analytics** * Activity * Contribution #### Recommended optimal structure of the contextual navigation: **Project Level** * **Project** * Details * **CI/CD** * Pipelines * Jobs * Schedules * Environments * **Issues** * List * Boards * Labels * Service Desk * Milestones * **Repository** * Files * Commits * Branches * Tags * Compare * Locked Files * Snippets * Merge Requests * **Statistics** * Activity * Cycle Analytics * Contributors * Graph * Members * Charts * **Wiki** * **Registry** * **Settings** (if applicable) **Group Level** * **Project** * Details * **Issues** * List * Boards * Labels * Merge Requests * **Planning** * Epics * Milestones * Roadmap * **Statistics** * Activity * Contribution Analytics * Members * **Settings** (if applicable) There is no way to quantify from this card sort what the value of my recommendation is. Whilst I am confident that it is an improvement over the existing navigation, I cannot put a % value on it. However, if I run a tree test, I can provide this data. A tree test may also highlight further tweaks/improvements which can be made to my recommendation, especially in regards to the points below. I will be conducting a tree test as my next piece of research. **Justification** * Moving `Details` to `Project` (Project Level) - At a project level, `Details` received 30 unique categorisations. 31% of people categorised `Details` under the heading of `Project`. This was actually the most frequently used categorisation. However, the reason why `Details` ended up within `Repository` is that more people frequently grouped it with other content within `Repository` than on its own. `Project` was also a weaker term for collectively describing all the content within `Repository`. At a group level, `Project` was the most frequent categorisation for `Details`. Subsequently, we could create some consistency between project and group level navigation. I do want to flag that some users felt `Details` and `Files` were a repetition of one another and that both pages of content were not necessarily needed - I think we should investigate this further. * `Issues` as a top-level category (Group Level) - Obviously we can't have two categories entitled `Project` at a group level. Despite two categories having the same heading very few users grouped `Details` and `Members` with any of the other content available within the second `Project` category, so users do think the content is different. Therefore, we shouldn't just combine the `Project` categories. 36% of people grouped the issues `List`, `Boards`, `Labels` and `Merge Requests` together. Surprisingly the strongest relationship is the issues `List` and `Merge Requests` (76% of people grouped them together). The most popular headings for this cluster of content were `Actionables` and `Issues`. I'm inclined to go with `Issues` as I feel this is a more universal phrase. * Planning as a top-level category (Group Level) - `Roadmap`, `Milestones` and `Epics` were grouped together by 57% of users. `Plan` and `Planning` were the most popular headings for this cluster of content. I cross-checked how many people combined the content of `Planning` and `Issues` together - 13%. Therefore, the content is stronger in two individual categories if we don't use the heading of `Project`. * Changing the heading of `Analytics` to `Statistics` (Group Level) - This is predominantly for consistency purposes between the project and group navigation. `Statistics` was 3 x stronger than the heading `Analytics` at a project level. However, the word `Statistics` was not used at all at a group level. So why am I suggesting it? I believe some bias has occurred at a group level. At a project level, there's more content within `Statistics` and therefore greater diversity in what content is called. At a group level, we have two pieces of content, one of which contains the word `Analytics`. I feel users have been led by this. * `Members` added to `Statistics` (Group Level) - Again it offers consistency between the project and group navigation. 64% of people categorised `Contribution Analytics` and `Members` together so I feel the content works well together. * I didn't actually test `Wiki` and `Registry` (since not all projects use them and from previous rounds of usability testing I've hardly seen them enabled) but some users noticed they were missing from the sort! Based on the feedback from users these are still best as their own categories as, like suspected, not all people use them. --- #### Follow-up Research * Conduct a tree test of the existing and the recommended contextual navigation to further test and understand the value of the new hierarchy. (#43) * Investigate whether users are using `Snippets`. (#44) * Investigate whether `Details `and `Files` should be consolidated into one view. (#45)