1.2 General navigation tasks - follow-up testing.
Follow-up testing to: #3 (closed)
Related to: #4 (closed)
Videos
User 1 - http://bit.ly/2oaiKkb
User 2 - http://bit.ly/2nY445m
User 3 - http://bit.ly/2oCtZ5K (Restrictive access, the user has not given GitLab permission to share this video publicly).
User 4 - http://bit.ly/2nmcmXO
User 5 - http://bit.ly/2oNaW8A
User 6 - http://bit.ly/2ohhb43
Findings
Icon recognition and functionality (out of context)
Key
- Success = User was able to correctly identify the icon and it's purpose.
- Partial = User exhibited some recognition and understanding of the icon but wasn't confident in their answer.
- Fail = User did not correctly identify the icon
| + | todos | issues | merge requests | |
|---|---|---|---|---|
| User 1 | Success | Success | Success | Success |
| User 2 | Success | Partial | Partial | Success |
| User 3 | Partial | Fail | Success | Success |
| User 4 | Partial | Fail | Success | Success |
| User 5 | Task skipped | Fail | Fail | Success |
| User 6 | Success | Success | Fail | Success |
| Total | 3 success, 2 partial, 1 skipped. | 2 success, 1 partial, 3 fail. | 3 success, 1 partial, 2 fail. | 6 success |
Usability issues and recommendations
| Area affected | Problem | Number of affected users | Priority | Recommendations / further comments | Related issue number |
|---|---|---|---|---|---|
| Groups I am not a member of | Whereas most users passed this task (5/6), it was only by process of elimination. All users clicked back and forth between ‘Your Groups’ and ‘Explore Groups’. Had the group lists been longer and the difference between the group lists not as obvious, I believe more users would have failed this task. | 6 | High | Users mentioned looking for a “filter”, “label” or “icon”. | |
| Issues icon | Issues icon was mistaken as ‘chat messages’ / related to Slack (out of context). Further emphasised by users typically hovering over the icon to read the tool tip before clicking it (within context). | 4 | High | Try testing a different icon with users. | |
| Creating a new project within a group. | When creating a new project within a group, users often thought they needed to be located within the group they were asked to add the project to. | 4 | High | The addition of the + icon has not solved this issue. If we push users to interact with + icon we could aggravate the problem outlined below. | |
| Creating a new project within a group and + icon. | In relation to the above, if the user navigates to the group where they wish to add the project to and then clicks the + icon.They do not realise that they are now creating a project within their namespace (by default) rather than the group they have just navigated to. | 1 | High | Despite only one user performing this action, I feel this could grow into a large usability issue for the reason outlined above. | |
| Interface as a whole. | Interface looks like a ‘“tablet”, “mobile” or “responsive view” with “wasted space”, rather than designed for desktop. | 3 | High | Reduce white space. The hamburger menu may be contributing to the users’ perspectives that the interface is designed for a mobile/tablet. The users I tested with generally had large screens, perhaps consider increasing the size of the prototype for future testing. | Related to UX polish issues. |
| Issues | Users expect to see an issues board view from the Issues page (in the sidebar). | 3 | High | As I reported in #3 (closed), I didn’t feel that users initially realised that the sidebar navigation was relevant to them. I felt more research was required to support this theory. The behaviour observed in this round of testing provides further evidence to support this theory - i.e users initially visiting Issues to see an “overview of tasks in a GitLab project” demonstrates a lack of understanding that the Issues page is personal to them. | Related to gitlab-ce#29878 and gitlab-ce#29882 |
| Projects | The heading ‘Memberships’ implies that you are only a member of projects which appear under this heading. | 3 | High | Considering changing this heading and re-testing. | Related to gitlab-ce#29045 |
| + icon | + icon was mistaken as contextual, rather than global. | 2 | High | We should re-test this in the next round of usability testing. | |
| Sidebar | A GitLab user explained that he has the sidebar pinned by default, but comments that the new issues and MRs icons will help alleviate the two clicks he now has to make. He further comments that he uses ‘Projects’ a lot - he typically navigates to them via the breadcrumb (his comments are supported by his behaviour during the testing), but if he can’t find them he uses the sidebar. A non GitLab user comments that it would be nice if the sidebar wasn’t hidden and that the navigation was more exposed on the screen. | 2 | High | Supports existing data and proposed changes. | Related to gitlab-ce#29878 |
| MRs | When viewing MRs within the sidebar, one user commented that he did not know whether these were just related to the group of ‘Customers’. This demonstrates two problems: 1) Lack of understanding that everything in the sidebar is personal to the user 2) Lack of understanding of where he was in his user journey. | 1 | High | Already in progress. | Related to gitlab-ce#29878 and gitlab-ce#29882 |
| Hamburger menu | One user overlooked the sidebar. Sarah had to prompt the user to interact with the hamburger menu. | 1 | High | Already in progress. | Related to gitlab-ce#29878 |
| Project settings | A GitLab user initially struggled to locate Project / Members as he was looking for the gear icon. He described his experience as “unsettling”. | 1 | High | From the persona work, this is a massive pain point for some of GitLab’s users (frequency of changes to the UI). I’m not sure if there’s anything we can do in relation to this exact change, but I wanted to document this feedback and share it with you all. | |
| Todos | Todos icon was mistaken as ‘builds’. Users stated that it may indicate that a test has succeeded or failed and subsequently show them an error log (out of context). When the icon was shown in context, 3 users still overlooked it’s intended purpose. | 3 | Medium | Try testing a different icon with users. | |
| Comprehension of todos (non GitLab users) | All three non GitLab users struggled to understand the concept of todos.They all had different ideas about what content they would find within todos. | 3 | Medium | As reported in #3 (closed), similar behaviour has been observed in previous testing with different non GitLab users. | |
| Using todos (GitLab users) | All three GitLab users which we tested with mentioned that they hardly used todos. 2 out of 3 users commented that they just repeat issues/MRs, subsequently when they visit todos they are no longer relevant (one user commented that his todos are normally closed or merged) | 3 | Medium | It could be that ‘todos’ become obsolete with the addition of issue and MRs icons. | |
| Todos | When viewing the list of todos, one user mistakenly thought that all issues were ‘Done’, rather than realising that the true intent of the CTAs was to mark a task as ‘Done’. | 1 | Medium | We seem to be gathering a lot of insight into ‘todos’, perhaps ‘todos’ is it’s own separate project? | |
| Milestones | Users expect to see an issues board view from the Milestones page. | 2 | Low | ||
| Project ‘owner’ | Two users commented that they did not understand what project ‘owner’ meant. Both users were non GitLab users. | 2 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| Activity View | Users assume they can change their default project view by accessing ‘Projects’. | 2 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| All icons (colour) | One user thought that the use of grey for the icons implied status, i.e. pending, completed, etc. | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| All icons | One user completely overlooked all icons. When questioned at the end of testing, he admitted that he hadn’t spotted them. | 1 | Low | Possibly an edge case. No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| MR icon (colour) | The use of orange on the MR icon indicated to a user that it was a list of ‘pending’ MRs (rather than assigned MRs). User mentioned that GitHub uses ‘orange’ to depict ‘pending’ / ‘modified’. | 1 | Low | Perhaps we can review GitHub’s use of colour in order to gauge what assumptions users moving from GitHub to GitLab might have in relation to colour. | |
| Todos | When attempting to interact with the ‘Type’ filter on the Todos page, one user thought it would show him the progress of the Todos (‘open’, ‘in progress’, ‘closed’) | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| Todos | When attempting to interact with the ‘Action’ filter on the todos page, one user thought it was “what I’m going to do with it”. An action he expected to perform was ‘re-opening a closed todo’. | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| Todos | When interacting with the Todos page, a user commented that by clicking on the avatar of a user he could chat with or message the user. This user may have thought there was a chat functionality within GitLab due to an earlier task where he identified the issues icon as relating to chat messages. | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| Search | The search bar should be smart enough to search by group name. “I want something that lets me type in whatever I want” and finds the appropriate content. | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
| Creating a new project within a group | When creating a new project and attempting to change the group name, one user tried to amend the IP address. | 1 | Low | The heading ‘Group or user namespace’ is above the IP address rather than the group/namespace field, perhaps it needs to be moved? | |
| Groups I am not a member of (icons) | One user investigated the group icons but with no tool tips, it was unclear to her what the icons represented. | 1 | Low | I could investigate the usability of these icons in a future testing session. | |
| Project labels | One user commented that she would like labels at a project level, so she could quickly understand more about the contents of a project. | 1 | Low | No action required. Sarah will continue to watch for similar comments / behaviour in future usability testing. | Not Applicable |
Progress since last usability testing session (#3 (closed))
| Area affected | Observation | Affected users | Related issues |
|---|---|---|---|
| Personal projects | All users were able to find their personal projects | 6 | gitlab-ce#29045 |
| Issues and MRs | Despite some confusion about what the issues icon represents, once users had clicked the icon and viewed the content, they generally understood that these were their personal issues. Similarly, the majority of users who clicked the MR icon understood that the MRs displayed were relevant to them. The new icons are better than the current sidebar at conveying to users that the content is personal to them. | 4 | gitlab-ce#29866 |
| Issues board | The ‘issues board’ task had an improved success rate due to secondary level items appearing on hover of primary level items. | 4 | gitlab-ce#22069 |
| Tanuki | Increase in interaction with the tanuki. Users commented that they expected the tanuki to take them ‘home’ to their personal dashboard. Users expected to see the following on their personal dashboard: issues (both issues I’m assigned to and watching), merge requests, projects and recent activity. Basically “everything associated with me”. | 2 | |
| Number of projects | No users manually counted their projects, following the addition of the badge count to project/group/subgroup tabs | 6 | gitlab-ce#29798 |