Research Proposal: Use Web IDE for all file/code editing

The Web IDE is now stable and reasonably feature complete. We should aim to have a single editing experience for files in GitLab (e.g. where Ace is currently used) so that there is consistent experience and less duplicate code. (source: &498 (closed))

What questions are you trying to answer?

Preference & Usage

• How do Ace edit workflows compare to the Web IDE workflows?
  • Do users currently utilize the Web IDE or Ace Editor more? Why?
  • Do users see a difference between the Web IDE vs Ace editor?
• Do users complete a set of tasks differently when using the Web IDE vs Ace editor?

Naming

• Should we rename the Web IDE button to Edit and remove the old Edit button?
• Should we have an Edit with Web IDE or Open in Web IDE button instead of a plain Edit button?

Other considerations

• What are the current performance characteristics of the Ace editing interface vs the Web IDE?
  • How does the launch time of the Web IDE compare to the Ace editing interface? - example here: &498 (comment 113129104)
• Are there functionalities currently supported in the Ace editor that are not supported in the Web IDE?

What assumptions do you have?

• Users are confused by seeing both the Web IDE and Edit button.
• It makes sense to open files in Web IDE by default instead of the blob editor on desktop devices. In mobile devices its best we stick to the blob editor which is better designed for small screens. - https://gitlab.com/gitlab-org/gitlab-ce/issues/46829#note_80547081
• We need to include 'Edit' in the button because that's a really well-understood action that most users will want to perform (e.g. describe the action rather than the tool) - https://gitlab.com/gitlab-org/gitlab-ce/issues/46829#note_80770499
• Being able to delete all the non-web-ide edit code in GitLab would be a great experience - https://gitlab.com/gitlab-org/gitlab-ce/issues/46829#note_113120427

Methodology

• User Interviews
• Data analysis - https://gitlab.com/gitlab-org/growth/issues/18

Progress

• Write screener survey [Deadline: Fri Dec 14th]
• Write usability testing script [Deadline: Tues Dec 18th]
• Schedule users. [Deadline: Weds Dec 19th]
• Test the script. Conduct 1 usability testing session. Edit the script if required. [Deadline: Weds Dec 19th]
• Conduct remaining usability testing sessions. [Deadline: Fri Dec 21st]
• Pay users. [Deadline: Fri Dec 21st]
• Edit and analyze videos and add them to the Research proposal issue. [Deadline: Mon Jan 7th]
• Write research report. [Deadline: Mon Jan 7th]
• Update the Research proposal issue. Link to the research report. Unmark as confidential if applicable. Change status to done and close issue. [Deadline: Tues Jan 8th]

Results

• Full Research Report

Who we spoke with

• User 1: Software Engineer, organization size of 101-500 people, edits files/code in GitLab several times a week, using Core for work use
• User 2: DevOps Engineer, organization size of 11-100 people, edits files/code in GitLab 1-3 times a month, using GitLab.com for work use
• User 3: Development Team Lead, organization size of 1,001-10,000 people, edits files/code in GitLab several times a week, using GitLab.com and Core for work use
• User 4: DevOps Engineer, organization size of 1,001-10,000 people, edits files/code in GitLab once a week, using GitLab.com and Core for work use

Key Findings

• How do Ace edit workflows compare to the Web IDE workflows?

  • Ace Editor: editing one file, light-weight small changes
  • Web IDE: making changes to multiple files, editing code (e.g. editing functions, changing logic)

• Which of the file editing methods do users prefer? Why?

  • User 1: He’d prefer to get rid of the edit button/Ace Editor and treat the Web IDE as part of all the other parts of GitLab. He says that the Web IDE has more features and can accomplish everything the Ace Editor does, so it makes sense for it to be the main editing path.
  • User 2: In his personal workflow, he prefers using the smallest resource possible and feels that the Ace Editor is a simpler approach. However, if he had to edit code, like editing a function or changing logic, he would use the Web IDE.
  • User 3: For really quick changes (for example, changing less than 50 letters in one file) he prefers the Ace Editor. He feels that it is lighter even though he hasn’t measured the time. He knows his way around - just click Edit, make the changes and quickly submit. With the Web IDE there are more features and he may be more compelled to wait for other changes to be made and make one large commit. This serves editing multiple files best.
  • User 4: This testing session was the first time he tried using the Web IDE. He really enjoyed the experience of using the Web IDE because it supports all the standard features associated with a Git editor. He liked the overview that you get when using the Web IDE, otherwise he’d have to use “git status.” He sees his normal workflow as “change things, commit, push” and thinks that the Web IDE is good tool to be able to edit directly in the repository he would push to anyway.

• How can we improve the Web IDE?

  • User 1: No specific feedback. He thinks GitLab’s editing features work well for his needs.
  • User 2: Fix the loading time of the Web IDE. Slow loading times would make completing super small tasks very cumbersome. Additionally, it can be very confusing to understand how to access other parts of the project, when you are inside the Web IDE. He states that it feels as if he is detached from the rest of the project.
  • User 3: Normally he and his team have multiple monitors setup so he thinks it’d be useful to have the Preview Markdown running as a separate window so that he can have a separate screen and he can look at it. Additionally, he’d like to be able to do a 1-click commit rather than staging changes. He thinks this is what made the Web IDE feel slower.
  • User 4: He comments on the fact that the Web IDE opens slower than the Ace editor and suggests loading the editor before the file tree, so that you can start typing while still waiting for the Web IDE to load.

• Web IDE Launch speed in each session:

  • User 1: 13 seconds (video not shared)
  • User 2: 10 seconds (16:56-17:06)
  • User 3: 8 seconds (19:06-19:14 in video)
  • User 4: 14 seconds ((15:59-16:13 in video)

Conclusions/Next Steps

• Are there any factor to consider, if we choose to reduce duplication of the editing paths?
  • If we make the Web IDE the primary path, we need to make sure that it is able to feel as lightweight and easy to use. The Web IDE should also feel more integrated and less like users are being taken to a separate application.
    • User 2 stated that we need to consider the fact that the Ace Editor is more “user friendly” because it is simpler to look at, faster to load, and has less “distracting” features.
    • User 4 mentioned the fact that clients he works with may be intimidated by the Web IDE and he may have to support them however, he thinks it still can be easy enough to use.
      • He also was concerned that it may be harder to get back to repository file list view then stated that he probably wouldn’t need to do that anymore because the file browser would be right there in the Web IDE.