[VS Code] Spike: Diagnostics command

Problem to solve

When part of the extension fails/doesn't work, it's hard to find out if the issue is in the extension configuration or somewhere else. (see the parent epic for full problem description. This is partially through the relevant information being implicitly and explicitly shared in many places in the extension (status bar, sidebar, two output (log) panels).

Proposal

Create a diagnostic output that would give the user an overview of the extension configuration and enabled/disabled features.

Initial POC

I created a small POC but all the data is stubbed:

vse-diagnostics

MR: draft: POC diagnostics (!1903)

Example diagnostics output

Spike output

• The desired final diagnostics output is documented
• Incremental issues to implement that state are created and linked under the parent epic
• We know how to expose diagnostics from LS
• POC with partially dynamic values is created

Further details

• We should be able to reuse the FeatureStateManager (search for this class in LS) for the feature section gitlab-org/editor-extensions/gitlab-lsp!637 (merged) the problem is that we only send the engaged (read "failed") checks to VS Code Extension, we would need to start sending all
• We might first want to remove the support for multiple active accounts in a single VS Code Extension window [VS Code] Only support single account per works... (#1377 - closed) that will simplify the diagnostics (and mental model)
  • from arbitrary number of project-instance-account combination
  • to single account, single instance, possibly multiple GitLab projects

Links / references

added Category:Editor Extensions Editor ExtensionsVS Code devopscreate groupeditor extensions typefeature labels

added sectiondev label

added gitlab-org/editor-extensions#82 as parent epic

mentioned in issue gitlab#473283 (closed)

Question: What do you think about having a UI element in the menu that can tigger your diagnostic check in additional to the command GitLab Diagnostics?

@aregnery I think that's a good idea, maybe we can do it as a fast follow-up. The "Health Check" (diagnostics) feature (as I envision it) won't be limited to Duo. The command in the duo menu is only one place where I'd show this (a standalone command would be another one).

@viktomas a fast-follow would be awesome. I like that you are encapsulating more than just GitLab Duo, but since it is related I think it makes sense to tie them together in some way.

The "Health Check" (diagnostics) feature (as I envision it) won't be limited to Duo.

@aregnery @sguyon @viktomas I agree with the general statement that a healthcheck would be generally valuable outside of Duo. However, I'm also mindful that it's a significant expansion of scope at this moment to generalize our admin-based healthchecks to cover non-Duo cases. I can see this becoming more of a priority as we integrate additional non-AI cloud connected features.

Do y'alls see a need where we need to co-ordinate the scope of releasing "healthcheck beyond Duo" between IDEs and Admins? I want to be intentional whether or not we may be taking on UX debt when different healthchecks have different scopes (Duo vs larger). Would be great to know whether we can confidently decouple these efforts, vs planning in a bit more detail.

@rogerwoo I'm trying to meet the spirt of the spike by not pushing one way or another too hard, so I'm open to what make sense. I wanted to roll with what @viktomas already deemed feasible and find a method to connect it to the GitLab Duo experience.

@aregnery @rogerwoo I expect we'll focus on Duo first (that's where most support request lie ATM). However the information at the top of the screenshot

## System versions

- IDE: Visual Studio Code (1.93.0-insider)
- Plugin: GitLab Workflow (5.9.0)
- Language Server version: 6.3.0

## Project and account for active file

- Account: `viktomas`
- Project: `gitlab-org/gitlab-vscode-extension`
- Workspace folder: `/Users/tomas/workspace/gl/gitlab-vscode-extension`
- GitLab instance:
  - URL: `https://gitlab.com`
  - Version: `17.4.0-pre`

is IMO important for any debugging including Duo. Regarding the features section, we can start with Code Suggestions and iterate from there.

What I meant by

The "Health Check" (diagnostics) feature (as I envision it) won't be limited to Duo.

was that I'd prefer not to call this Duo diagnostics and force us to create other screens/commands when we want to add diagnostics for more features.

I'd prefer not to call this Duo diagnostics

@viktomas yep that's fair.

I'd prefer not to call this Duo diagnostics and force us to create other screens/commands when we want to add diagnostics for more features.

Thanks for the clarification @viktomas -- I'm strongly aligned with you on this ^.

### ❌ Duo code suggestions
- ✅ enabled in settings
- ✅ supported GitLab version
- ✅ user has Duo license
- ❌ enabled for project
### ❌ Duo chat
- ✅ supported GitLab version
- ✅ user has Duo license
- ❌ enabled for project

@viktomas I got some more confirmation on the text output. We should avoid using emojis here, particularly for accessibility reasons. See this internal Slack thread for more details. Two ideas I have:

(1) use markdown formatting to convey a similar meaning

### Duo code suggestions (Off)
- [x] enabled in settings
- [x] supported GitLab version
- [x] user has Duo license
- [ ] enabled for project

(2) Use [true] / [false] blocks

### Duo code suggestions (Off)
- [true] enabled in settings
- [true] supported GitLab version
- [true] user has Duo license
- [false] enabled for project

Question: What do you think about that type of tweak to your POC?

Makes sense @aregnery, I updated the POC and screenshot in the issue description. Mind you, until this spike is done, we don't have dynamic data powering the Diagnostics, it's all hardcoded text.

changed the description

It would be useful to probably name the file to something other than Untitled, maybe GitLab Duo health check. What do you think?

@aregnery This sounds like a good idea. I'm not sure if we'll have to save that file somewhere first for the tab having a name. But we can find that out during this spike

I'd prefer not to call this Duo diagnostics

Cool and like you said @viktomas we could use a more generic title since it encapsulates more than just Duo checks presently.

changed the description

mentioned in merge request gitlab!165032 (merged)

mentioned in issue gitlab-org/cloud-connector-team/team-tasks#186 (closed)

@viktomas I was curious what you need to get !1903 to a point where it's ready. Is it capacity or is there another limiting factor?

@aregnery I can help answer the priority question. We do milestone planning every month and by the 4th of each month I have a planning issue created to slate work for the next milestone. This helps us stay on track as a team and be clear on what we're delivering. Not too different from other teams. @viktomas and I chatted and we'll be adding this to our 17.6 milestone.

That's awesome to hear @dashaadu! I think this will be super impactful to our support teams and their customers. Thanks for sharing a tentative timeline.

changed milestone to %17.6

mentioned in issue gitlab-org/editor-extensions/meta#159 (closed)

added workflowplanning breakdown label

mentioned in issue #1616

marked this issue as related to #1616

mentioned in issue #1617

@viktomas This was picked up as Deliverable for you in %17.6!

added Deliverable label

assigned to @viktomas

Setting health status to on track as the milestone has just begun.

Issue participants are welcome to override this by setting the health status to another value.

changed health status to on track

mentioned in issue gitlab-com/Product#13688 (closed)

set weight to 3

@kisha.mavryck @dashaadu I don't think I can finish this issue by the end of the milestone. It seems like the new milestone starts 2024-11-11 and I'm mostly OOO till then.

I can maybe start working on this spike on Friday but after that, I'm OOO till 2024-11-11 (which means %17.7 already).

changed milestone to %17.7

mentioned in issue gitlab-org/editor-extensions/meta#181

changed milestone to %17.8

mentioned in merge request !2118 (merged)

Async issue update 2024-11-13

• Current status: I plan to start on this issue tomorrow.
• Next steps: I'll define the information I plan to show in the initial spike version of diagnostics and get sign off from @dashaadu @aregnery and @tvanderhelm
• Shipping this milestone: no, the result of this spike will be available in %17.7 the original spike description says that output of this issue will be POC, but maybe we'll have something to ship to prod, I'll keep you all posted.
• Scope reduction opportunities: I don't know yet.

@dashaadu I see that you moved this issue to %17.8, is this still a priority for me to deliver in %17.7?

/cc @kisha.mavryck

mentioned in issue #1688

marked this issue as blocking #1688

mentioned in issue gitlab-org/editor-extensions/meta#188 (closed)

added UI text documentation twtriaged labels

added workflowin dev label and removed workflowplanning breakdown label

mentioned in merge request !2233 (merged)

Async issue update 2024-12-18

• Current status:
  • I started working on the POC. I have some code running and I have idea about what needs to be done.
  • I also checked with @lauraionel about how we present health-checks in JetBrains
• Next steps: I'm planning to do two things as a part of the POC:
  • add production ready command and infrastructure for diagnostics into the extension (behind FF since the diagnostics page will be empty)
  • write up about the spike result & break down the result in to issues in the [VS Code Extension & LS] - Self-served troubles... (gitlab-org/editor-extensions&82) epic
    • I will ping both @tvanderhelm and @aregnery there for feedback
• Shipping this milestone: YES, probably shortly after the New Year
• Scope reduction opportunities: I could not write any production code as part of the spike if we really needed to reduce the scope

As a sneak-peak, this is the POC so far:

cc: @dashaadu, @kisha.mavryck

@viktomas Since we have both Duo Pro and Duo Enterprise now, as part of this spike, can we ensure that we're showing the right add-on in the diagnostics or am I scope creeping?

@dashaadu I think you are scope creeping Do you know if we have that information available anywhere in the extensions? In other words do we change the UI anywhere based on if the Duo is Pro or Enterprise? (I'm not aware of any "feature" like that)

Either way, I will create an issue for it. In the parent epic.

@viktomas I knew it . Currently we do not have anything that is in the extension that is part of Enterprise and not part of Pro, so both are covered in our use case. To keep it safe, if we can remove the add-on type in the diagnostics, that would be great. So that it's GitLab Duo Chat and we can use the convention throughout.

Hey @viktomas

• Solid progress ....
• And, I am going to borrow this phrase scope creeping from you ... I dig it ...

Hey @dashaadu

• We can make space capacity wise to queue up the additional issue at the top Milestone %17.9 as a Deliverable if that works for you

mentioned in merge request gitlab-org/editor-extensions/gitlab-lsp!1193 (merged)

Spike result

@dashaadu @tvanderhelm @kisha.mavryck @aregnery I finished the spike.

Please read the full spike result in the epic description: gitlab-org/editor-extensions&82 (careful, it might be below the fold) (read it, it's really nice )

• it contains a list of issues that I created, all the issues are in the epic

Next up: We must implement [VS Code]: Version Diagnostics Section + Diagno... (#1761 - closed) first, that's the plumbing and the screen itself that will allow us to add the remaining Diagnostics sections in parallel

Spike discussion: Let's discuss topics that affect all the diagnostics section on the epic, and let's discuss each section in its own issue

/cc @ohoral @lauraionel @tristan.read @dbernardi you might be interested in the technical solution

Nice progress on this!!! ... . ....@viktomas

Wishing you a happy and joy-filled holiday season

cc: @ohoral @tristan.read @dashaadu @dbernardi

closed

mentioned in issue gitlab-org/editor-extensions/meta#197