Loading
docs(skills): add ui-feature-testing skill for VS Code E2E verification
What does this MR do?
Adds a new agent skill, ui-feature-testing, that gives the agent a
deterministic, repeatable path for verifying user-visible features of the
GitLab VS Code extension end to end with WebdriverIO (wdio-vscode-service).
The skill encodes the knowledge an agent needs to go from "a feature was changed" to "it's verified running inside a real VS Code instance" without rediscovering the build/run/debug mechanics each time.
What the skill covers
- When to use E2E vs. cheaper layers — unit (Jest), integration (mocha + msw), or webview component (vitest) — so the agent doesn't reach for slow E2E tests unnecessarily.
- Tiered verification workflow — unit → build → E2E wiring check, fastest signal first, with the exact commands for each tier.
- Building the extension under test —
npm run build:desktopintodist-desktop/, plus how to confirm a command id actually landed in the bundle (grep gl.<command> dist-desktop/extension.js) and the manifest gotchas (desktop.package.jsonvs. generateddist-desktop/package.json). - Authentication —
TEST_GITLAB_TOKENsetup and the secret-safe logging pattern. - Running tests — full suite vs. single spec, and the
E2E_*env overrides. - Spec & helper conventions — the mocha skeleton, the helper barrel, core
WebdriverIO patterns (getWorkbench,executeWorkbench, webview iframe
switching,waitUntilwithtimeoutMsg, notifications, selectors). - Invoking commands hidden from the palette — how to handle actions
contributed with"when": "false"(tree-item / webview-only) and assert their
wiring viagetCommands(true). - What cannot be tested deterministically — flows that require specific live
GitLab state (e.g. an approvable MR) or a linked workspace repo. - Recordings — why a video can come out empty (the reporter records the test
body, notbefore/afterhooks; invisible API calls produce no frames), how
to force-capture a passing run, and how to sanity-check frame count. - Debugging — Allure report, failure videos/screenshots, wdio logs, live
DevTools DOM inspection, and rate-limit-vs-real-failure triage.
How has this been tested?
- You can test the skill with Duo chat - ask it to
run e2e test with the video recording for this community contribution https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/3215 - or copy the skill in the
.claude/skills/ui-feature-testing, runclaudeand promptrun e2e test with the video recording for this community contribution https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/merge_requests/3215
Once its done, ideally you'd have the link to the video recording of the feature. You'll be prompted about the cleanup later .
I have tested simple UI scenarios - checking that agent platform icon is there, that auth link in the list - and both Duo and claude could do that.
- If
src/browserorsrc/commonhas been modified, please consider interoperability with the Web IDE. See Running the Extension in WebIDE. - Consider an end-to-end test for significant new features that aren't covered by integration tests.
Screenshots (if appropriate)
What CHANGELOG entry will this MR create?
- None - other non-user-facing changes
Edited by Olena Horal-Koretska
