## Summary We will help developers fix security problems in their code more efficiently by showing scan results before they even have to push code to GitLab. ## User experience/workflow needs ### The scanning process - We should scan **when a file is saved**. - At first, scan _only_ when files are explicitly saved. Reasoning: - We can always expand, over time, the number of triggers for scanning. But until we have a sense of real-world scan performance and scaling, it would be better to have a more predictable and low-volume load profile. - Scanning while someone is actively typing could actually create more confusion or frustration than it's worth: - The developer could still be exploring and not yet feel ready to analyze security results. - Practically, the code may be half-done and may not even be parseable. - **Low latency** is very important. - To an extent, the faster the better :slight_smile:. But to anchor in concrete terms, in reality the right ballpark is probably: - Acceptable: <5 seconds - Ideal: <2 seconds - Reasoning: Developers are more likely to move on than wait too long for scan results. - Users are likely to be less tolerant here than other workflows, since in this case users will not have the primary purpose of seeing scan results. And in cases like Code Suggestions, users are expecting something that will be explicitly helpful as their next step; the expected benefit of security scanning is less direct. - Scan results need to be **consistent** with the scans that will happen on merge requests (in CI/CD pipelines). - Note that "consistent" does not mean "identical". Concretely: - IDE-based scanning can return fewer results than the MR or default branch scan (in a CI/CD pipeline) will. - However, IDE-based scanning should not return a result that: 1. _Won't_ be detected in pipeline-based scanning. (That is, IDE-based scanning must return a subset of pipeline-based results.) 2. Has already been Dismissed in the Vulnerability Report. (This should rely on the same tracking semantics as are used in pipeline-based scans; this requirement does not mean that our tracking approach has to change.) - Custom rules should be consulted. - Latency and predictability are both more important than deep scanning coverage for this use case. This means that, among other tradeoffs, we can conduct less detailed analysis in order to either: - Reduce latency to a desirable level (discussed above). - Reduce the amount of approximation or estimation needed in this use case. - Specifically, the set of languages supported for real-time scanning can be more limited than those supported for pipeline-based scanning, and cross-file/cross-function scanning can also be omitted. ### Interacting with scans and results - Results should **decorate the open editor**. - Users should not have to switch to a completely separate view to see results, as they do with the current CI/CD-pipeline-based integration. - Note: Broadly speaking, the initial integration was a step in the right direction but does not dictate where we should take the user workflow going forward. - Results do not _need_ to be persisted after tabs are closed; if there are ways to save results, we should analyze the overall UX. - We must be able to give a quick high-level summary of a result due to the limited space available; ideally we would be able to link to related resources or open more details in a side panel/drawer, similar to the way results are handled in the existing scan-results IDE integration. - Consult with Editor Extension on any UI/UX patterns used. - We **do not need to allow vulnerabilities to be dismissed** in the editor. - Reasoning: This would add new surface area in an area where it's difficult to keep concepts/behavior clear (between pipeline vuln dismissals, default-branch vulns, and related workflows). Plus, Developer-level users are no longer going to be able to dismiss vulnerabilities directly by default in 17.0+ ([ref](https://docs.gitlab.com/ee/update/deprecations.html?removal_milestone=17.0#deprecate-change-vulnerability-status-from-the-developer-role)), so dismissal would have limited audience. - **Show scan status in a visible but unobtrusive way.** - It should be clear when scans have or haven't happened, but we should avoid interrupting users, showing red/error-level status, or otherwise getting in the way. - Reason: - When developers are actively working, code is much more likely to be in a broken state than it is in an MR where we scan today. - Most of our users will be _writing code_ as their primary task, meaning that they are not likely to be proactively seeking out security scan results. So, our UX "hooks" are less like "the user has explicitly asked for SAST results, and should be informed in a blocking way if they fail", and more like "developers are working; SAST is providing early feedback on security problems; and if it's too annoying it will be disabled". - Specifically, we will need to write documentation about how users can spot problems and fix them. We'll need to document at least the following cases: - Users are not authenticated. - A project is not entitled to Ultimate/is not entitled to scanning. - A scan has failed. ### Enabling/disabling/configuring scanning - Scanning must be **restricted to GitLab Ultimate**. - However, the feature must not cause problems for non-Ultimate users. (This is not hypothetical—the initial Secure IDE integration actually caused problems for non-Ultimate users due to the way the security-scan-results API calls were made and errors were handled.) - Scans should be **enabled by default for projects that already use SAST**. - But, to provide appropriate control: - Users should be able to disable scanning as a local preference in their IDE. - We could also _consider_ offering a similar setting at the project/group/organization/instance level to allow security teams to control the rollout of the feature. - Reasoning: - This is a way to avoid rejection of the feature in cases where it malfunctions or gets in the way of a developer's workflow. - Effectively, any malfunction would now have a workaround ("check this box to disable scanning in your iDE"), which would reduce the severity of any incident/escalation. - Aligning with whether SAST is enabled in the repository helps Security teams create a consistent security scanning experience in their organization. Having IDE-based scanning enabled in projects that don't already use pipeline-based SAST would seem rather surprising; it would ignore the fact that security teams typically try to apply different levels of scanning to different projects based on criticality, and could ignore any customizations that are typically applied in the org's scans. ## Technical/platform needs - To start, if this is a useful scope limitation, we **can target VS Code or the Web IDE**. However, any editor-side code needs to interoperate with Editor Extensions technical components/system architecture. - Reasoning: This should allow for expansion to additional IDEs and keep the editor extensions a consistent, maintainable customer experience. - There should be **as few local SAST-specific component installation steps as possible—ideally none**. - Reasoning: Development teams will not adopt IDE scanning (and Security teams will not push its adoption) if the feature is difficult to operationalize. - Note that IDE-based features pose special supportability concerns because they are run on individual machines (not as part of a single GitLab instance/runner). Organizations also would provide first-level support to developers; only a certain number of named contacts at each customer organization are able to raise tickets with GitLab Support. - We can require a **network connection** to scan. - We can use **cloud infrastructure** to enable quicker delivery/better scaling/etc. on GitLab.com. However, we will need to identify a **path forward to support self-managed** (including fully offline contexts). ## References * [BrownBag session on LSP-based IDE-integration](https://docs.google.com/document/d/1UnE57kr9XebwGyNp2hGTgdaR1B1LRDwqznjfhIJwhJE/edit). * [How to prototype (LSP) language servers and IDE integartion (Slides)](https://docs.google.com/presentation/d/1M5qrXt2_-vOCWlE6KnKCTFHrw6_2Iy9rRZLX_jdrcMc/edit#slide=id.p1) * [Road to production for LSP-based IDE integration(s)](https://gitlab.com/gitlab-org/gitlab/-/issues/407646#proposal) * [GitLab CodeSuggestions LSP Experiment](https://gitlab.com/gitlab-org/editor-extensions/experiments/gitlab-code-suggestions-language-server-experiment) * [GitLab CodeSuggestions move from Golang to Typescript (Epic)](https://gitlab.com/groups/gitlab-org/-/epics/10746) * [GitLab LSP Code Walkthrough (Video)](https://www.youtube.com/watch?v=bPsRl7_V5gU) * [Fast Code Scanning and Refactoring with Micro-grammars (Video)](https://drive.google.com/file/d/1KcOSTOotcLI7dAxGN7dzIMbil2qBh31I/view) * Previous Work (POCs): * https://gitlab.com/gitlab-org/secure/vulnerability-research/pocs/leftimus * https://gitlab.com/gitlab-org/secure/vulnerability-research/pocs/gitlab-mux * https://gitlab.com/gitlab-org/secure/vulnerability-research/pocs/gitlab-mux-vscode * https://gitlab.com/gitlab-org/secure/vulnerability-research/pocs/microscope