# The Problem There is currently no naming convention for slash commands. Nor is there a convention of the default context they should be applied to. This leads to the proliferation of slash commands (e.g. in addition to `/explain` we are introducing `/explain_vulnerability`). Also the default context is different in different clients (independent of slash commands or user authored questions) i.e. in the web app we send the whole file if the user asks e.g. "what does this file do?", vs. in the IDE we only send the selected code and if no code is selected it doesn't work at all. Also the addition of additional context to LLM Powered features can be a powerful antidote to hallucination or irrelevancy, but choosing which context to add can be difficult to determine. Finding the right context within a large embedding index can be time consumptive. The more context you inject in the prompt, the more latency you add. Adding too much context can result in 'lost in the middle' problems, where the LLM loses the thread of the actual question. # Goal 1. _Define_ a naming convention for slash commands 2. _Define_ a default context that is intuitive and the same everywhere (or communicated well) 3. In those cases where a default context does not cover 80% of use cases **define** a context referencing pattern allowing users to explicitly state which context they want to have used. (This is probably the case for a lot of code related use cases in the IDE, e.g. code selection vs. file vs. open files vs. the entire repo. It may also be the case for scenarios in the web UI, e.g. summarize issue vs. summarize discussions vs. summarize discussion thread) 4. _Adjust the implementation_ of slash commands to match the naming and context convention defined in the three bullets above. ### Important references: * [Proposed list of slash commands verbs](https://docs.google.com/spreadsheets/d/1qhNF16eMJBjkCwWOkUg2k-aSuPqH-khjgHrKFSwksJE/edit?gid=778821004#gid=778821004) that should support a large set of features * [Pajamas MR defining the slash command patterns](https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/merge_requests/3909) # Slash commands available in Duo Chat ### Default slash commands :warning: Do not edit this table without consulting ~"group::duo chat". :warning: The below table represents the vision for the default slash commands, as agreed with the Duo Chat. Not all slash commands are implemented yet. This table is here for other teams to reference the vision for defaults, naming conventions, etc. | Command | Explanation | Status GitLab | Status IDEs | | ----------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | /new_conversation | clears history, starts with a blank screen | Not implemented ([issue](https://gitlab.com/gitlab-org/gitlab/-/issues/470818)) | Not implemented ([issue](https://gitlab.com/gitlab-org/gitlab/-/issues/470818)) | | /help | explains what Duo can do | Not implemented ([issue](https://gitlab.com/gitlab-org/gitlab/-/issues/462122)) | Not implemented ([issue](https://gitlab.com/gitlab-org/gitlab/-/issues/462122)) | | /troubleshoot | suggest a fix to a problem, but does not actually perform an action | Not implemented ([epic](https://gitlab.com/groups/gitlab-org/-/epics/14404)) | n/a | | /test | write tests for code | n/a | Partially implemented ([issue](https://gitlab.com/gitlab-org/gitlab/-/issues/471278), [epic](https://gitlab.com/groups/gitlab-org/-/epics/14407)) | | /explain | explains what content means in other words | Not implemented ([epic](https://gitlab.com/groups/gitlab-org/-/epics/14403)) | Partially implemented ([epic](https://gitlab.com/groups/gitlab-org/-/epics/14407)) | | /refactor | suggests improvements to code | n/a | Partially implemented ([epic](https://gitlab.com/groups/gitlab-org/-/epics/14407)) | | /summarize | summarizes an issue or epic, including the body and comments | Not implemented ([epic](https://gitlab.com/groups/gitlab-org/-/epics/14493)) | n/a | | /fix | suggests a fix to code | n/a | Implemented | ### Arguments default slash commands accept <details><summary> Click to see default slash command arguments </summary> | Command | Works without arguments (GitLab page) | Works without arguments (IDE) | Optional arguments (GitLab only) | Allows free text argument? | | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | | /new_conversation | every page | every page | n/a | n/a | | /help | every page | every page | n/a | n/a | | /troubleshoot | [failing job page](https://arc.net/l/quote/vadxipek) | [entire code file](https://gitlab.com/groups/gitlab-org/-/epics/14407), code selection | [url](https://gitlab.com/gitlab-org/gitlab/-/issues/471246) | free text allowed | | /test | Not available in GitLab | [entire code file](https://gitlab.com/groups/gitlab-org/-/epics/14407), code selection | n/a | free text allowed | | /explain | [blob code file page](https://gitlab.com/gitlab-org/gitlab/-/issues/471240), [vulnerability page](https://gitlab.com/gitlab-org/gitlab/-/issues/471242), [issue page, epic page](https://gitlab.com/gitlab-org/gitlab/-/issues/471243) | [entire code file](https://gitlab.com/groups/gitlab-org/-/epics/14407), code selection | [url](https://gitlab.com/gitlab-org/gitlab/-/issues/471247), [Issue ID (#123), epic ID (&123), vulnerability [vulnerability:123]](https://gitlab.com/gitlab-org/gitlab/-/issues/470705) | free text allowed | | /refactor | Not available in GitLab | [entire code file](https://gitlab.com/groups/gitlab-org/-/epics/14407), code selection | n/a | free text allowed | | /summarize | [issue or epic pages](https://gitlab.com/gitlab-org/gitlab/-/issues/472606) | Not available in IDE | url, Issue ID (#123), epic ID (&123) | free text allowed | | /fix | Not available in GitLab | code selection | n/a | free text allowed | </details> ### Deprecated slash commands Not all deprecations have been implemented yet. <details><summary>Click to see deprecated slash commands</summary> | Command | Explanation | | ---------------------- | --------------------------------------------------------------------------------------------- | | /vulnerability_explain | [Will be replaced by /explain](https://gitlab.com/gitlab-org/gitlab/-/issues/471242) | | /reset | [Will be replaced by /new_conversation](https://gitlab.com/gitlab-org/gitlab/-/issues/470818) | | /clear | [Will be replaced by /new_conversation](https://gitlab.com/gitlab-org/gitlab/-/issues/470818) | | /clean | [Will be replaced by /new_conversation](https://gitlab.com/gitlab-org/gitlab/-/issues/470818) | | /tests | [Will be replaced by /test](https://gitlab.com/gitlab-org/gitlab/-/issues/471278) | </details> # Initial proposal for context referencing We can leverage human-in-the-loop processes to leverage user's own contextual knowledge to smartly select context to be injected. For example, if a user knows that the code they are writing is based on a specific library, they could use @libraryx. This would then narrow the search space or would result in no search at all if the @reference narrows it down to something defined like a file. Define the space in which RAG configuration are expected to add context of value. There will be both deterministic and non-deterministic spaces for RAG injection. For example, @Current might look at the entire content of the current file for prompt injection (so the RAG retrieval is fully the users). Additionally, `@console` might provide the entire text of the console in the prompt injection (also fully defined by user). A non-deterministic example might be @libraryx which would an embedded index. When the user invokes the @libraryx call, we would narrow the vector match search to just that library and add just the semantically matched context from that library. Determining whether an object invocation would be deterministic or not could be based on size, the availability of an embedding index. In the UI we would need to make clear to the user when something in deterministic or not. #### Step 1: * Define the objects for which RAG configuration are expected to add context of value. To do this look at customer feedback, interviews, and competitor solution. * Determine whether those objects should be embedded or not (non-deterministic or deterministic) Some potential objects - @File (deterministic) - @ console (deterministic) - @repo (non-deterministic using embedding vectors) - @issues (non-deterministic using embedding vectors) - open windows - specific libraries Ideas for combining slash commands and objects: - `/debug @job/1` `/debug @pipeline/10` `/debug @merge_request/2` instead of a specific command just for pipeline root cause analysis: `/troubleshoot @pipeline/10` # Customization When users want to add additional invocations that are non-standard to the feature -- for example to their own documentation, proprietary code, etc -- they would have to declare those objects to Gitlab and set up @ handles for those objects. # Related work * [Discussion about naming convention](https://gitlab.com/groups/gitlab-org/-/epics/11978#note_1945571912) * [Collection of existing slash commands or slash commands being worked on that need to be cleaned up](https://gitlab.com/gitlab-org/gitlab/-/issues/467389#note_1949133216) * https://gitlab.com/gitlab-org/editor-extensions/gitlab-lsp/-/merge_requests/371#note_1932920455 proposes to define which context should and should not be included on the basis of a `.ai-context-policy.yml` file in the repo, but it is not clear if this is needed.