## Problem to solve Reviewing code by reading a file top to bottom is insufficient. This is because code isn't written to be read top to bottom, or page by page like a novel. Source code is terse and deduplicated. Functions and libraries allow reuse, and are linked together in a large and complex graph of dependencies. ~~Reading~~ Understanding code is about understanding these relationships. Understanding the relationships between changed code to other changed code, or unchanged code is difficult. Features like **jump to definition** or seeing a list of places that call a specific function are very helpful, and tools like `ctags` which dates to BSD 3.0 (1980) show the long history and importance of such tools. ## Vision Code navigation (and search) should work by default on every GitLab instance, so that administrators don't have to install, configure, and manage an additional tool for this critical capability. ## Further details In GitLab 12.5, [**Sourcegraph**](https://sourcegraph.com) contributed https://gitlab.com/gitlab-org/gitlab/issues/20642 which is wonderful because it provides code navigation features for every public project on GitLab.com :tada: However, the open source version of Sourcegraph does not support code intelligence (jump to definition, references etc). Thus, for code nav to work for all projects on all instances of GitLab cannot ship Sourcegraph as part of Omnibus or the K8s charts. The [Language Server Protocol](https://microsoft.github.io/language-server-protocol/) was developed by Microsoft, and the [LSIF format (Language Server Index Format)](https://github.com/Microsoft/language-server-protocol/blob/master/indexFormat/specification.md) has been developed with collaboration between Microsoft and Sourcegraph. The LSIF format is used by Microsoft, Sourcegraph and Kibana for code intelligence. :white_check_mark: GitLab should use the LSIF format for code intelligence given it's quick adoption, growing language support, and ability to be generated in a CI pipeline (e.g. AutoDevOps). ## Competitive Analysis - XXXX-XX-XX: BitBucket apprears to indirectly support annotations/navigation through the marketplace https://devpost.com/software/code-navigation-for-bitbucket - 2019-06-11: [GitHub releases jump to definition](https://github.blog/changelog/2019-06-11-jump-to-definition-in-public-repositories/) - 2019-07-08: [SourceHut releases annotations](https://drewdevault.com/2019/07/08/Announcing-annotations-for-sourcehut.html) There are a number of tools that generate different kinds of protocols/annotation tools that make got to definition, autocomplete and the like possible. - https://github.com/universal-ctags/ctags - https://github.com/github/semantic - https://oracle.github.io/opengrok/ - https://microsoft.github.io/language-server-protocol/ - https://github.com/kythe/kythe - https://about.sourcegraph.com/ ## Proposal :warning: Initially limited to the default branch to avoid problems of duplicate storage for each feature branch, or needing to merge ASTs to deduplicate. Standing on the shoulders of LSIF, GitLab should implement support for: - displaying **Jump to Definition** and **Show references** natively in GitLab using the uploaded LSIF data - GitLab will need to parse the LSIF file, and make the contents available in the repository browser and diff - [LSIF](https://code.visualstudio.com/blogs/2019/02/19/lsif) upload/download via API and AutoDevOps - generate using GitLab CI, uploaded to a new API, and then made available to via an API for local editor use (see https://code.visualstudio.com/blogs/2018/12/04/rich-navigation) - update AutoDevOps templates to include LSIF generation and upload - relying on CI provides support for very large number of runtimes Challenges: - learning about the LSIF format - exposing it efficiently via API (probably GraphQL only?) - storage policy for raw LSIF files (use S3, but they become out of date) - handle stale LSIF indexes - estimating DB storage requirements of this kind of data ## Buyer tier Code navigation is useful for individual contributors, either browsing their own code, or reviewing the code of a colleague. It makes the individual more efficient, provides a more IDE like experience, and allows individuals to understand code more deeply faster. Given this, a basic version supporting **jump to definition** and **find references** should be available in GitLab Core (free, open source). ## Links / references - https://github.com/sourcegraph/sourcegraph/tree/master/lsif#architecture - https://github.com/sourcegraph/sourcegraph/blob/master/lsif/docs/datamodel.md