The JetBrains plugin doesn't use the Language Server yet.
Problem to solve
There is duplication of effort and inconsistency in implementation across the extensions. Any feature/improvement related to Code Suggestions currently needs to be implemented separately in 3 places: VS Code extension, Language Server, and JetBrains plugin.
Once VS Code uses the Language Server (gitlab-org&11723 (closed)), this will be brought down to 2 separate implementations.
By integrating the Language Server in JetBrains, we will have a single codebase to host most of the Code Suggestions logic.
This will increase efficiency for the addition of new AI features. The Language Server acts as a proxy to the AI backend, and provides shared capabilities that enable extensions to integrate AI features.
Goal
Investigate which approach would make it viable to use the Language Server in our JetBrains plugin.
I've did a quick research on how to ship a binary with the jetbrains plugin, and while there is nothing mentioning that that is not allowed, I did not see that pattern being used anywhere.
As an example, in order to integrate the database in RubyMine with Postgres, the IDE requires a .jar file that can provide a client for Postgres. That would be a perfect candidate to be bundled with the IDE itself, but they opted to download and install them as a separate step.
That makes sense for a lot of reasons: you don't have to ship additional (optional) software, which would make your installation take longer, but it also avoids you from bundling a potential insecure piece of software: If a few months after the release a vulnerability is discovered in that bundled library, it means anyone installing that months old version will start with a vulnerable code, until it is updated.
By always downloading the dependency as a separate step, you can provide with the latest safe compatible version.
I believe that would be the desirable path on how to integrate an external component like that.
--
As a separate discussion, whether an LSP makes sense or whether makes sense to ship everything behind an LSP protocol:
On how to share code:
When you have to support a variety of platforms and share some code, you usually think about a "shared library" that contains what is common. If we were developing a windows application, that would likely be a .dll file, for a linux one, that could be a .so. This is common practice and allows you to dynamically call methods and procedures in that shared library from an external process.
To use a .dll or a .so you usually need to be coding both in the same languages, and expect the language to provide support for that (like when you use C or C++). When you want to interact with two distinct / incompatible languages, you need a translation layer or a compatibility layer / interface. The most widely used is FFI: https://en.wikipedia.org/wiki/Foreign_function_interface.
With FFI, languages like Ruby on JavaScript can use libraries written in C, C++, Rust as long as those libraries have implemented an FFI "interface".
Another option is something like "gRPC", which also allows code to be executed through the network, but can be used as a "local IPC" (through sockets). Whenever we use something that was designed for network, locally, it means we are paying extra latency costs. There is an analysis here: https://www.mpi-hd.mpg.de/personalhomes/fwerner/research/2021/09/grpc-for-ipc/
Because the gRPC is binary, it is still well suited for that job, it also allows a library to be implemented in a different language than the "client" one.
On whether LSP is the best choice for IPC/RPC:
The LSP was not designed to be a generic IPC/RPC protocol. It seems we are looking at LSP as if it were "REST/GraphQL" or a "generic RPC interface". I think if the goal is to build a shared library that contains the "shared functionality" that should exist among the different editors, we should think of it from a "library" perspective, and build it like so.
Provide all the correct interfaces for things like: "provide code suggestions for this file, repository, language, with this prompt", "analyze this file for vulnerability", "communicate with the API with this functionalities", "send this content as a crash report to our crash endpoint", and so on.
Then for each editor we want to support we "plug into the shared code" and call the right functions passing whatever context/state it is needed. This allows for the reusability without the limitations of relying solely on "LSP".
This also avoids us of trying to use an "LSP server" as if it was a "USB interface" that you can plug anything and it will somehow work.
That approach doesn't exclude us implementing an LSP for the cases where it makes sense (but the LSP server would then plug into the library, instead of it containing everything). This is important because it allows us to build the library in a language that is not the same that the LSP is being built, which can help with supporting the many OS, and architectures we are looking at. It also removes complexity as we would not have to ship and maintain nodejs across many different OS, make sure it is up-to-date, make sure it is using the latest secure version etc.
When you have to support a variety of platforms and share some code, you usually think about a "shared library" that contains what is common.
@brodock In my experience this is not easy and adds allot of risk/complexity. Also it would likely require a WASM solution for the WebIDE which has it's own drawbacks identified while investigating using typescript vs. golang.
Whenever we use something that was designed for network, locally, it means we are paying extra latency costs.
Yes, but the analysis you provided also concluded the benefits outweigh the latency. This is also my conclusion at the moment.
The language server supports being used over STDIN/STDOUT, which would likely lower the latency some over a local network connection.
So the question we need to ask is: Is the latency an actual problem? And if so, how big of a problem?
I should point out that VS Code works using LSP for all of it's syntax highlighting, code completions, etc. for all languages. This is also how our competitor for code suggestions works. This makes me think the latency is not something we need to be overly concerned with at this time.
After all, how noticeable is 100us-150us when the completions are taking seconds to comeback from the model gateway?
If in the future we find the latency overhead is too high, we could always switch to a different approach, such as a shared library.
The LSP was not designed to be a generic IPC/RPC protocol. It seems we are looking at LSP as if it were "REST/GraphQL" or a "generic RPC interface". I think if the goal is to build a shared library that contains the "shared functionality" that should exist among the different editors, we should think of it from a "library" perspective, and build it like so.
The LSP protocol was designed to allow a variety of integrations with an IDE, including code suggestions (see textDocument/completion). So we are not using it as a generic RPC interface, but rather as intended. The visual studio ide extension uses the code suggestions language server experiment with only 4 LSP messages: initialize, textDocument/didOpen, textDocument/didChange, and textDocument/completion.
The LSP protocol also allows custom messages to implement features outside of the specification. These messages start with $ token to distinguish them. So in a sense it also supports being used as a generic RPC interface.
It seems the exception to the rule here is the Web IDE. Perhaps we need a specific solution for it (something that runs on the browser) and something else for all the other platforms.
I know you can write JavaScript/TypeScript code that runs on the Web and in Node, but it's always a convoluted process, as you are limited to what APIs you can call due to the different "runtime environments" (it's the common denominator dilema again, you end-up with the "worst of both worlds"... a heavy library for the web with a bunch of shims and/or a not ideal solution for the desktop due to having to use only some APIs).
If we keep those two separate, we can build a "much better version" for the desktop that does not have to be a server of any sort (we can go for the binary library approach), building the "integration library" and for each editor we want to support we just need to build the "connector plugin" that calls into the libraries. This is the ideal solution for the desktop as there is no runtime environment we need to maintain, no proces to crash, restart, update, upgrade, watch for vulnerabilities, etc.
@brodock Thanks for the discussion, I think that's still an important discussion for the context of the language server itself because we will always want to have a language server be an output to allow other IDEs that don't have a plugin being developed to be covered and support Code Suggestions. How features get written inside it, it's up to us. Could/should we use a library inside the LS? Unclear.
If we know how to build such a library that is compatible out of the gate with the required contexts, that's definitely useful. I propose we move this point to an issue in https://gitlab.com/gitlab-org/editor-extensions/gitlab-language-server-for-code-suggestions Especially since there are pros/cons on both scenarios, so even if we roll out a first version without this, doesn't mean it might make sense down the road.
Write application service or manager which downloads binary and make it executable.
After download, if token is present then start the server. If token is not present then trigger server start when user enters token.
LSP Client:
Write down LSP client interfaces (need to research on this as JetBrains don't implement LSP)
Incorporate LSP client with Ghost Text API to trigger suggestions.
For first iteration, we are planning to keep it simple and consume suggestions api directly. In next iteration we can look into incorporating LSP in JetBrains.
After going through the article, I just want to highlight that LSP is supported by ultimate version only. So it will not be available for community version.
intellij { version = "232-EAP-SNAPSHOT" type = "IU"}
We may want to quickly verify whether or not https://github.com/huggingface/llm-intellij works in Community-edition IDEs. I couldn't find anything about this in the README.
@phikai@ali-gitlab I did a quick test now and installed the Huggingface plugin in InteliJ IDEA Community Edition. I didn't do the full setup because it requires a Huggingface API token, which I don't have.
Once installed, I typed some Kotlin code and triggered an immediate exception in the plugin: java.lang.NoClassDefFoundError: com/intellij/platform/lsp/api/LspServerManager
Expand for stack trace
Unhandled exception in [CoroutineName(com.intellij.codeInsight.inline.completion.listeners.InlineCompletionEditorListener), StandaloneCoroutine{Cancelling}@4fa6eeef, Dispatchers.Default] java.lang.NoClassDefFoundError: com/intellij/platform/lsp/api/LspServerManager at co.huggingface.llmintellij.LlmLsCompletionProvider$getProposals$2.invokeSuspend(LlmLsCompletionProvider.kt:31) at co.huggingface.llmintellij.LlmLsCompletionProvider$getProposals$2.invoke(LlmLsCompletionProvider.kt) at co.huggingface.llmintellij.LlmLsCompletionProvider$getProposals$2.invoke(LlmLsCompletionProvider.kt) at kotlinx.coroutines.flow.ChannelFlowBuilder.collectTo$suspendImpl(Builders.kt:320) at kotlinx.coroutines.flow.ChannelFlowBuilder.collectTo(Builders.kt) at kotlinx.coroutines.flow.internal.ChannelFlow$collectToFun$1.invokeSuspend(ChannelFlow.kt:60) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at com.intellij.openapi.application.impl.DispatchedRunnable.run(DispatchedRunnable.kt:43) at com.intellij.openapi.application.TransactionGuardImpl.runWithWritingAllowed(TransactionGuardImpl.java:208) at com.intellij.openapi.application.TransactionGuardImpl.access$100(TransactionGuardImpl.java:21) at com.intellij.openapi.application.TransactionGuardImpl$1.run(TransactionGuardImpl.java:190) at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:861) at com.intellij.openapi.application.impl.ApplicationImpl$4.run(ApplicationImpl.java:478) at com.intellij.openapi.application.impl.FlushQueue.doRun(FlushQueue.java:79) at com.intellij.openapi.application.impl.FlushQueue.runNextEvent(FlushQueue.java:121) at com.intellij.openapi.application.impl.FlushQueue.flushNow(FlushQueue.java:41) at java.desktop/java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:318) at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:792) at java.desktop/java.awt.EventQueue$3.run(EventQueue.java:739) at java.desktop/java.awt.EventQueue$3.run(EventQueue.java:733) at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:86) at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:761) at com.intellij.ide.IdeEventQueue.defaultDispatchEvent(IdeEventQueue.kt:690) at com.intellij.ide.IdeEventQueue._dispatchEvent$lambda$10(IdeEventQueue.kt:593) at com.intellij.openapi.application.impl.ApplicationImpl.runWithoutImplicitRead(ApplicationImpl.java:1485) at com.intellij.ide.IdeEventQueue._dispatchEvent(IdeEventQueue.kt:593) at com.intellij.ide.IdeEventQueue.access$_dispatchEvent(IdeEventQueue.kt:67) at com.intellij.ide.IdeEventQueue$dispatchEvent$processEventRunnable$1$1$1.compute(IdeEventQueue.kt:369) at com.intellij.ide.IdeEventQueue$dispatchEvent$processEventRunnable$1$1$1.compute(IdeEventQueue.kt:368) at com.intellij.openapi.progress.impl.CoreProgressManager.computePrioritized(CoreProgressManager.java:787) at com.intellij.ide.IdeEventQueue$dispatchEvent$processEventRunnable$1$1.invoke(IdeEventQueue.kt:368) at com.intellij.ide.IdeEventQueue$dispatchEvent$processEventRunnable$1$1.invoke(IdeEventQueue.kt:363) at com.intellij.ide.IdeEventQueueKt.performActivity$lambda$1(IdeEventQueue.kt:997) at com.intellij.openapi.application.TransactionGuardImpl.performActivity(TransactionGuardImpl.java:105) at com.intellij.ide.IdeEventQueueKt.performActivity(IdeEventQueue.kt:997) at com.intellij.ide.IdeEventQueue.dispatchEvent$lambda$7(IdeEventQueue.kt:363) at com.intellij.openapi.application.impl.ApplicationImpl.runIntendedWriteActionOnCurrentThread(ApplicationImpl.java:861) at com.intellij.ide.IdeEventQueue.dispatchEvent(IdeEventQueue.kt:405) at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:207) at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:128) at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:117) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:113) at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:105) at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:92) Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [CoroutineName(com.intellij.codeInsight.inline.completion.listeners.InlineCompletionEditorListener), StandaloneCoroutine{Cancelled}@4fa6eeef, Dispatchers.Default]Caused by: java.lang.ClassNotFoundException: com.intellij.platform.lsp.api.LspServerManager PluginClassLoader(plugin=PluginDescriptor(name=LLM, id=co.huggingface.llm-intellij, descriptorPath=plugin.xml, path=~/Library/Application Support/JetBrains/IdeaIC2023.2/plugins/llm-intellij, version=0.0.2, package=null, isBundled=false), packagePrefix=null, state=active) at com.intellij.ide.plugins.cl.PluginClassLoader.loadClass(PluginClassLoader.kt:156) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525) ... 46 more
There's also no trace of the LS having been installed locally on my machine. Maybe that only happens when the setup is fully completed.
Visual Studio uses https://github.com/vercel/pkg to create an executable version of the LSP that they can ship with the plugin. It's probably not feasible to do the same with JetBrains because we would have to bundle an LSP for every OS/arch with the plugin, which would result in a very large file.
@john-slaughter suggested we could have an install process that grabs the correct LSP, which seems like a viable approach.
@phikai@ali-gitlab I've updated the issue description with more context, and to call out that option 1 is currently considered unviable. Let's keep the description updated with summaries of findings, additional options, etc.