## The Problem We want to implement real-time issue boards but the current way of doing real-time (polling with ETag caching) would not work well with it. We'd have to track a lot of things in an issue board (lists changing, issues within lists changing, etc..) and polling for each of those isn't feasible. It is possible to use just one polling endpoint, but it just makes the code harder to understand. Even with other pages like the MR page for example, we have multiple polling requests happening: one for the title and description, one for notes, one for the MR widget, etc.. So this could potentially be useful in other places too. With polling, it's also not very real-time unless we lower the polling interval. There's also a plan to implement real-time collaboration and for that we'd need faster updates. We decided to explore Pub/Sub messaging systems because it would make these simpler. On a page, we could subscribe to multiple channels / events and they would go through one connection. This could also potentially take some load off Redis and our web servers when we eliminate those polling requests. We'd have extra load going to the websocket server and Redis for Pub/Sub but I think that's more efficient so we'd still see a gain overall. ## Other Alternatives ActionCable was the first choice because it is included with Rails. Though scalability is a known concern from reading blog posts. But it's a good thing that there's anycable which implements the same API so we could switch to that in the future with minimal to no changes in the application code. 1. Long-polling / SSE This has the same problem above with having to poll / request multiple endpoints. Also, even if we do this, we'd have to implement some custom backend logic similar to our current ETag caching that checks Redis or something similar. And it doesn't seem worth it when there's ActionCable that provides the full stack. There's the `message_bus` gem though that implements this multiple subscriptions in one polling endpoint. But since we're planning to do real-time collaboration which would need lower latencies and bi-directional communication, I think it's better to just go with websockets directly. 1. Go / Erlang / Elixir websocket servers While it is known that these languages are way better than Ruby at concurrency. But without booting our Rails app / Ruby libraries, we can't reuse the code we already have. For example: permissions checks. These are complex and very easy to get wrong so we definitely don't want to re-implement this. We could do a separate API call to our Rails backend but more on that below. 1. Anycable Anycable has websocket servers in Go / Erlang and it solves the problem of not having Rails context by using gRPC. The downside is that we'd have to spin up another gRPC server which boots our Rails app. This complicates our infrastructure setup and would take longer to setup everything that's needed. This option was discussed in [this issue](https://gitlab.com/gitlab-org/gitlab/issues/21249). Since it is easy to switch to this later on if needed, we decided to defer this and start with ActionCable. 1. Other Ruby websocket servers (Faye) Gitter uses this and we looked into this briefly but we didn't really have a strong reason to choose this over ActionCable. ## Choosing how to run ActionCable There are 2 ways to run the ActionCable server: embedded mode with the existing web servers (Unicorn/Puma) or standalone mode. In embedded mode, ActionCable hijacks the connection and lets its own thread pool handle processing of messages so that the existing web workers / threads are not blocked. This is the default mode and saves us some memory because we don't need to start another process that boots Rails. In standalone mode, same thing happens but it's now handled in a separate Unicorn/Puma server. We [decided to do standalone first](https://gitlab.com/gitlab-org/gitlab/-/issues/17589#note_289622519) for isolation as we test out this new service / feature. In the future, it is possible to make this configurable in Omnibus. We could even make embedded mode the default. ## How it's setup in our Rails application This is subject to change as we work on this. 1. Our ActionCable endpoint is set to `/-/cable` 1. Workhorse is set to proxy requests to `/-/cable` to the `cableBackend` if specified. If not, it just proxies to `authBackend`. This makes it support both embedded and standalone mode. 1. The ActionCable server is a Puma server run with a different rackup file found in `cable/config.ru` 1. In these ActionCable channels (similar to controllers), you could do anything you could do in the web context like using our models, reading from cache, etc.. so it is important that this process is treated like our existing web process and should have the same configuration and should be able to connect to the DB, Redis cache, shared state, sidekiq, etc.. Although we probably would just be doing permission checks in the initial implementation, it could be a source of weird bugs in the future if these dependencies aren't setup properly.