Switcher-io
Summary
Switcher-io, Switcher's new Socket.io server
Motivation
Right now, there exists no easy way for a front-end interface to communicate with Switcher. Scenic, a React-based webapp (currently the only known front-end for Switcher), must use Scenic Core to do so, which is a separate Node.js server. Scenic Core communicates with Scenic using the Socket.io protocol, and relays requests to Switcher using Node-Switcher, which is a custom Javascript wrapper around C++ API calls.
This solution is far from ideal, since this introduces two additionnal layers in the stack (Scenic Core and Node-Switcher). This in turn increases maintenance and development costs, complexifies the installation process and multiplies the potential sources of bugs. Additionally, any new front-end for Switcher must either develop its own method of communication with the back-end (using websockets, HTTP requests, or something else) or conform to the Scenic Core API, which was designed predominantly for the Scenic use case.
In recent years, the creation and standardisation of Switcher's new Python API (PyQuid) opened new possibilites for communicating with Switcher. Using one of the widely used Websockets library for Python, it would be possible to write a Python server to completely replace the Scenic Core/Node-Switcher duo with something far more lightweight, easier to maintain and directly integrated with Switcher.
Proposal
This RFC is about adding a new Websocket server in Switcher, named switcher-io. The aim of this server is to be as lightweight as possible and to act as a wrapping layer around Switcher, so that any webapp can communicate with and control a running instance of Switcher using a clear Websocket API, based on the existing Python API.
The server would essentially act as a bidirectionnal bridge between Switcher and the front-end, and would make heavy use of PyQuid. In fact, the goal is to roughly have a 1:1 equivalency between a Websocket route and a PyQuid call.
Apart from Switcher, the server will mainly use 2 libraries to do its work:
-
python-socketio: this is a Python implementation of Socket.IO, a Javascript library for quickly building robust Websocket clients and servers (it is also the same library currently used by Scenic to communicate with Scenic Core). The Socket.IO library and its Python implementation are both FOSS under the MIT license, are well documented and benefit from a wide adoption by developers. Since Scenic already uses the same library, it also has the added benefit of bascially being plug-and-play with Scenic's current API.
-
aiohttp: this is a Python package for deploying an efficient asynchronous HTTP server or client. It is actively maintained since 2013, is FOSS (Apache v2) and is also well-documented. It is supported out-of-the-box by python-socketio.
Some work has already been done in separate branches for this new proposal. The feat/switcher-ws
branch contains a basic working version of the server, while the feat/qwiss-file-api
and feat/qwiss-method-api
feature branches focus on adding specific functionalities (these two last branches are out-of-date)
The server will be located in the wrappers/websockets
directory. (Note that in the current branches, the server is located in /tools
instead of /wrappers
)
The structure of the directory will be:
- wrappers
- websockets
- examples # These are also unit tests
- 01-basic.py
- 02-infotree.py
...
- CMakeLists.txt
- README.md
- server.py
The server.py
will contain the list of Socket.io routes and the main loop for the server.
Dependencies for the server are currently handled with Pipenv.
Here is a high level overview of the routes (API) categories planned for switcher-io. Some of them are already implemented in the existing feature branches on this repo. Each category includes calls going from the client to the server, but also, in some cases, calls going from the server to the client(s):
Category | Prefix | Description |
---|---|---|
Quiddity | quiddity. |
Creating and deleting quiddities |
Property | property. |
Getting and setting properties |
Methods | method. |
Invoking a quiddity method |
Nickname | nickname. |
Getting and setting quiddity nicknames |
InfoTree | tree.info. |
Getting values from the InfoTree |
UserTree | tree.user. |
Getting, grafting and pruning the UserTree |
Switcher | switcher. |
General introspection, setting custom bundles, etc. |
Session | session. |
Handling Switcher sessions (loading, saving, etc.) |
Impact on already existing code
This work will be mainly self-contained, since the goal is not to change the existing APIs, but rather to build upon it.
Some feature requests for PyQuid may be submitted at a later time, if changes to PyQuid prove to be necessary. However, most of the necessary changes to the API have already been done beforehand, so very few changes to the Python API are expected.
There are however two exceptions to the above:
-
Switcher's file handling capabilities will need to be expanded in order to fully implement the Session API. These changes will impact Switcher's handling of sessions. A feature request for this has already been filed: see https://gitlab.com/sat-metalab/switcher/-/issues/11.
-
A new
extraConfig
key-value pair will need to be added in Switcher's configuration file (switcher.json
). ThisextraConfig
will allow front-ends such as Scenic to specify additional configuration files to be read by Switcher. A feature request has also been created for this specific feature: see https://gitlab.com/sat-metalab/switcher/-/issues/12
Obviously, switcher-io will deprecate the Scenic Core/Node-Switcher stack, and Scenic is expected to migrate to this new server as soon as it is in place.
Roadmap
-
RFC approval and modification, if necessary -
Merge branch feat/qwiss-method-api
infeat/switcher-ws
-
Delete branch feat/qwiss-file-api
- The FileAPI contained in that branch has been evaluated as a security risk, and is no longer viable
-
Remove existing Network API from feat/switcher-ws
- It has been evaluated as a security risk and is therefore no longer desirable
- It will be replaced by the
systemusage
quiddity (see issue https://gitlab.com/sat-metalab/switcher/-/issues/9)
-
Update the existing switcher-io code to make it compatible with the most recent Switcher version - Switcher 2.2.0 introduced breaking changes with the removal of the quiddity name.
- The current switcher-io code was developed with Switcher 2.1.39
-
Implement the new session api for Switcher (see https://gitlab.com/sat-metalab/switcher/-/issues/11) -
Implement the new extraConfig
API in Switcher (see issue https://gitlab.com/sat-metalab/switcher/-/issues/12) -
Implement the rest of the Socket.io API in the feat/switcher-ws
branch -
Add examples/unit tests -
Submit MR of feat/switcher-ws
intodevelop
for review -
Merge branches
More details
Keep in mind that Socket.IO != Websockets.
Websocket: TCP communication protocol allowing bidirectional communication between a server and a client. The Websocket protocol is a standard specified in RFC6455. It is supported by all modern browsers.
Socket.IO: Open source library (MIT) allowing real-time bidirectional communication between a server and a client. Socket.IO does this by using the Websocket protocol internally, but it can fallback to alternative methods (AJAX requests, long polling...) in the rare cases where the browser doesn't support Websocket (ex.: Opera Mini, old Internet Explorer versions). Socket.IO also supports broadcasting, namespaces and other features that are not part of the basic Websocket protocol, but are extremely useful for handling server/client communication. Socket.IO offers two libraries: one for clients and one for servers.
High-level comparison between the two (in french): https://www.educba.com/websocket-vs-socket-io/