This is to track progress of switching Xonotic to the daemon engine: - [ ] Language - [ ] Sandboxing - [ ] Builtins # Language Currently daemon can't run our gamecode since it uses C++/NaCl and we use QC/QCVM, otherwise it is very similar in design (e.g. can use our assets) but based on slightly newer tech. The plan is to compile QC to C(++) or rust and run that sandboxed in daemon. - We have 2 (or 3) options 1. Rewrite everything manually and gradually - part of out code would be in QC and part in c(++)/rust - the former part would keep getting smaller and the latter bigger. We'd compile the QC portion to c++/rust and that to WASM and our rust portion to WASM separately and somehow link/load them together in daemon. This requires transpiling but not into readable code (see jcompile). 2. Convert QC to c(++) or rust once and use the generated code as our main source. This mean the transpiler needs to produce readable output including handling macros in a sane manner (see qc2rust). 3. (Technically, we could add QC support to daemon but QC is a dead end lang responsible for a lot of bugs and we need to move from it sooner or later.) - [jcompile](https://github.com/TimePath/jcompile) - A QC compiler that tries to output QC bytecode, c++ and lua - Only the Q1VM backend compiles currently, but the output is not tested - The frontend/grammar needs updating to work with latest qcsrc. - Uses kotlin 1.0, the project needs updating to work properly in latest intelliJ IDE - @TimePath says the compiler is not designed to generate readable code (it completely ignores macros and compiles everything as one file, throws away comments and formatting) - [qc2rust](https://gitlab.com/martin-t/qc2rust) - tries to produce readable rust code - preserves comments and macros - still very unfinished, contributions welcome - see [roadmap](https://gitlab.com/martin-t/qc2rust/blob/master/README.md#roadmap) in readme - [Daemon compatibility layer](https://gitlab.com/xonotic/daemon-glue) - contains a QC VM for testing that dp and daemon builtins work the same, no idea what else - @Lyberta rewrote the QCVM into modern c++: https://gitlab.com/ftz/qcvm - contains mostly unimplemented builtins # Sandboxing - Daemon currently uses NaCl (WASM support is WIP). Rust supports WASM as a target, not NaCl afaik. - If qc2rust is usable before daemon is ready to support xon (doesn't have sandboxing or builtins) we might wanna get the benefits of a saner lang earlier - @morosophos proposed adding WASM support to darkplaces. - Might wanna investigate how much work it would take since it would allow us to test the transpiler before committing to daemon. - Also could help with mixed client/server compatibility. - For testing qc2rust, we could just use native bindings. QC clients might even connect to rust servers. - Available WASM runtimes (for DP or daemon): - [wasmi](https://github.com/paritytech/wasmi) - in rust, interpreter, used for cryptocurrency, probably not suitable - [wasmer](https://github.com/wasmerio/wasmer) - in rust, JIT, has C/C++ bindings - [wasmtime](https://github.com/CraneStation/wasmtime) - in rust, JIT, has ([standard-ish](https://github.com/WebAssembly/wasm-c-api)) C/C++ bindings, readme says it doesn't have Spectre mitigations - [lucet](https://github.com/fastly/lucet) - in rust, compiler and runtime, has C bindings, [security overview](https://github.com/fastly/lucet/blob/master/SECURITY.md), [Spectre](https://news.ycombinator.com/item?id=19514971) - [WAMR - intel WASM micro runtime](https://github.com/intel/wasm-micro-runtime) - in C # Builtins - These are functions in the engine that QC can call - https://gitlab.com/xonotic/daemon-glue/tree/master/src/vm/sys - Not all dp builtins are used - those that are need to be either implemented in daemon or QC needs to be changed to not use them # Testing - Should probably extend [sv_game](https://gitlab.com/xonotic/xonotic-data.pk3dir/blob/master/.gitlab-ci.yml) test to cover more code (add mutators, change cvars using defer, ...) - Not a good idea to switch engine + sandbox + language at the same time - hard to track down bugs. - Could record engine/gamecode interactions ([entry point](https://gitlab.com/xonotic/xonotic/wikis/Introduction-to-QuakeC#entry-points) calls, [builtin](https://gitlab.com/xonotic/xonotic-data.pk3dir/tree/master/qcsrc/dpdefs) calls, global state) from sv_game, then "replay" it to the transpiled code # Mixed client/server compatibility This would be nice to avoid risking a split in the community. Currently they're not compatible but it should be theoretically doable. The engine defines just the protocol, gamecode defines the content of the communication. If the engines were running the same gamecode (just transpiled into a different lang) it should be possible to have either a proxy that translates between the two or make daemon support the dp protocol directly. # Alternatives - The [FTEQW](http://fte.triptohell.info/) engine - has a (nearly?) compatible protocol, uses slightly extended QC, still Q1 technology - Improving dp - [darkplaces-upgrade-plan](https://gitlab.com/Visse44/darkplaces-upgrade-plan) - [horsepower](https://gitlab.com/horsepower/engine) - intends to maintain backwards compatibility with dp