Integrate FGCom-mumble into FGFS

So, I took the weekend to investigate and make a plan.

Context

Referring to the evaluation of removal of classic (IAX based) FGCom (#3075), I want to open this ticket to discuss the feasability and coordinate potential efforts for integrating FGCom-Mumble directly into FGFS.

Integrating FGCom-mumble is by no means a small task, but in my opinion, it should be possible already. There needs to be no rush, FGCom-Mumble is aleady relatively easy to be installed and used as mumble plugin and FGFS addon. But an integration into fgfs would really ease the use of the system (the user just needs to click "connect") and may enrich multiplayer with realistic ATC and radio usage, and thus this is a good thing.

There was also already a little discussion on the mailing list, which was summarized at the FGFS FGCom-mumble wiki entry.

Overview

Here I now present the components (and a bit of discussion) in verbose detail, in my current state of knowledge.
I also whipped up a small diagram to show the big picture in my mind (a picture is worth a thousand words), showing the needed components and how they interact from a higher level.
Bear in mind, this is only my current state of knowledge without touching or deeply investigating FGFS code.

Mumble Server

The mumble server has a fixed binary protocol, which is encapsulated and accessible via libmumble. This binary protocol consists of API calls to the server (connect, join channel, ...), audio data (opus encoded UDP packets) and plugin-data (I think they are TCP).
Other users will connect to mumble and sharing data using this protocol. They can be arbitrary mumble clients (vanilla users, bots, fgcom-mumble users, ...).

The people providing/hosting the mumble server do in no means have FGCom-mumble knowledge. The idea here is to connect to the well-known radio-mumble.flightgear.fr service, which has moderation and also (to my knowledge) technical fallback methods should one server fail.

FGFS

Flightgear has a subsystem architecture and the idea is, that we write a completely new "FGCom-Mumble client" subsystem which ties all strings together as the single integration point.
My overall strategy would be to see this as a kind of "manager" and "glue" for the other systems. There are different integration dephts possible, but I would suggest, that we keep it simple and try to use the existing FGCom-Mumble mumble plugin code as much as possible. This way, we don't need to reinvent the wheel and can keep FGFS development and FGCom-mumble development mostly separate - we treat FGCom-mumble as a separate third party module.
So to speak: As FGCom-mumble is a mumble plugin, we need to write up a (libmumble based) mumble client supporting the needed subset of the mumble plugin architecture.

The subsystem needs to setup and manage FGCom-mumble's internal client data structure. Also it needs to setup libmumble, and broker the data between mumble and FGFS' audio system. It will need to use FGCom-mumble to alter incoming mumble audio streams before sending them to the audio system for playback.
For sending, the captured audio stream from the microphone needs to be encoded into opus and then handed to libmumble for putting it on the wire. Opening the mic must be coupled by a state change of FGCom-mumble (setting PTT of the radio and emitting a plugin message to other clients).
For receiving, the incoming audio stream must be decoded to PCM and then fed to FGCom-Mumbles mumble_onAudioSourceFetched function.

libmumble

libmumble is a lib which will be needed to be integrated into FGFS. Afaik the architecture is so, that it starts various threads which will call callbacks on the integrating application (our subsystem).

The first data type concerned is receiving audio data which must be fed into our existing audio system, so the user can actually hear something. Before we replay the audio stream however, we must route it through the FGCom-mumble core. And the microphone data needs to be encoded and handed over into libmumble for sending to the mumble server.

The other data type is plugin messages. Received plugin messages from libmumble need to be catched and fed into "FGCom-mumble core". We also need to catch "FGCom-Mumble core"s requests to send plugin data and hand them to libmumble.

FGCom-Mumble core

We need to treat it like any other 3rd party library.
Its internal state is changed by a) mumble plugin messages (to get knowledge of remote clients) and state from the simulator.

FGCom-Mumble serves as a sophisticated "incoming mumble audio stream modifier". It sits somewhat between libmumble and FGFS' audio system. It functions as "plugin" to the envisioned libmubmle client. libmumble is not altered directly, but its data streams are routed trough some functions of FGCom-mumble to make the radio simulation magic happen. Also, the FGCom-Mumble wants to share data with other users; it does this by a mumble standard interface called "plugin messages". From an integration perspective this is FGCom-Mumble just calls some mumble API functions which need to be implemented by the FGFS subsystem (they basicly just need to route the data from/to libmumble, however).

When building, we can safely skip some components: make CFLAGS+="-DNO_UPDATER -DNO_CFG -DNO_COMMENT" SSLFLAGS= plugin. (compiling FGCom-mumble is just a bunch of gcc's wrapped in a makefile, easing builds for linux, windows and mac).

A) Mostly decoupled scenario:
When initializing, FGCom-mumble will start threads implementing a UDP server. This server accepts the messages from the FGFS-Addon. If we just integrate the FGFS-Addon into FGData and always load that, this will result in FGFS generating UDP packets to a local port (16661). On that port the FGCom-Mumble UDP server will read the messages and alter its internal knowledge of the com-devices.
B) More integrated scenario: Like A), but we do not build the UDP server or UDP client (make CFLAGS+="... -DNO_UDPCLIENT -DNO_UDPSERVER"). Instead, the subsystem updates the internal state of FGCom-Mumble: The internal state is a global struct currently (see global_vars.h). A solution might be to periodically read the UDP-packet strings which are generated from the FGFS-Addon (they are written into the prop tree!) and hand it over to the UDP-Servers parse functions. This way, state changes are autodetected and properly initiate plugin messages to other clients, and the subsystem does not need to know low-level logic of internal structures.
Also, it needs to periodically read the RDF data and hand it to the addons property tree. The nasal code of the FGFS addon receives this and do all of its magic.

FGCom-Mumble addon

The FGCom-mumble FGFS addon is not showed on the diagram, but is also a part which needs to be taken care of. Basicly, the addon generates UDP packet strings for the plugin, which are traditionally sent to a local UDP port. This port was previously opened by the FGCom-mumble plugin running in the mumble instance. When integrating without UDP server (option B) like described before), we can still use these messages, but relay them directly to the udp server parsing function, which will then ensure proper internal state and remote client updates. This way, the subsystem does not need to know anything about FGCom-mumbles internals but we also need no open UDP port.

For features like RDF, the addon detects and relays the RDF messages to the individual radios, which then do nasal stuff to set some properties (like the ADF needle or receiving-flag). That is all already finished and working.

The FGFS-addon should be integrated into FGData and "silently" always autoload as ordinary addon.
Maybe it's worth to also rework the addon a bit and integrate its code more "directly", so its not visible as addon in the prop tree - but that could easily be left for optimizing the future in my opinion.

An alternative would be to recode the UDP message generation stuff into the C++ subsystem part. That would probably be easy and obsolete the addon alltogether. The global aim of the addon was to generate and receive UDP messages to/from the mumble plugin, but since the subsystem has access directly, it can generate the packets directly without the need for nasal.
In this case, the RDF features must be recoded too.

The settings dialog need to be reworked/enhanced also (maybe we should have a new distinct dialog for managing the libmumble client). It at the very least needs to be enhanced to make libmumble connections easy (and here we sadly reinvent mumbles wheel):

Roadmap

This could be a rough plan to make it happen:

Edited Mar 03, 2025 by Hbeni