Skip to content

Draft: Internationalization with fluent-cpp

Benjamin Winger requested to merge bmwinger/openmw:fluent-cpp into master

I finally got back to working on this. This merge request supersedes !620 (closed) and implements the same thing using a new C++ Project Fluent library that I've written: https://gitlab.com/bmwinger/fluent-cpp.

See !620 (closed) for discussion of the other options for internationalization. To add on to the end, as my last comment may not have been entirely clear in terms of my intentions, I basically decided that ICU MessageFormat is too much of a hassle to work with, mostly due to a complete lack of direct support in tools like pontoon. You basically either have to edit messageformat txt files manually, or convert them to xliff using a java library, and then convert them back once translation is done, which rules out the sort of continuous workflow that would be possible with pontoon and other localization frameworks (and while xliff is supported by pontoon, it's not supported particularly well since it's a container and pontoon doesn't support the underlying messageformat language. I think most of the other tools are in a similar situation, as the one tool I found which supports MessageFormat only supports MessageFormat yaml files for the javascript library). I decided to try my hand at writing my own Fluent library for C++, and it turned out to be relatively straightforward once I'd found a good parser-combinator library.

It's still lacking support for a few of fluent's language features (just a matter of time), and is quite new and rough, but it does work.

Note there is a dependency on ICU (which is causing build failures on windows at the moment. I'm not sure what the best way of getting the ICU dependency there is as it doesn't seem to be on chocolatey). Fluent-cpp currently only uses it for parsing locale identifiers, and the code on the OpenMW side of things uses it for locale detection. Boost::locale could also do both of these things, and while fluent-cpp will eventually need something for localizing numbers (another thing I was planning on using ICU for), boost::locale can also do that, so if it's a lot of work to get ICU working as a dependency on all platforms, boost::locale could likely take its place.

Other things which need to be done:

  1. Making use of this for the other localizable strings: Lots of these and it could easily be done in later merge requests. I could probably do a few more now though, perhaps internationalize the rest of the CLI help messages just so that there's a decent number that use this.
  2. Pontoon. As I mentioned previously at some point, I have a pontoon instance set up here and I could set up an openmw project there (I already have one pointing at this branch). It would need push permissions for the repository, or alternatively a separate repository could be set up which is pulled in during compilation (I currently have it set up with an API token for my @guar account).
  3. Support for localizing mods with fluent: Including this here as a reminder that it's part of the purpose of this, but it will require a bunch of additional work, so I'll write up an issue where it can be discussed.

Merge request reports