Lua UTF-8 conversion probably doesn't work for code points above 65536 on Windows
The following discussion from !3950 (merged) should be addressed:
-
@AnyOldName3 started a discussion: @Kuyondo are you still around? This is broken on Windows as
wchar_t
is UCS2/UTF-16 there thanks to Microsoft implementing support for Unicode before UTF-8 or UCS4/UTF-32 were invented (for some reason, The Unicode Consortium thought sixteen bits were enough to represent every human alphabet for a while). The conversion needs to go via a fixed-size type big enough to hold a whole code point, i.e.char32_t
.!3327 (comment 1542230814) looks relevant. We use C++20 now (although not all of it yet as some of the older compilers in CI only support a subset) so
std::codecvt<char32_t, char8_t, std::mbstate_t>
is available.