|
|
Gnulib
|
|
|
==
|
|
|
|
|
|
The gnulib portability layer found in many GNU packages continues to be an endless source of delight for cross-compiling developers. In particular, its attitude of "Let's assume the worst!" tends to make it replace large parts of the standard library, hoping to circumvent an obscure bug on an old rarely-used Unix. Often the attempted replacement code causes more trouble than it actually solves, worse, it often gets used even when it doesn't need to be used and potentially cause security issues as the safe standard library implementations are not used.
|
|
|
Gnulib is a portability layer found in many GNU packages and other software using the GNU build system. It is not a library, but rather a set of source files that are copied directly into programs. Gnulib performs three principal tasks:
|
|
|
|
|
|
As such, when you have a decent operating system, you often find yourself wishing for gnulib to just go away. Unfortunately, it's almost always heavily integrated into packages and often contains code that isn't portability wrappers, but actual reusable code that is part of the program.
|
|
|
* Providing implementations of missing standard library interfaces.
|
|
|
* Replacing buggy standard library interfaces.
|
|
|
* Providing common utility functions and other reusable code.
|
|
|
|
|
|
However, it's possible to actually make gnulib assume the best when cross-compiling to unknown systems. This largely fixes most issues with gnulib as the problematic replacement code doesn't get compiled in. This is done by setting these environment variables. In Sortix ports, this is normally done by pasting this whole block into the main configure script after the shebang line.
|
|
|
Gnulib is a portability layer but ironically fails to be portable, as it requires knowledge of implementation details of every supported operating system. It is not possible to write a perfect operating system according to the standards (and de-facto standards) and have all of gnulib just work. The gnulib files included n software releases tends to be rarely updated and bug fixes in gnulib can take several years to appear in most packages. Some parts of gnulib are unnecessary and cause trouble in the modern world as it attempts to fix bugs in irrelevant ancient Unix systems. Occasionally gnulib wraps standard library interfaces so code can be written with glibc assumptions. For instance, whether malloc(0) returns NULL or a unique pointer, which can't be detected at compile time, and the code should be rewritten to not make such assumptions (wrapping malloc might even make it more difficult for code analysis tools to detect bugs). Gnulib can occasionally cause more trouble than it solves. Gnulib tends to be very tightly embedded in the software packages and can't easily be removed or disabled. Since gnulib contains more than just portability code, i.e. utility functions, removing it will in the general case break the software. Gnulib may even indirectly have exploit mitigation counter-measures, as it prevents the standard library with exploit mitigations from being used, potentially making vulnerabilities even worse.
|
|
|
|
|
|
Adapting gnulib to Sortix
|
|
|
--
|
|
|
Gnulib can be especially bad when cross-compiling, as it assumes the very worst about the operating system when it can't perform runtime tests, which leads to the maximum amount of gnulib being enabled. Depending on how much of gnulib is included, in the worst case, gnulib might attempt to replace large parts of the standard library. The build system tends to assume that if it can't perform a runtime test, then you might have a very specific bug that got fixed in OpenBSD 1.4, even though you are not OpenBSD, and that OpenBSD 1.4 is horribly ancient. This behavior punishes good operating systems and rewards broken operating systems. The result is that compatibility code gets used, which needs to know about operating system details to possibly work, even though there's no reason whatsoever to use the compatibility on this operating system.The correct behavior should be assume the very best about unknown operating systems, and only assume a bug if the operating system is known to have the bug, and require the user to set the applicable environment variables with the true answers if gnulib guesses wrong. (I need to upstream patches for this). The below environment variables changes the gnulib defaults to assume the standard library works correctly, effectively disabling as much of gnulib's replacement function as possible.
|
|
|
|
|
|
The general policy is to upstream gnulib patches so it behaves reasonably on unknown operating systems. However, that's not always realistic and in practice gnulib has to be adapted and verified. Here is a partial check-list of things that should be done:
|
|
|
Overall, you will likely find yourself wishing for gnulib to just go away.
|
|
|
|
|
|
* Integrate the below long variable list into the configure script just after the shebang line to supress its suspicion of the standard library.
|
|
|
* Build the port (and don't clean its directory). Look at what object files got built. Look for any object files that replace standard library functionality. If you found something that shouldn't be used, look at the corresponding m4 file that does the check (for lib/foo.c that's often m4/foo.m4) and understand why it wsa included. The configure output and config.log might be of use. Perhaps a bad guess was made when cross-compiling (it should assume the best, not the worst), if so, add its cache variable to the below list.
|
|
|
* If the `lib/xmalloc.c module is used`, ensure `HAVE_CALLOC_GNU` got defined so calloc is the system calloc is trusted. If the package is buggy and the m4 file didn't get used in configure, you need to add `|| defined(__sortix__)` to the preprocessor conditional in the file.
|
|
|
* If the `lib/secure_getenv.c` module is used, then you have to #define `issetugid()` to 0 if `defined(__sortix__)`.
|
|
|
|
|
|
TODO
|
|
|
--
|
|
|
* What the fuck: freadptr.c, freadseek.c, freadahead.c and SLOW_BUT_NO_HACKS and sometimes aborting.
|
|
|
* What the fuck: utimens.c not correctly realizing Sortix does the awesome option.
|
|
|
* What the fuck: sprintf calls.
|
|
|
* What the fuck: If getgroups isn't available, `GETGROUPS_T` is assumed to int, not `gid_t`.
|
|
|
It's possible to actually make gnulib assume the best when cross-compiling to unknown systems. Doing so largely fixes most issues with gnulib as the problematic replacement code doesn't get compiled in. This fix is done by setting these environment variables. In Sortix ports, this is normally done by pasting this whole block into the main configure script after the shebang line.
|
|
|
|
|
|
Assuming the Best
|
|
|
--
|
|
|
|
|
|
Warning: This makes gnulib assume the best in a lot of cases, and makes it not do a proper check even if a runtime-check was possible. Trouble will potentially occur if you lie here
|
|
|
It's possible to actually make gnulib assume the best when cross-compiling to unknown systems by setting a large number of environment variables to the value not-buggy. Doing so largely fixes most issues with gnulib as the problematic replacement code doesn't get compiled in. In Sortix ports, this is normally done by pasting this whole block into the main configure script just after the shebang line.
|
|
|
|
|
|
Warning: This makes gnulib assume the best in a lot of cases, and makes it not do a proper check even if a runtime-check was possible. Trouble will potentially occur if you lie here.
|
|
|
|
|
|
This list disables all the cases where gnulib assumes a bug is present when unable to check:
|
|
|
|
... | ... | @@ -150,4 +143,22 @@ This list disables all the cases where gnulib assumes a bug is present when unab |
|
|
export ac_cv_func___fseterr=yes # Only if the OS actually supports this, Sortix does.
|
|
|
export gl_cv_func_getcwd_null=yes # Only if the OS actually supports this, Sortix does.
|
|
|
export gl_cv_struct_dirent_d_ino=yes # Only if the OS actually supports this, Sortix does.
|
|
|
export ac_cv_func_realloc_0_nonnull=yes # TODO: Investigate this, found in flex. |
|
|
\ No newline at end of file |
|
|
export ac_cv_func_realloc_0_nonnull=yes # TODO: Investigate this, found in flex.
|
|
|
|
|
|
Adapting gnulib to Sortix
|
|
|
--
|
|
|
|
|
|
The general policy is to upstream gnulib patches so it behaves reasonably on unknown operating systems. However, that's not always realistic and in practice gnulib has to be adapted and verified. Here is a partial check-list of things that should be done:
|
|
|
|
|
|
* Integrate the above variable list into the configure script just after the shebang line to suppress its suspicion of the standard library.
|
|
|
* Build the port (and don't clean its directory). Look at what object files got built. Look for any object files that replace standard library functionality. Due to the utility functions and mostly innocuous wrappers (malloc(0) for instance), it requires a close look to determine whether the object file is undesirable. If you found something that shouldn't be used, look at the corresponding m4 file that does the check (for lib/foo.c that's often m4/foo.m4) and understand why it was included. The configure output and config.log might be of use. Perhaps a bad guess was made when cross-compiling (it should assume the best, not the worst), if so, add its cache variable to the above list.
|
|
|
* If the `lib/xmalloc.c module is used`, ensure `HAVE_CALLOC_GNU` got defined so calloc is the system calloc is trusted. If the package is buggy and the m4 file didn't get used in configure, you need to add `|| defined(__sortix__)` to the preprocessor conditional in the file.
|
|
|
* If the `lib/secure_getenv.c` module is used, then you have to #define `issetugid()` to 0 if `defined(__sortix__)`.
|
|
|
|
|
|
TODO
|
|
|
--
|
|
|
|
|
|
* What the fuck: freadptr.c, freadseek.c, freadahead.c and SLOW_BUT_NO_HACKS and sometimes aborting.
|
|
|
* What the fuck: utimens.c not correctly realizing Sortix does the awesome option.
|
|
|
* What the fuck: sprintf calls.
|
|
|
* What the fuck: If getgroups isn't available, `GETGROUPS_T` is assumed to int, not `gid_t`. |
|
|
\ No newline at end of file |