lib: Avoid declaring zero-length VLAs in various messaging functions
As reported in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=252157, on recent versions of FreeBSD 13.0-CURRENT with clang/llvm 11.0.0, the various samba ports (net/samba411
, net/samba/412
and net/samba413
) sometimes dump core when shares are accessed.
This turned out to be due to variable length arrays (VLAs) being used in source3/lib/messages.c
, where sometimes the length would end up being zero, which is undefined behavior. For example, in messaging_recv_cb()
there is a VLA fds64[]
, being declared using incoming num_fds
parameter, which is sometimes zero, resulting in:
Program received signal SIGSEGV, Segmentation fault.
0x0000000801c784a7 in messaging_recv_cb (ev=0x805475060, msg=0x7fffffffdbe8 "\035#", msg_len=98, fds=0x7fffffffdbdc, num_fds=0, private_data=0x80546e300) at ../../source3/lib/messages.c:394
394 int64_t fds64[MIN(num_fds, INT8_MAX)];
(gdb) print num_fds
$6 = 0
To fix this, I propose to use MAX(1, number)
to ensure the VLA is always declared with a length of at least 1. In the FreeBSD port case, this fixes the aforementioned core dumping occurrences.
I found two other places in source3/lib/messages.c
using VLAs and incoming length parameters, which can also be fixed in the same way.
This was reported in https://bugzilla.samba.org/show_bug.cgi?id=14605.
Checklist
-
Commits have Signed-off-by:
with name/author being identical to the commit author -
(optional) This MR is just one part towards a larger feature. -
(optional, if backport required) Bugzilla bug filed and BUG:
tag added -
Test suite updated with functionality tests -
Test suite updated with negative tests -
Documentation updated -
CI timeout is 3h or higher (see Settings/CICD/General pipelines/ Timeout)
Reviewer's checklist:
-
There is a test suite reasonably covering new functionality or modifications -
Function naming, parameters, return values, types, etc., are consistent and according to README.Coding.md
-
This feature/change has adequate documentation added -
No obvious mistakes in the code