Clang builds can cause failures when packing buffers during session

Originally reported by @davidwed in !599 (closed) with proposed fix that really did not solve the issue, but worked around it. For some reason, I was not able to reproduce it locally nor in CI, but the problem is present and might depend on toolchain configuration.

The cause is the conversion if uint32_t (return value from ssh_buffer_get_len()) to size_t (argument of ssh_buffer_pack()) and back to uint32_t inside of that function. This worked before I introduced a stricter check and explicit cast in b7018c17. The compiler does not complain about this issue, as the arguments to the buffer_pack are variadic.

Suggested solutions:

  • Explicitly the returns of ssh_buffer_get_len() to size_t in all the arguments to the P type of buffer_pack() functions
  • Use temporary size_t variables
  • ?

Some debugging is already done in !607 (closed). The reproducer is available in https://gitlab.com/davidwed/libsshex

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information