fix: test failure in pkd against dropbear with libgcrypt
Bug Description:
When libssh is built with the libgcrypt, the RSA hostkey signature generated during the ECDH_REPLY stage fails to match the RSA modulus length (rsalen) if the signature value contains leading zeros.
For a 3072-bit key where the modulus (rsalen) is 384 bytes, libgcrypt produces a 383-byte signature (0x17f =383 bytes) (sig_len), which is one byte shorter. Dropbear expects the signature to be exactly the size of the modulus length treats this as an invalid, and aborts the connection with “Bad hostkey signature,” causing torture_pkd_dropbear_e256_rsa_aes256_ctr to fail.
Changes:
- In
pki_do_sign_hash()(RSA path), extract the s value from the gcrypt S-expression and left‑pad it with zeros to rsalen - Validate modulus length, check that the signature blob is not larger than the modulus, and handle errors cleanly.
- Store the padded blob in
sig->raw_sig
Closes: #343
Checklist
- Commits have
Signed-off-by:with name/author being identical to the commit author - Code modified for feature
- Test suite updated with functionality tests
- Test suite updated with negative tests
- Documentation updated
- The project pipelines timeout is extended at least to 2 hours.
Reviewer's checklist:
- Any issues marked for closing are addressed
- There is a test suite reasonably covering new functionality or modifications
- Function naming, parameters, return values, types, etc., are consistent and according to CONTRIBUTING.md
- This feature/change has adequate documentation added
- No obvious mistakes in the code
Edited by Yağmur Çiçekdağı