Skip to content

Some PKCS#11 modules return EC_POINT in DER encoding

While the PKCS#11 is clear that the EC_POINT for ECDSA and EDDSA keys should be "Public key bytes in little endian order", some HSMs return the public key in DER encoding, resulting in 59 bytes for Ed448 keys. This explodes further down the road when the sequoia-openpgp tries to convert this slice to the fixed-length buffer for public key:

https://gitlab.com/sequoia-pgp/sequoia/-/blob/main/openpgp/src/crypto/mpi.rs?ref_type=heads#L609

While this is an issue of the HSM, this is very common issue and the Sequoia Cryptoki should be able to handle this oddball as the specs is quite vaguely written and many implementers got it wrong.

Given that for the public keys we have fixed length for each of the types, the encoding should be easy to distinguish with few magic numbers. Other option might be to try to decode the ASN1 and if it fails, just take the bytes as they are. This is what we use in the pkcs11-provider:

https://github.com/latchset/pkcs11-provider/blob/main/src/objects.c#L880

(this is in contrast with the ECDSA keys that need to be DER encoded)

Looking at the code, I was not sure what would be the best plate to fix this so leaving as an issue now.