Skip to content

rsa: fix verify_recover size query, digest output and no padding - #517

Open
ColtonWilley wants to merge 2 commits into
wolfSSL:masterfrom
ColtonWilley:fix/rsa-verify-recover
Open

ColtonWilley wants to merge 2 commits into
wolfSSL:masterfrom
ColtonWilley:fix/rsa-verify-recover

Conversation

@ColtonWilley

Copy link
Copy Markdown
Contributor

wp_rsa_verify_recover failed the NULL-buffer size query, returned the DigestInfo instead of the digest, and rejected RSA_NO_PADDING. In replace-default mode this breaks pyca/cryptography recover_data_from_signature(), openssl pkeyutl -verifyrecover, and OpenSC C_Verify for CKM_RSA_X_509 and all RSA-PSS mechanisms (CKR_GENERAL_ERROR). It now matches the default provider; verified with real pyca and OpenSC pkcs11-tool --verify on a virtual CAC card.

The NULL-buffer size query failed, breaking pkeyutl -verifyrecover and
pyca recover_data_from_signature(). With a digest set, check the
DigestInfo and output only the digest; support RSA_NO_PADDING, used by
OpenSC C_Verify for CKM_RSA_X_509 and RSA-PSS. Matches the default provider.
Size query, digest-only output, malformed DigestInfo and no padding,
against the default provider.
Copilot AI balanced review requested due to automatic review settings October 26, 2026 22:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cryptographic recovery changes need human review and runtime validation across supported wolfSSL/OpenSSL configurations.

No open issues 🎉

What changed in this PR

This PR updates RSA signature recovery in the wolfSSL provider to improve compatibility with OpenSSL’s default provider and consumers such as pyca/cryptography and OpenSC.

Changes:

  • Supports NULL-buffer size queries and RSA_NO_PADDING recovery.
  • Validates DigestInfo and returns only the digest when a digest algorithm is configured.
  • Adds regression tests for recovery output, malformed inputs, buffer sizes, and default-provider compatibility.
File Description
test/​unit.h Declares the new recovery test.
test/​unit.c Registers the new recovery test.
test/​test_rsa.c Adds recovery and compatibility regression cases.
src/​wp_rsa_sig.c Updates recovery behavior, validation, and API documentation.

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants