[embhorn]

Description

Two inputs let a certificate escape an excluded name constraint subtree:

• Empty excluded dNSName. excluded: DNS: (zero-length), the CA/Browser Forum encoding for a sub-CA that may not issue DNS names, was treated as matching nothing. Per RFC 5280 4.2.1.10 it now matches every DNS name, in the native and OpenSSL compat paths. A lone "." base is treated the same way unless OPENSSL_COMPATIBLE_DEFAULTS is
  defined.
• Quoted local-part rfc822Name. "x@y"@evil.com escaped excluded: email:evil.com because two '@' characters made the name "invalid". The mailbox is now split at its last '@'. An '@' outside a quoted local-part makes the name uncomparable, and the certificate is rejected on both permitted and excluded subtrees.

The subject CN fallback for DNS constraints now applies only to CNs with host name syntax, so an empty excluded dNSName does not reject S/MIME or client-auth leaves whose CN is a person's name.

Fixes zd22724

Testing

• New and updated cases in tests/api/test_asn.c, tests/api/test_ossl_x509_ext.c and tests/unit-mcdc/test_asn_ext_whitebox.c.
• asn and ossl_x509_ext groups and the white-box smoke run pass with --enable-all and with --disable-openssl-compatible-defaults.
• TLS 1.3 end to end with the example client and server: the bypass chains are now rejected with -198, and the valid quoted-mailbox and person-name-CN chains are still accepted.

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[github-actions]

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m0plus

• FLASH: .text +260 B (+0.4%, 70,939 B / 262,144 B, total: 27% used)

gcc-arm-cortex-m3

• FLASH: .text +248 B (+0.2%, 131,673 B / 262,144 B, total: 50% used)

gcc-arm-cortex-m4

• FLASH: .text +192 B (+0.1%, 208,624 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m4-baremetal

• FLASH: .text +256 B (+0.3%, 73,591 B / 262,144 B, total: 28% used)

gcc-arm-cortex-m4-crypto-only

• FLASH: .text +192 B (+0.1%, 181,779 B / 262,144 B, total: 69% used)

gcc-arm-cortex-m4-dtls13

• FLASH: .text +256 B (+0.1%, 195,352 B / 1,048,576 B, total: 19% used)

gcc-arm-cortex-m4-min-ecc

• FLASH: .text +192 B (+0.3%, 68,185 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m4-openssl-compat

• FLASH: .text +192 B (+0.0%, 797,244 B / 1,048,576 B, total: 76% used)

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +192 B (+0.1%, 222,868 B / 262,144 B, total: 85% used)

gcc-arm-cortex-m4-pq

• FLASH: .text +192 B (+0.1%, 312,016 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m4-rsa-only

• FLASH: .text +256 B (+0.1%, 342,920 B / 1,048,576 B, total: 33% used)

gcc-arm-cortex-m4-sp-math

• FLASH: .text +192 B (+0.3%, 68,185 B / 262,144 B, total: 26% used)

gcc-arm-cortex-m4-tls12

• FLASH: .text +256 B (+0.2%, 132,433 B / 262,144 B, total: 51% used)

gcc-arm-cortex-m4-tls13

• FLASH: .text +256 B (+0.1%, 248,994 B / 262,144 B, total: 95% used)

gcc-arm-cortex-m7

• FLASH: .text +256 B (+0.1%, 208,688 B / 262,144 B, total: 80% used)

gcc-arm-cortex-m7-pq

• FLASH: .text +256 B (+0.1%, 312,912 B / 1,048,576 B, total: 30% used)

gcc-arm-cortex-m7-tls13

• FLASH: .text +256 B (+0.1%, 249,058 B / 262,144 B, total: 95% used)

stm32-sim-stm32h753

• FLASH: .text +320 B (+0.2%, 204,908 B / 2,097,152 B, total: 10% used)

zephyr-frdm_rw612

• FLASH: log_const_area -8 B, rodata -8 B, text +1,848 B (+0.6%, 284,296 B / 67,108,864 B, total: 0% used)

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[embhorn]

Fix confirmed by customer