defense in depth hardening for x509 extension create by OBJ and EVP decode update

[JacobBarthelmeh]

ZD21992

[Copilot]

Pull request overview

This PR adds defensive hardening to OpenSSL-compat APIs by preventing negative-length input in base64 decode and by eliminating a use-after-free hazard when reusing an X509 extension’s own ASN1 object during extension recreation.

Changes:

• Reject negative inl in wolfSSL_EVP_DecodeUpdate() to avoid length underflow/casts.
• In wolfSSL_X509_EXTENSION_create_by_OBJ(), duplicate the source ASN1 object before freeing any existing object to prevent alias-driven UAF.
• Add API tests covering both the negative-length decode case and the self-object reuse case for X509 extension creation.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
wolfcrypt/src/evp.c	Adds inl < 0 validation to harden EVP base64 decode update.
src/x509.c	Duplicates extension object earlier to prevent freed-memory reads when caller passes ex’s own object.
tests/api/test_ossl_x509_ext.c	Adds regression test for extension recreation using its own object pointer.
tests/api/test_evp.c	Adds regression test ensuring negative inl is rejected and outl is cleared.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

MemBrowse Memory Report

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pkcs7
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• linuxkm-pie
• linuxkm-standard
• stm32-sim-stm32h753