Standards: enforce RFC 7493 noncharacter prohibition across JCS inputs

[samjanny]

Summary

RFC 7493 section 2.1 says that object member names and string values in arrays and object members MUST NOT contain Unicode Noncharacters. RFC 8785 constrains JCS input to I-JSON and requires I-JSON validation before a received value is acted upon.

Entangled currently enforces strict UTF-8 and rejects isolated surrogates, but its Unicode 15.0 assigned-only gate applies only to user-visible fields subject to NFC. Section 07 explicitly permits opaque state values to contain "any other Unicode scalar values permitted by the wire schema". Submit-body fields values and request_state.value are also arbitrary UTF-8 strings and are JCS-canonicalized for transaction.request_hash. A noncharacter such as U+FDD0 is therefore admitted in at least those paths even though it is outside the I-JSON input domain on which RFC 8785 relies.

This is a verified standards-compatibility gap, not an ambiguity between conformant readings of the current Entangled text.

Ground-truth passages

• RFC 7493 section 2.1: object member names and string values "MUST NOT include code points that identify Surrogates or Noncharacters": https://www.rfc-editor.org/rfc/rfc7493.html#section-2.1
• RFC 8785 section 3.1 says input data MUST be adapted for I-JSON; section 5 requires received data to be checked for I-JSON before signature verification: https://www.rfc-editor.org/rfc/rfc8785.html#section-3.1 and https://www.rfc-editor.org/rfc/rfc8785.html#section-5
• Entangled rc.62 §04 lines 115-143 enforce strict UTF-8 and malformed-surrogate rejection, while lines 144-174 scope assigned-only enforcement to fields subject to NFC.
• Entangled rc.62 §07 line 376 permits opaque state values to contain any Unicode scalar value allowed by the wire schema.
• Entangled rc.62 §09 lines 139 and 182-224 JCS-hash submit bodies while allowing arbitrary UTF-8 fields and request_state values.

Recommendation

Add a document-wide and submit-body-wide prohibition on every Unicode noncharacter before canonicalization. The set is U+FDD0..U+FDEF and every code point whose low 16 bits are FFFE or FFFF through U+10FFFF. Apply it to every JSON string value and member name, not only user-visible/NFC fields.

Reject the signed-document case at Stage 5 as E_SCHEMA_FIELD_SYNTAX unless the maintainer prefers a dedicated code. E_SCHEMA_MALFORMED_UNICODE should not be reused without changing its definition: noncharacters are Unicode scalar values, not malformed UTF-8 or isolated surrogates. Submit bodies should reject the same condition during their schema/I-JSON validation.

Add at least:

• a correctly signed transaction whose opaque state value contains U+FDD0, expecting rejection;
• a submit-body validation test with U+FDD0 in a user field or request-state value;
• implementation unit tests covering U+FDD0, U+FFFE, U+1FFFF, and a nearby permitted scalar.

This changes validation behavior but adds no wire field and no diagnostic if E_SCHEMA_FIELD_SYNTAX is reused. It should land in its own rc and lockstep commit rather than being folded into the editorial §04 framing correction.