feat: expose type-aware partial evaluation (TPE) as tpe_authorize

[h0rv]

Rebased onto main now that #108 has merged. Eight commits: the five that were here, plus three that pick up your review changes.

• 956bd9a adds tpe_authorize
• 4090666 completes the TPE surface and settles the context contract
• 7f2e75e is a typing cleanup with no behavior change, easy to drop if you would rather it went in on its own
• 32cf41d adds the entity_uids-on-a-residual tests you asked for, and pins the difference between the two residual views
• 611b50c trims the prose this branch adds to match the rest of the library
• ff28445 moves the TPE documentation into docs/guides/type-aware-partial-evaluation-guide.md and cuts the README section down to one example and a pointer, following the doctrine you codified in CLAUDE.md while reviewing feat: expose the PST as typed nodes via policies_to_pst #108
• b4167c6 states the TPE contracts where your feat: expose the PST as typed nodes via policies_to_pst #108 changes state the PST ones: the pydocs and Rust doc comments now name the error each entry point raises and its condition, TpeAuthzResult gets the class docstring it lacked, and CLAUDE.md records the tpe feature, the TpeResponse schema borrow, and the two-residual-views gotcha
• 715747b is the same plain-language pass you did on the feat: expose the PST as typed nodes via policies_to_pst #108 prose, applied to this branch's docstrings, doc comments, and the new guide

What it adds

tpe_authorize, an entry point for type-aware partial evaluation. Use it when you know the type of the principal or resource but not which one: some User, not yet sure which, rather than a concrete User::"alice".

This is separate from is_authorized_partial. It does not call it, change it, or share its response shape. The two partial-evaluation implementations in cedar-policy differ in a way that matters here: is_authorized_partial's residuals can hold untyped unknown(...) nodes that PST cannot represent (see #108 and #107), while TPE's residuals are checked against the schema and do convert. Confirmed by calling to_pst() on real output from both.

Inputs

principal and resource each take one of:

• a concrete entity, as 'User::"alice"' or {"type": "User", "id": "alice"}
• a type whose id is unknown, as "User", pst.EntityType("User"), or {"type": "User"}

action must be concrete and takes either concrete form. The dict form matters for the same reason it does on is_authorized: it accepts entity ids that Cedar's surface parser rejects as needing normalization.

schema is required, unlike is_authorized and is_authorized_partial where it is optional, because TPE builds its request and entity types against it.

context follows is_authorized_partial. Leaving it out, or passing None, means the context is unknown, so a policy reading it stays residual. {} means a known-empty context. This is what PartialRequest itself documents (None "will result in a residual for partial evaluation"). An earlier revision of this branch mapped None to Context::empty(), which quietly meant known-empty and left no way at all to ask for an unknown context, so the same argument name meant opposite things in the two functions. Fixed in 1c73e03, with tests for unknown, explicit, and empty.

What comes back

A TpeAuthzResult. permits and forbids stay separate, each a TpeClassification of residual_ids, true_ids, false_ids, error_ids. residual_policies maps a policy id to a cedarpy.pst.Template, the typed node from #108, not JSON.

Every failure raises ValueError. Unlike is_authorized, there is no decision to fall back to when the input itself cannot be resolved.

Binding the unknowns later

The point of TPE is that you re-evaluate only the residuals once you know the rest, not the whole policy set. That was missing from the first revision, which returned the residuals and stopped:

decided = result.reauthorize(
    {"principal": 'User::"alice"', "action": 'Action::"view"', "resource": 'Doc::"d1"'},
    entities=...,
)

It returns an ordinary AuthzResult. entities defaults to whatever the tpe_authorize call ran against. This goes through the engine's own TpeResponse::reauthorize, so Cedar checks the concrete request and entities against the partial ones first: a request naming a different principal than the one TPE was given raises, rather than returning a decision the partial evaluation never sanctioned. cedarpy.tpe_reauthorize(...) is the same thing as a free function, taking the inputs explicitly.

result.residual_policy_set is every residual as a reusable PolicySet handle, for driving evaluation yourself. It drops straight into is_authorized.

Which entities a residual still needs

This is what pst.entity_uids from #108 is for, and the case its own tests did not cover until 56ebb65. A residual is a pst.Template, so you can ask it what to load before finishing the evaluation:

policies = '''
    permit(principal, action == Action::"view", resource)
    when { resource.owner == User::"bob" };
'''

result = tpe_authorize('User::"alice"', 'Action::"view"', "Doc", policies, "[]", schema)
entity_uids(result.residual_policies)
# frozenset({EntityUid(type=EntityType(basename='User', namespace=()), id='bob')})

The two views of the residuals are not interchangeable, which is easy to trip over. residual_policies holds only the policies still undecided, as the evaluator reduced them, with the concrete parts of the request already substituted in. residual_policy_set holds every residual, including the ones that came out concretely true, false or erroring, each keeping its original scope. So the entities they name differ, and being a handle rather than nodes, the latter needs to_pst() before entity_uids will take it. 56ebb65 pins all of that.

One implementation note: TpeResponse<'a> borrows the schema, so it cannot be stored in a #[pyclass] without a self-referential struct. reauthorize therefore re-runs TPE from the inputs the result carries, and then reauthorizes. That keeps the engine's consistency checks, with no unsafe. If you would rather avoid the second TPE pass, residual_policy_set is the cheap path, minus those checks.

Entities that are only partly known

A concrete entity set asserts that every entity's attributes, parents, and tags are known. TPE also accepts a document where an entity exists but one of those is not loaded yet, so policies reading it stay residual. That is what lets a caller work out what to fetch before fetching it, and the first revision could not express it:

from cedarpy import PartialEntities

result = tpe_authorize(
    'User::"alice"', 'Action::"view"', 'Doc::"d1"', policies,
    PartialEntities.from_json([{"uid": {"type": "Doc", "id": "d1"}, "parents": []}]),
    schema,
)

Each of attrs, parents, and tags must be wholly present or wholly absent per entity, and a parent entity cannot itself have unknown parents. That is the engine's rule, not ours. Plain concrete entities still work exactly as before.

The tpe feature

Enables tpe alongside partial-eval in Cargo.toml. git diff Cargo.lock is empty: cedar-policy's tpe = ["cedar-policy-core/tpe"] and cedar-policy-core's tpe = [] have no dependency edges. 4.12.0 puts partial-eval and tpe in the same experimental bucket, and cedarpy already ships partial-eval as its documented partial-authorization API.

Turning tpe on is also what makes pst.ResidualError real, so this PR adds it back as a node type. TPE emits a subexpression it knows will error as a call to error(), and cedar's PST intercepts that name and returns Expr::ResidualError, but the interception is behind the feature. #108 drops the node for that reason, and the variant audit it adds is feature-aware, so with tpe enabled here it fails unless the node is modelled. That is the audit doing its job across the two branches.

Your #108 review corrected the pst half of the 4.12 changelog entry. This branch corrects the other half: the sentence said the engine's breaking changes were confined to features cedarpy does not enable, listing tpe, so it now reads "did not then enable".

The typing cleanup commit

caa252a is separable and changes no behavior. It brings cedarpy's own annotations in line with what the functions actually accept and return: PEP 604 and 585 syntax throughout, the raw-JSON dicts the result wrappers hold are parameterized (Mapping[str, str] for the @id annotation maps, Mapping[str, int] for metrics) instead of a bare dict, and the entities and schema normalization no longer rebinds a parameter to a type its own annotation forbids. cedarpy.__all__ now declares the public surface, so import * stops re-exporting json, copy, and the typing imports. mypy cedarpy/ goes from 10 errors to clean.

Testing

This machine has no Rust toolchain any more, so I could not rebuild the extension locally after the rebase. CI is the verification for everything that needs compiled code.

Check	Where	Result
pytest tests/unit	CI, all five platforms	314 passed, 14 subtests
pytest tests/integration/test_cedar_integration_tests.py	CI, all five platforms	78 passed
mypy cedarpy/	local	clean
git diff Cargo.lock	local	empty

The corpus suite and cargo clippy do not run in CI and I cannot run them here. At the pre-rebase tip they were 60801 passed and 10 clippy warnings, the same 10 as on main, none in this diff. Read those two as verified at 60d097f, not at the current tip. The rebase changed no Rust behavior, only doc comments.

The tests cover a type-only resource, a concrete resource resolving allow and deny, a satisfied forbid overriding a satisfied permit while the permit stays in true_ids, a residual forbid stopping an otherwise-true permit from becoming a decision, the residual as a typed node with a working pattern match, trivial residuals not repeated in residual_policies, is_authorized_partial behaving exactly as before, and errors for a missing schema, an unparseable principal, a bare-type action, unparseable policies, and unparseable entities.

The later tests cover context unknown against explicit against empty, all four entity-uid input forms plus an id the surface parser rejects, residual_policy_set feeding is_authorized, reauthorization reaching allow and deny, a contradicting request being refused, the correlation id surviving, the free function, a hand-built result refusing to reauthorize, unknown entity attributes staying residual, the same entities fully known resolving, and a partial document requiring concrete entities to reauthorize.

b4167c6 adds one more. PolicySet.from_pst raises TypeError on a node tree holding pst.ResidualError. read_expr has no arm for it, so that was already the behavior, undocumented and untested.

The one example left in the README is byte-identical to test_type_only_resource_produces_a_residual, so a test covers it. The guide's examples are the ones the README carried before this rebase.

Not run: make benchmark-compare, same reason as #108.

[h0rv]

@skuenzli Thanks for the other review - this one is rebased and ready for review 😄

[skuenzli]

Hey Robby - I haven't forgotten about this PR. Thanks for your patience. I'll review it as soon as I can.