diff --git a/CHANGELOG.md b/CHANGELOG.md index 9bf4fbd..973afc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,14 @@ Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may inc - **Deep workflow composition (#327):** workflow graphs deeper than serde_json's 127-level recursion limit no longer silently collapse into SQL text. Nested children are deserialized one graph level at a time, and `df.explain()` now enforces the configured graph-depth limit before traversal. - **Silent Durofut envelope corruption (follow-up to #327):** `Durofut::ensure()` now fails loudly when a JSON object carrying a `node_type` cannot be deserialized (e.g. a non-object child) instead of silently wrapping the raw envelope as a SQL node that only fails at execution time. `df.explain()` no longer panics on an undeserializable child, raising a clean PostgreSQL error consistent with `df.start()`. +### Changed + +- **Conditional operators:** `?>` / `!>` now delegate operand normalization to `df.if()`, matching the function-call syntax and removing the duplicated internal `df.ensure_durofut()` validator. + +### Removed + +- **`df.ensure_durofut(text)`:** removed this undocumented internal helper. The `0.2.5 -> 0.2.6` upgrade replaces its operator callers before dropping it with `RESTRICT`; customer-owned dependent objects must be changed or removed before upgrading. + ## [0.2.5] - 2026-07-30 ### Added diff --git a/docs/upgrade-testing.md b/docs/upgrade-testing.md index ea5f2d2..294fcae 100644 --- a/docs/upgrade-testing.md +++ b/docs/upgrade-testing.md @@ -205,11 +205,13 @@ what the upgrade script handles, and any backward compatibility considerations. ### v0.2.5 → v0.2.6 -#### Preserve parser resource errors in `df.ensure_durofut()` -- **DDL change (function body only):** `df.ensure_durofut(text)` no longer catches `WHEN OTHERS` while deciding whether an operand is Durofut JSON or plain SQL. It still treats `invalid_text_representation` as plain SQL and re-raises its explicit unknown-node-type error, but PostgreSQL stack/resource errors now propagate instead of silently wrapping the serialized graph as a SQL node. The signature, volatility, search path, grants, and schema shape are unchanged. -- **Upgrade script:** `sql/pg_durable--0.2.5--0.2.6.sql` uses `CREATE OR REPLACE FUNCTION` with the same body emitted for fresh installs from `src/lib.rs`. This keeps Scenario A snapshots identical without dropping the function or changing dependent operators. -- **Scenario B1 considerations:** The new `.so` works against pre-0.2.6 schemas without runtime schema detection because no Rust SQL query or C symbol changed. Rust composers receive the opaque-child deserialization fix immediately from the new binary. Until `ALTER EXTENSION UPDATE` replaces the cataloged PL/pgSQL helper, the `?>` / `!>` operator path retains its older broad exception handler and can still misclassify a graph if PostgreSQL itself raises a stack/resource error while parsing it. -- **Scenario B2 considerations:** No data migration and no durable-state or replay change. Existing serialized graphs retain the same wire format. +#### Remove `df.ensure_durofut()` +- **DDL change:** Fresh installs no longer create the undocumented `df.ensure_durofut(text)` PL/pgSQL helper. `df.if_then_op()` now stores its condition and then-branch operands as text in the partial marker; `df.if_else_op()` extracts those operands and passes all three directly to the Rust-backed `df.if()`, which already performs Durofut normalization. +- **Upgrade script:** `sql/pg_durable--0.2.5--0.2.6.sql` replaces both operator helpers before dropping `df.ensure_durofut(text)` with `RESTRICT`. The new `df.if_else_op()` uses JSON text extraction, which accepts both new string-valued partial markers and object-valued markers emitted before the upgrade. `RESTRICT` deliberately aborts rather than silently removing a customer-owned object that depends on the undocumented helper. +- **Behavior change:** The operators now classify operands exactly like `df.if()`. In particular, JSON with an unknown `node_type` is treated as plain SQL during composition instead of being rejected by the former PL/pgSQL helper. +- **Scenario A considerations:** Fresh and upgraded schemas both omit `df.ensure_durofut(text)` and expose byte-equivalent `df.if_then_op()` / `df.if_else_op()` definitions, including their pinned `search_path`. +- **Scenario B1 considerations:** A binary-only update against any supported pre-0.2.6 schema leaves the cataloged PL/pgSQL helper and old operator bodies intact. They continue calling the unchanged `df.sql()` and `df.if()` C bindings; no binary symbol is removed because `df.ensure_durofut()` was not C-backed. B1 explicitly composes a `?>` / `!>` expression against every supported old schema. +- **Scenario B2 considerations:** No durable data or graph wire format changes. A partial `?>` value materialized before `ALTER EXTENSION` can still be completed with `!>` afterward. The upgrade can fail only when a customer-owned catalog object depends directly on `df.ensure_durofut(text)`; the operator helpers themselves are replaced before the drop. ### v0.2.4 → v0.2.5 diff --git a/scripts/run-pgspot.sh b/scripts/run-pgspot.sh index 94fd813..450b6b5 100755 --- a/scripts/run-pgspot.sh +++ b/scripts/run-pgspot.sh @@ -42,18 +42,21 @@ PGSPOT_ALLOW=( # example, `df.sql(...) ~> df.sql(...)`) so users do not need df in search_path. # pgspot reports the generated CREATE OPERATOR name as an unqualified object. '^PS017: Unqualified object reference: ~> at line [0-9]+$' - # Upgrade scripts CREATE OR REPLACE df.grant_usage()/df.revoke_usage()/ - # df.ensure_durofut() to migrate pre-existing installs. pgspot flags PS002 + # Upgrade scripts CREATE OR REPLACE selected df helpers to migrate + # pre-existing installs. pgspot flags PS002 # because a standalone upgrade script has no `CREATE SCHEMA df` to prove df is # extension-owned (the install SQL does, so it is not flagged there). # PostgreSQL 14.5+ blocks a CREATE OR REPLACE in an extension script that # would replace a non-extension object, so this is safe. Scoped to these - # three functions only. ensure_durofut's PS005/PS001/PS017 are fixed at the - # source (its search_path omits df — see the function's NOTE comment), so - # only its inherent PS002 needs allowing. + # these functions only. Their search paths and references are safe, so only + # the inherent PS002 needs allowing. ensure_durofut remains here solely + # because the released 0.2.4 -> 0.2.5 script still defines it; fresh 0.2.6 + # installs and the 0.2.5 -> 0.2.6 result remove the function. '^PS002: Unsafe function creation: df\.grant_usage\(p_role text,include_http boolean,with_grant boolean\) at line [0-9]+$' '^PS002: Unsafe function creation: df\.revoke_usage\(p_role text\) at line [0-9]+$' '^PS002: Unsafe function creation: df\.ensure_durofut\(val text\) at line [0-9]+$' + '^PS002: Unsafe function creation: df\.if_then_op\(condition text,then_branch text\) at line [0-9]+$' + '^PS002: Unsafe function creation: df\.if_else_op\(partial_if text,else_branch text\) at line [0-9]+$' ) # Whole codes to suppress globally (pgspot --ignore). Prefer PGSPOT_ALLOW. Empty. diff --git a/scripts/test-upgrade.sh b/scripts/test-upgrade.sh index 2135e92..6975e31 100755 --- a/scripts/test-upgrade.sh +++ b/scripts/test-upgrade.sh @@ -778,6 +778,10 @@ test_b1_dsl_chain() { assert_sql_contains "SELECT df.sql('SELECT 1') ~> df.sql('SELECT 2');" '"node_type":"THEN"' } +test_b1_conditional_operators() { + assert_sql_contains "SELECT ('SELECT true' ?> 'SELECT 1') !> 'SELECT 0';" '"node_type":"IF"' +} + # Verify that release_extension_owned_duroxide_objects de-registered all # duroxide objects from the extension. On a fresh install there are none; # on a v0.1.1-schema upgrade the BGW must have removed them before this runs. @@ -901,6 +905,7 @@ else run_test "B1 [v${B1_VERSION}]: df.version()" test_b1_version run_test "B1 [v${B1_VERSION}]: df.sql() construction" test_b1_dsl_construction run_test "B1 [v${B1_VERSION}]: DSL chain (~>)" test_b1_dsl_chain + run_test "B1 [v${B1_VERSION}]: conditional operators (?>/!>)" test_b1_conditional_operators run_test "B1 [v${B1_VERSION}]: df.start()/wait_for_completion()" test_b1_start_and_complete run_test "B1 [v${B1_VERSION}]: df.status() on real instance" test_b1_status_instance run_test "B1 [v${B1_VERSION}]: df.result()" test_b1_result diff --git a/sql/pg_durable--0.2.5--0.2.6.sql b/sql/pg_durable--0.2.5--0.2.6.sql index 24b1ec2..1b26579 100644 --- a/sql/pg_durable--0.2.5--0.2.6.sql +++ b/sql/pg_durable--0.2.5--0.2.6.sql @@ -6,26 +6,43 @@ -- See docs/upgrade-testing.md for the upgrade-script and backward-compatibility -- requirements (Scenario A / B1 / B2). -- --- Preserve stack/resource errors while classifying plain SQL operands. A broad --- WHEN OTHERS handler silently wrapped over-depth Durofut JSON as SQL. -CREATE OR REPLACE FUNCTION df.ensure_durofut(val text) RETURNS text AS $$ +-- The conditional operators now carry their first two operands as text until +-- !> completes the expression. df.if() performs Durofut normalization for all +-- three operands, removing the duplicated PL/pgSQL validator. +CREATE OR REPLACE FUNCTION df.if_then_op(condition text, then_branch text) RETURNS text AS $$ DECLARE - node_type_val text; + result_obj jsonb; BEGIN - BEGIN - node_type_val := (val::jsonb)->>'node_type'; - IF node_type_val IS NOT NULL THEN - IF node_type_val NOT IN ('SQL', 'THEN', 'IF', 'JOIN', 'LOOP', 'BREAK', 'RACE', 'SLEEP', 'WAIT_SCHEDULE', 'HTTP', 'HTTP_MULTIPART', 'SIGNAL') THEN - RAISE EXCEPTION 'Unknown node_type ''%''. Valid types: SQL, THEN, IF, JOIN, LOOP, BREAK, RACE, SLEEP, WAIT_SCHEDULE, HTTP, HTTP_MULTIPART, SIGNAL', node_type_val; - END IF; - RETURN val; - END IF; - EXCEPTION WHEN invalid_text_representation THEN - NULL; - WHEN raise_exception THEN - RAISE; - END; + result_obj := pg_catalog.jsonb_build_object( + '_partial_if', true, + 'condition', condition, + 'then_branch', then_branch + ); + RETURN result_obj::pg_catalog.text; +END; +$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp; + +CREATE OR REPLACE FUNCTION df.if_else_op(partial_if text, else_branch text) RETURNS text AS $$ +DECLARE + partial jsonb; + cond_text text; + then_text text; +BEGIN + partial := partial_if::pg_catalog.jsonb; - RETURN df.sql(val); + IF partial OPERATOR(pg_catalog.->>) '_partial_if' IS NULL THEN + RAISE EXCEPTION 'Invalid if-then-else: left side of !> must be a ?> expression'; + END IF; + + -- ->> accepts both the new text operands and object operands emitted by the + -- old helper, preserving partial expressions created before ALTER EXTENSION. + cond_text := partial OPERATOR(pg_catalog.->>) 'condition'; + then_text := partial OPERATOR(pg_catalog.->>) 'then_branch'; + + RETURN df.if(cond_text, then_text, else_branch); END; -$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp; \ No newline at end of file +$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp; + +-- RESTRICT is intentional: do not silently remove customer-owned objects that +-- depend on this undocumented helper. +DROP FUNCTION df.ensure_durofut(text) RESTRICT; diff --git a/src/lib.rs b/src/lib.rs index c4963bd..01dd9e1 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -741,76 +741,39 @@ CREATE OPERATOR | ( -- Helper: cond ?> then creates a partial if (stores condition and then branch) CREATE OR REPLACE FUNCTION df.if_then_op(condition text, then_branch text) RETURNS text AS $$ DECLARE - cond_fut jsonb; - then_fut jsonb; result_obj jsonb; BEGIN - -- Ensure both are durofuts - cond_fut := df.ensure_durofut(condition)::jsonb; - then_fut := df.ensure_durofut(then_branch)::jsonb; - - -- Return a special marker object for the partial if - result_obj := jsonb_build_object( + -- Keep operands as text until !> completes the expression. df.if() then + -- performs the same Durofut normalization as the function-call syntax. + result_obj := pg_catalog.jsonb_build_object( '_partial_if', true, - 'condition', cond_fut, - 'then_branch', then_fut + 'condition', condition, + 'then_branch', then_branch ); - RETURN result_obj::text; + RETURN result_obj::pg_catalog.text; END; -$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, df, pg_temp; +$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp; -- Helper: partial_if !> else completes the if node CREATE OR REPLACE FUNCTION df.if_else_op(partial_if text, else_branch text) RETURNS text AS $$ DECLARE partial jsonb; - else_fut text; cond_text text; then_text text; BEGIN - partial := partial_if::jsonb; + partial := partial_if::pg_catalog.jsonb; -- Check if it's a partial if IF partial->>'_partial_if' IS NULL THEN RAISE EXCEPTION 'Invalid if-then-else: left side of !> must be a ?> expression'; END IF; - cond_text := partial->'condition'::text; - then_text := partial->'then_branch'::text; - else_fut := df.ensure_durofut(else_branch); - - -- Now call the real df.if function - RETURN df.if(cond_text, then_text, else_fut); -END; -$$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, df, pg_temp; - --- Helper to ensure a value is a durofut (returns JSON string) --- Rejects JSON with unknown node_type values. --- NOTE: The valid node type list here must be kept in sync with --- VALID_NODE_TYPES in src/types.rs (the Rust constant is the canonical source). -CREATE OR REPLACE FUNCTION df.ensure_durofut(val text) RETURNS text AS $$ -DECLARE - node_type_val text; -BEGIN - -- Try to parse as JSON to check if it's already a durofut - BEGIN - node_type_val := (val::jsonb)->>'node_type'; - IF node_type_val IS NOT NULL THEN - -- Has a node_type - validate it - IF node_type_val NOT IN ('SQL', 'THEN', 'IF', 'JOIN', 'LOOP', 'BREAK', 'RACE', 'SLEEP', 'WAIT_SCHEDULE', 'HTTP', 'HTTP_MULTIPART', 'SIGNAL') THEN - RAISE EXCEPTION 'Unknown node_type ''%''. Valid types: SQL, THEN, IF, JOIN, LOOP, BREAK, RACE, SLEEP, WAIT_SCHEDULE, HTTP, HTTP_MULTIPART, SIGNAL', node_type_val; - END IF; - RETURN val; - END IF; - EXCEPTION WHEN invalid_text_representation THEN - -- Not valid JSON, treat as SQL - NULL; - WHEN raise_exception THEN - -- Re-raise our validation error - RAISE; - END; + -- ->> handles both the text operands emitted above and object operands + -- emitted by the pre-0.2.6 helper, so partial expressions survive upgrade. + cond_text := partial->>'condition'; + then_text := partial->>'then_branch'; - -- It's plain SQL, wrap it - RETURN df.sql(val); + RETURN df.if(cond_text, then_text, else_branch); END; $$ LANGUAGE plpgsql IMMUTABLE SET search_path = pg_catalog, pg_temp; @@ -1551,6 +1514,35 @@ mod tests { assert_eq!(fut.result_name, Some("my_name".to_string())); } + #[pg_test] + fn test_conditional_operator_accepts_legacy_partial() { + let result = Spi::get_one::( + r#"SELECT df.if_else_op( + pg_catalog.jsonb_build_object( + '_partial_if', true, + 'condition', df.sql('SELECT true')::jsonb, + 'then_branch', df.sql('SELECT 1')::jsonb + )::text, + 'SELECT 0' + )"#, + ) + .unwrap() + .unwrap(); + let fut = Durofut::from_json(&result); + assert_eq!(fut.node_type, "IF"); + assert!(fut.validate_recursive().is_ok()); + } + + #[pg_test] + fn test_ensure_durofut_is_not_installed() { + let exists = Spi::get_one::( + "SELECT pg_catalog.to_regprocedure('df.ensure_durofut(text)') IS NOT NULL", + ) + .unwrap() + .unwrap(); + assert!(!exists); + } + #[pg_test] fn test_multiple_starts_different_ids() { // The same graph JSON can be reused with df.start() multiple times,