Fix run duplication problem#18
Conversation
There was a problem hiding this comment.
Code Review
This pull request replaces the per-block blockId with a content hash (pl7.app/contentHash) derived from the source table's CanonicalID to namespace exported property columns, enabling downstream deduplication of identical results. This logic is moved from main.tpl.tengo to a new post-processing template process.tpl.tengo. Feedback on the changes points out a potential runtime panic in process.tpl.tengo when accessing i.Value without checking if the key exists in the map.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Stamp pl7.app/contentHash (base32 of the source table's CanonicalID) on the exported property columns instead of the per-block blockId, and drop the blockId-keyed trace step id. Identical results across blocks/projects now produce content-identical columns that dedupe downstream, while different results stay distinct. Mirrors the humanization-score approach. The Python step and Parquet import already deduped; this extends it to the export. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
info().CanonicalID is only readable once the table resource is resolved, which it is not in main's body (the same body that creates the Python exec — the saveFile output field has Value==0, so smart.resource(i.Value) errors with "null resource id"). Move the TSV import + content-tagging + trace stamping into a new process.tpl.tengo that receives the Python output tables as resolved render inputs. Also probe info() with maps.containsKey (it is a strict map; accessing a missing key throws). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
7900e67 to
ccadf23
Compare
| contentHash := func(table) { | ||
| i := table.info() | ||
| if maps.containsKey(i, "CanonicalID") { | ||
| return ll.base32Encode(i.CanonicalID) | ||
| } | ||
| ri := smart.resource(i.Value).info() | ||
| return ll.base32Encode(ri.CanonicalID) | ||
| } |
There was a problem hiding this comment.
Unguarded accesses in the fallback branch of
contentHash
The first branch correctly uses maps.containsKey(i, "CanonicalID") before accessing i.CanonicalID, but the fallback branch accesses i.Value and then ri.CanonicalID without the same protection. Since info() is documented as a strict map that throws on missing keys, if a resource arrives with neither CanonicalID nor Value (e.g. an error-state or future resource shape), or if the dereferenced resource lacks CanonicalID, the function throws a cryptic "key not found" rather than surfacing a useful error. Consider guarding with maps.containsKey(i, "Value") before the dereference, and similarly for ri.CanonicalID.
Prompt To Fix With AI
This is a comment left during a code review.
Path: workflow/src/process.tpl.tengo
Line: 27-34
Comment:
**Unguarded accesses in the fallback branch of `contentHash`**
The first branch correctly uses `maps.containsKey(i, "CanonicalID")` before accessing `i.CanonicalID`, but the fallback branch accesses `i.Value` and then `ri.CanonicalID` without the same protection. Since `info()` is documented as a strict map that throws on missing keys, if a resource arrives with neither `CanonicalID` nor `Value` (e.g. an error-state or future resource shape), or if the dereferenced resource lacks `CanonicalID`, the function throws a cryptic "key not found" rather than surfacing a useful error. Consider guarding with `maps.containsKey(i, "Value")` before the dereference, and similarly for `ri.CanonicalID`.
How can I resolve this? If you propose a fix, please make it concise.…pdate related specs
Namespace exported property columns by a content hash of the source table (pl7.app/contentHash, derived from the backend CanonicalID) instead of the per-block blockId, and drop the blockId-keyed trace step id. Identical results across blocks/projects now produce content-identical columns that dedupe downstream instead of being made unique per block; different results stay distinct. The heavy Python step and Parquet import already deduped; this extends dedup to the exported pframe identity.
Greptile Summary
This PR replaces per-block
blockIdnamespacing on exported PFrame column domains with a content-derived hash (pl7.app/contentHash) pulled frominfo().CanonicalIDof the Python output tables. The pFrame-building logic is split into a newprocess.tpl.tengorender template so that the Python outputs arrive as resolved render inputs (makingCanonicalIDavailable), enabling identical results across different blocks or projects to share the same column identity and deduplicate downstream.workflow/src/process.tpl.tengo— new render template that imports the TSV outputs into Parquet-backed PColumns, derives a content hash from each source table, and stamps columns withpl7.app/contentHashin their domain instead ofpl7.app/blockId; the trace step no longer carries anidfield so it is content-identical across runs.workflow/src/main.tpl.tengo— removes the inline pFrame-building (andxsv/pSpecimports), imports the newprocessTpl, and defers the import+stamp work viarender.create; theparamsJSON resource is built withcanonical.encodeso its CID is deterministic across runs.workflow/src/columns.lib.tengoandtest/src/wf.test.ts— comment and TODO-description updates to reflect the terminology change fromblockIdtocontentHash.Confidence Score: 5/5
Safe to merge — the split into a separate render template is architecturally clean and the content-hash derivation is correct for resolved render inputs.
The core change (moving pFrame building into process.tpl.tengo so Python outputs arrive as resolved resources) achieves its goal cleanly. The canonical JSON params resource, sorted chain key lists, and removal of the blockId trace id all cooperate to make the render CID deterministic. The aaFractionTsv is always produced by the exec (empty file in non-peptide mode) so passing it unconditionally as a render input is safe. No new logic defects were found beyond what was already raised in prior review threads.
workflow/src/process.tpl.tengo — the contentHash helper has an unguarded fallback branch (covered by a prior thread); no other files require special attention.
Important Files Changed
Sequence Diagram
sequenceDiagram participant MB as main.tpl.tengo participant PY as Python exec participant RC as render.create participant PT as process.tpl.tengo participant PF as pFrameBuilder MB->>PY: exec input.tsv + plan.json PY-->>MB: properties.tsv, aa_fraction.tsv MB->>RC: render.create processTpl with properties, aaFraction, params Note over RC: dedups on content-addressed CID RC->>PT: body with resolved inputs PT->>PT: contentHash via info().CanonicalID PT->>PT: xsv.importFile properties to scalarOut PT->>PT: xsv.importFile aaFraction to aaOut (peptide only) PT->>PF: pf.add with stamp(spec, contentHash) Note over PT,PF: injects pl7.app/contentHash into domain PF-->>PT: built pFrame PT-->>RC: propertiesPf and exportPframe RC-->>MB: processResult MB-->>MB: outputs.propertiesPf and exports.propertiesReviews (2): Last reviewed commit: "Fix contentHash: read CanonicalID in a r..." | Re-trigger Greptile