Skip to content

feat(lint): report a flow-filter token neither the template evaluator nor ObjectQL resolves - #16732

Merged
baozhoutao merged 3 commits into
mainfrom
claude/issue-16096-filter-token-flow-reach
Sep 8, 2026
Merged

feat(lint): report a flow-filter token neither the template evaluator nor ObjectQL resolves#16732
baozhoutao merged 3 commits into
mainfrom
claude/issue-16096-filter-token-flow-reach

Conversation

@baozhoutao

Copy link
Copy Markdown
Contributor

Fixes #16096

What

A second gating rule, flow-filter-token-unknown, that walks flows and reports a {…} filter token in a flow node's config.filter that neither {…} dialect can resolve: a call to a name outside the flow template evaluator's closed function table ({TOMORROW()}, {ROUND(x)}, {Math.round(x)}, {DATEADD(day, -45)}). filter-token-unknown's root list is untouched.

The naive fix the card pre-refutes — { key: 'flows', kind: 'flow' } on the existing rule — is measured, not argued, in this PR: on this tree's own example apps it reports 4 findings on app-todo + app-crm (loaded through their configs) and 3 more on app-showcase (grep census), every one a legitimate run-time variable ({recordId} ×3, {record.id} ×3, {currentTask.id} ×1). The new rule reports 0 on the same corpus.

Reference set: the flow token grammar, mirrored and pinned

packages/lint/src/flow-template-grammar.ts mirrors resolveToken's whole-token dispatch from packages/services/service-automation/src/builtin/template.ts — the five regexes, the reserved-literal step, the NOW()/TODAY() whole-token form tried before the call-position scan (the only reason {TODAY() - 45} stays silent), and the six-name value-function table. @objectstack/lint may not depend on a runtime, so the mirror is pinned by flow-template-grammar.test.ts, which reads template.ts from disk (declared cross-package input; spelled per check:cross-package-test-inputs) and fails when any mirrored piece stops matching the original.

Verified against the evaluator, not the filename: every mirrored regex source is present verbatim in template.ts; EXPRESSION_FUNCTION_ARITY's key set equals the mirrored table; unknownFunctionError is still thrown on an unknown call (A4 holds — TOMORROW() is a guard refusal at run time, so the node cannot run; it is not resolved to anything).

Layering

Layer one asks classifyFilterToken (the same spec classifier filter-token-unknown uses); only its unknown verdict goes to layer two, classifyFlowTemplateToken; only unknown-function is a finding. So every finding is a strict subset of what the naive root addition would report (asserted in the test), and the open arm — bare and dotted identifiers addressing the run's VariableMap — is deliberately silent and says why.

Successor note — the inherited commit, diffed and verified

4c7d5c522 (7 files) was pushed by a dispatch that died before opening a PR. Each file was read against the evaluator and the registry pins rather than trusted; nothing in it was thrown away or rebuilt. One thing it was missing: registering the 44th author-time rule stales the four os build / os validate transcripts under content/docs that quote the count as a literal, and check:docs-transcript-drift went red on this branch. de9b5514 moves those four literals (43 → 44). The branch was also merged with origin/main (bd62a569, clean; packages/lint/src was uncontested at the time of the check — open PR #16727 now touches two other files in that directory, none of this PR's).

Evidence (tree de9b5514)

Reproduction of the card's control design on this tree, through runAuthoringRules('lint', …) — the registry line os lint runs (packages/cli/src/commands/lint.ts:514; exit 1 at :881 when any error-severity finding exists):

leg {TOMORROW()} in a view filter[].value {TOMORROW()} in a flow node config.filter
rule unregistered (base copies of authoring-rules.ts + index.ts, dist rebuilt, marker absent from all 14 built files) filter-token-unknown error → exit 1 0 findings → exit 0
rule registered (restored, blob hashes equal HEAD's, marker present in 6 built files) filter-token-unknown error → exit 1 flow-filter-token-unknown error → exit 1

{TODAY() - 45} and {record.id} in the flow position: 0 findings in both legs (the negative control triage asked for).

  • pnpm --filter @objectstack/lint test — 102 files / 3549 tests passed (includes the two new suites plus the wiring, barrel-export and runtime-gate pins).
  • pnpm --filter @objectstack/lint typecheck — OK, test layer under tsconfig.test.json OK (2 files / 6 pre-existing errors held in the ledger, unchanged).
  • Derived gates (dispatch-gates --commands, 83 families on the final file set): see the report comment on the card for the per-family table. Two families read PREREQUISITE NOT MET here (whole-repo dists) and are declared to CI: check:dual-build-cjs-loads, check:type-check-debt.
  • Not run locally: the CLI package's suites (57-package cold closure); the CLI consumes the registry unchanged and its command-parity test is a case list, not an exhaustive pin. Declared to CI.

验收备注

  • skills/objectstack-query/rules/filters.md (governed surface, not touched) describes filter-token-unknown as the rule that refuses a whole-string {…} token; the flow position now has its own id. Not wrong, only incomplete — noted, not filed; 承接者:无.
  • scripts/check-plugin-teardown-shape.mjs --self-test cannot run on a shallow clone (its positive control is pinned to commit 621a4876); fetching that one commit makes it green (47 cases). Environment, not a defect — noted, not filed.
  • fix(cli): os validate lowers inline handlers before its parse, so the hook write-set family judges handler-authored hooks there too #16727 is the other open PR in packages/lint/src; it does not touch authoring-rules.ts, so the rule count stays 44 whichever lands first.

Generated by Claude Code

… nor ObjectQL resolves

`filter-token-unknown` walks seven presentation collections and not `flows`,
so `{TOMORROW()}` in a list view's filter fails the build while the identical
string in a flow node's `config.filter` is silent — although this package's
sibling filter rules have reached `flows` all along.

Adding `flows` to that rule's root list is the wrong close. A flow node's
filter is interpolated by the automation template evaluator BEFORE ObjectQL
sees it, and only what the evaluator cannot resolve is handed on. Judging a
flow filter against the ObjectQL vocabulary reports every legitimate
`{record.id}` / `{recordId}`: 7 findings on this repo's own example apps, all
7 false positives.

So this is a second rule id with the flow dialect as its reference set, and
`filter-token-unknown`'s declared surface list is untouched. Only the class
NEITHER dialect resolves is reported — a call to a name outside the flow
template dialect's closed function table, where the evaluator already raises a
guard refusal, so the node cannot run at all. The open arm (bare and dotted
identifiers addressing the run's variable map) stays silent and says why.

`flow-template-grammar.ts` mirrors the evaluator's whole-token dispatch because
this package may not depend on a runtime; its drift is pinned by a test that
reads the original from disk.

Finding delta on this repo's example apps: 0.

Claude-Session: https://claude.ai/code/session_01Vbw3RPgdtqesx4azk9SbW8
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`flow-filter-token-unknown` is the 44th registered author-time rule, and
`check:docs-transcript-drift` derives that count from the registry for the
four `os build` / `os validate` transcripts under content/docs that quote it
as a literal. The inherited commit registered the rule without moving the
literals, so the gate went red on this branch; this is the other half.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/lint, touching 21 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/lint/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

1 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/deployment/validating-metadata.mdx (via AUTHORING_RULES (symbol, a top-level const object), runtimeTypes (symbol, a field of const object AUTHORING_RULES))

1 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/v17.mdx (via AUTHORING_RULES (symbol, a top-level const object), runtimeTypes (symbol, a field of const object AUTHORING_RULES))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/lint/src/index.ts) — pages documenting those are invisible to this run
  • 7 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 61 of 219 client-bound route-ledger rows — the other 158 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 158: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 56 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 102 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 5 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 5e53d73d83553ba4d87cba49797e79b6c8754e5apackageMentionDocs.

Which tree this was computed on

This run read content/docs from 5e54bf875145fd523f66d4a5f89530251064152f — the merge of head de9b551431d80e4e396664dc23167b1eeb4d3617 into base 5e53d73d83553ba4d87cba49797e79b6c8754e5a, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 5e54bf875145fd523f66d4a5f89530251064152f && git checkout 5e54bf875145fd523f66d4a5f89530251064152f
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 5e53d73d83553ba4d87cba49797e79b6c8754e5a de9b551431d80e4e396664dc23167b1eeb4d3617 && git checkout -B drift-repro 5e53d73d83553ba4d87cba49797e79b6c8754e5a && git merge --no-ff de9b551431d80e4e396664dc23167b1eeb4d3617

node scripts/docs-audit/affected-docs.mjs --json 5e53d73d83553ba4d87cba49797e79b6c8754e5a

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 5e53d73d83553ba4d87cba49797e79b6c8754e5a → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@github-actions github-actions Bot added documentation Improvements or additions to documentation tests tooling labels Sep 8, 2026
@baozhoutao
baozhoutao marked this pull request as ready for review September 8, 2026 03:13
@baozhoutao
baozhoutao enabled auto-merge September 8, 2026 03:13
@baozhoutao
baozhoutao added this pull request to the merge queue Sep 8, 2026
Merged via the queue into main with commit d61bad0 Sep 8, 2026
40 checks passed
@baozhoutao
baozhoutao deleted the claude/issue-16096-filter-token-flow-reach branch September 8, 2026 03:40
hotlong pushed a commit that referenced this pull request Sep 8, 2026
… position

The passage sat directly after "A flow node's `config.filter` takes these
tokens too", so its rule id read as the one that fails an unknown token in a
flow node. Since #16732 that position is owned by a second rule.

The two ids stay distinct, because the two positions do not share a token
vocabulary or a consequence:

  - flows -> `flow-filter-token-unknown` (validate-flow-filter-tokens.ts),
    which fires only when NEITHER dialect resolves the token and whose
    consequence is a guard refusal: "this node cannot run at all";
  - dashboards/objects/views/reports/datasets/pages/apps ->
    `filter-token-unknown` (validate-filter-tokens.ts), which deliberately
    does not walk `flows` and whose consequence is the silent zero.

Collapsing them into one statement would re-teach the confusion the split
exists to prevent, so the flow sentence names its own rule and the general
paragraph is scoped with "Outside a flow".

Wording (not the id distinction) was tightened to stay under the published
token ratchet: 2136 -> 2146 tokens against a 2149 ceiling.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P58euzUXCVJNwmhuPC9DXY
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants