fix(gqlize,ormize): honour nested pagination, and stop a scope filtering parents - #64
Merged
Conversation
…ing parents Validating the root execution map — the eager-loading include plan of `docs/specifications.md` §5 — against what the code actually does. Most of the contract holds. Three things did not, all of them silent. A nested connection's page bound was dropped whenever it could not be pushed into SQL. The planner sets `descriptor.limit` for any `first`/`last`, but the adapter applies `limit`/`offset` only on its `separate` branch — and `separate` is available only for an unrequired `hasMany`. So `btmItems(first: 2)` returned every row, and so did `items(first: 2, required: true)`. Nothing downstream compensated: the connection resolver maps every returned row to an edge, and `hasNextPage` is derived as `windowEnd < total - 1`, which then computes `false`. The client was handed an unbounded page and told it had seen everything. `first` is the only bound a caller has on a nested connection — which is why the planner clamps it — so this was an unbounded response, not merely a wrong one. Two shapes, two answers, because the right one differs: - Not `required` (a `belongsToMany`, which Sequelize cannot fetch separately): leave the relation out of the plan. Its own resolver runs a per-parent query that applies the limit in SQL, which is what §5 says should happen to relations the parent query cannot express. Measured: 3 parents cost 4 selects instead of 1, and return 2 rows each instead of 3. That is the trade, and it is the right way round. - `required`: keep the JOIN. The INNER JOIN is what filters the parent rows, and no per-parent query can recover that — dropping it would silently widen the parent set, which is worse than over-reading children. The adapter applies the window to the loaded rows instead. Unconditionally, because it is a no-op whenever SQL already did the work; doing it only where it seemed needed is how the bound went missing in the first place. A row-level scope could filter parent rows. Both `scopeIncludePlan` (§12) and `scopeIncludes` (§13) force `required: false` on a scoped child, with a comment saying a scope on a child must never become a filter on its parent — but both ran that only `if (inc.required === undefined)`, and the include planner always writes a defined boolean. On the path that reaches them the guard never fired. A caller writing `docs(required: true)` against a scoped relation had its parent list narrowed by rows it may not see, which reports their existence through their absence. Both are unconditional now. `required: true` keeps its meaning among visible rows; only discarding the parent along with its filtered children is refused. A nullable singular relation was re-queried after being eager-loaded. The eager check was truthy, so a `belongsTo`/`hasOne` that LEFT JOINed to nothing read as "not loaded" and issued the accessor query anyway — returning the same null, once per parent row. Absent means absent; `null` is an answer. Tests. `__tests__/helper/sql.ts` is the SQL-capture helper the two file-local copies become; it adds `joins()`, because query reduction is a claim about the statements emitted and the repo could previously assert on SQL text in exactly one place. `eager-conformance.test.ts` covers §5 by query count and SQL shape rather than returned data — the existing counts assert `<= 3`, which passes even when a relation degrades from one JOIN to a separate query. Several are negative, including the two documented fallbacks that had no test at all: `options.autoInclude = false` (read in one place, never exercised) and the planner's blanket catch, which was known to log but never shown to still answer correctly. It runs under sqlite, postgres and roundtrip — JOIN-versus-separate and LIMIT-in-a-subquery differ by dialect, and roundtrip is the only place a plan degrading after the schema artifact round-trip would show up. One thing checked and found already safe: a client `include:` entry carries no `target`, so §12 asks its predicate about `undefined` and imposes nothing. The join is still scoped — §13 works off the model on each native include rather than the portable descriptor. That is the backstop doing its job, and there is now a test that says so, asserting the relation was loaded as well as filtered so it cannot pass by never joining at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019fGumVzMfMXDZ5vS7PJGDW
§5 stated the include-plan contract in about a dozen claims and nothing said which of them were checked. This adds a conformance table naming, for each, the verdict and the test that holds it — so the next person to change the planner can tell a deliberate decline from a regression. Two rows are new behaviour rather than confirmation, and say so: a paginated `belongsToMany` now declines to eager-load, and a paginated `required` relation keeps its JOIN and has the window applied to the loaded rows. The known-limits list is the more useful half. Recorded because why something is absent is worth as much as what is: - no eager plan is built below a fallback, so any relation that resolves per-relation takes its whole subtree N+1 with it — broader than the cross-adapter entry already in the known-gaps list; - aliased relationship fields collapse, because the plan is keyed by field name; - `@skip`/`@include` are not consulted when flattening, so a skipped field is still joined — and a `required: true` on one still filters parents; - a client-supplied `include:` entry has no `target`, which costs it §12 scoping, its `afterFind`, and computed-`orderBy` expansion. The join is still scoped: §13 works off the model on each native include. That was checked rather than assumed, and is now pinned by a test that asserts the relation was loaded as well as filtered, so it cannot pass by never joining; - `countOptions` has no `distinct: true`, which would inflate `total` under a `hasMany` fan-out on a dialect without inline count — latent, since the three dialects that have one are the ones in use; - eager children are not column-projected and nesting has no depth limit. Also amends the `required` bullet, which described the argument without mentioning that a row-level scope overrides it back to a LEFT JOIN. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019fGumVzMfMXDZ5vS7PJGDW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Validating the root execution map — the eager-loading include plan of
docs/specifications.md§5 — against what the code actually does. Most of the contract holds. Three things did not, all of them silent.1. A nested page bound was dropped whenever it couldn't reach SQL
The planner sets
descriptor.limitfor anyfirst/last; the adapter applieslimit/offsetonly on itsseparatebranch — andseparateis available only for an unrequiredhasMany. SobtmItems(first: 2)returned every row, and so diditems(first: 2, required: true).Nothing downstream compensated. The connection resolver maps every returned row to an edge, and
hasNextPage = windowEnd < total - 1then computesfalse— the client was handed an unbounded page and told it had seen everything.firstis the only bound a caller has on a nested connection, which is why the planner clamps it. This was an unbounded response, not merely a wrong one.Two shapes, two answers, because the right one differs:
required(belongsToMany)requiredThe window is applied unconditionally, because it's a no-op whenever SQL already did the work. Doing it only where it seemed needed is how the bound went missing in the first place.
2. A row-level scope could filter parent rows
Both
scopeIncludePlan(§12) andscopeIncludes(§13) forcerequired: falseon a scoped child, with a comment saying a scope on a child must never become a filter on its parent — but both ran that onlyif (inc.required === undefined), and the planner always writes a defined boolean. On the path that reaches them the guard never fired.A caller writing
docs(required: true)against a scoped relation had its parent list narrowed by rows it may not see — reporting their existence through their absence. Both are unconditional now.required: truekeeps its meaning among visible rows; only discarding the parent along with its filtered children is refused.3. A nullable singular relation was re-queried after being eager-loaded
The eager check was truthy, so a
belongsTo/hasOnethat LEFT JOINed to nothing read as "not loaded" and issued the accessor query anyway — returning the same null, once per parent. The captured SQL shows both statements. Absent means absent;nullis an answer.Tests
__tests__/helper/sql.tsreplaces two divergent file-local copies of the SQL-capture helper and addsjoins()— query reduction is a claim about statements emitted, and the repo could previously assert on SQL text in exactly one place.eager-conformance.test.tscovers §5 by query count and SQL shape, not returned data. The existing counts assert<= 3, which passes even when a relation degrades from one JOIN to a separate query.Several are negative, including the two documented fallbacks that had no test at all:
options.autoInclude = false(read in one place, never exercised) and the planner's blanket catch, known to log but never shown to still answer correctly.Runs under sqlite, postgres and roundtrip — JOIN-vs-separate and LIMIT-in-a-subquery differ by dialect, and roundtrip is the only place a plan degrading after the schema artifact round-trip would surface.
One thing checked and found already safe
A client
include:entry carries notarget, so §12 asks its predicate aboutundefinedand imposes nothing. I expected a security finding. The join is still scoped — §13 works off the model on each native include rather than the portable descriptor. That's the backstop doing its job, and there's now a test that says so, asserting the relation was loaded as well as filtered so it can't pass by never joining.Also
docs/specifications.md§5 gains a conformance table and a known-limits list — including that no eager plan is built below a fallback (so any per-relation fallback takes its whole subtree N+1 with it, broader than the existing cross-adapter known-gap entry), aliased relationship fields collapsing, and@skip/@includenot being consulted.Verification
pnpm typecheckandpnpm lintclean at--max-warnings 0Branched off
main; independent of the open #62 and #63.🤖 Generated with Claude Code
https://claude.ai/code/session_019fGumVzMfMXDZ5vS7PJGDW