Expose project folder membership and root filtering (#88) - #205
Conversation
A model asked which projects sit outside folders and could not answer it. folderID and folderName were silently dropped because ProjectItem carried no membership at all, so the model could only sum folder projectCount values to infer that some projects were unfiled -- never identify them. list_projects now returns folderID, folderName, and folderPath, read from the documented project.parentFolder relationship rather than reconstructed by scanning folders. folderPath lists ancestors root-first so repeated folder names stay distinguishable, and is depth-capped so a malformed parent cycle cannot spin. A project at the library root reports null for all three. A rootOnly filter scopes the query server-side to projects whose parentFolder is null, so reviewing unfiled projects no longer means fetching the whole catalogue. rootOnly is part of the projects cache key. Without that, a cached full-catalogue page would satisfy a root-only request and vice versa, which is precisely the bug this feature exists to avoid. It also enters the query identity, so a cursor issued for one scope cannot resume the other. Measured against the 379-project UAT baseline, on a 392-project database: model-visible items 392 to 102 (74% fewer), payload 27024 to 8765 bytes (68% smaller), per-call latency unchanged at ~415ms. The latency is flat because the bridge still walks flattenedProjects; the win is that the question is answerable at all, and answerable compactly. Live verification: rootOnly returned exactly the 102 projects whose folderID is null in the full catalogue, every filed project carried a folderName, every folderPath ended at its folderID, no root project leaked folder data, and 62 projects reported nested paths. Validation impact: transport-reliability per focusrelay-dev classify. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Smoke re-run: also failed closed, but the failure movedSecond 10-minute smoke run on the same build: also failed closed, on a different scenario.
Run 2 detail: Why this reads as environmental rather than a defect in this change
Where that leaves the merge gateThe Decisive next step, if wanted: run the same smoke profile on Recommendation: do not merge on the current evidence. Either establish the master baseline, or re-run smoke on a quiesced host. |
Smoke gate now passes — plus a master baseline for attributionResult265 measured calls, 0 failures, at host load ~4.7 — the same load at which the two earlier runs failed.
Full picture across four runs
Run 3 used master's own plug-in and release binary in a separate worktree, so it is a like-for-like baseline rather than a mixed-version comparison. ReadingThe branch now has a clean smoke run at the load where it previously failed, so the gate is satisfied on its own terms. On attribution, the honest statement is that the earlier failures were never distinguishable from host flakiness: 2 failures in ~404 branch calls against 0 in 315 on master is about p≈0.5 by Fisher's exact test — indistinguishable from chance. Run 4 is consistent with that, and adds a positive result at matched load, but four runs cannot prove a ~0.4% event absent. What can be said concretely is that no failure has ever occurred in a code path this change touches, and that both failures were single 45 s stalls against scenario p95s of 2.8 s and 15.1 s respectively. Worth filing separatelyThis host runs close to the deadline regardless of branch: master's own passing run peaked at 29611 ms against the 45 s timeout. A suite that fails closed on one stalled call, on a host whose tail routinely reaches 60-70% of the deadline, will keep producing results like runs 1 and 2. That is an evidence-infrastructure problem rather than a product defect, and it sits close to #193, which already tracks benchmark evidence being disturbed by host conditions. Previous comment's recommendation not to merge is withdrawn: the required gate has now passed. |
Direct performance measurement of the changed pathThe smoke suite benchmarks
Sample stdev 20-30 ms, so standard error ≈ 9 ms at n=10. No regression for existing callers. 385 ms against 390 ms is indistinguishable from noise, which is what the implementation predicts: the folder work sits behind Opt-in cost is ~50 ms (+13%) on a full-catalogue query when folder data is requested. That is a real effect rather than noise (≈5 standard errors), and it buys information that was previously unobtainable at any cost. Adding The targeted workflow is strictly better: Conclusion: merging does not reduce responsiveness on any existing path. |
1 participant
Closes #88. Unblocks #171, which needs project paths so batch-resolved duplicate names stay distinguishable.
Problem
A model asked which projects sit outside folders and could not answer it.
folderIDandfolderNamewere dropped becauseProjectItemcarried no membership at all, so the model could only sum folderprojectCountvalues to infer that ~91 of 379 projects were unfiled — never identify which.Change
list_projectsreturnsfolderID,folderName, andfolderPath, read from the documentedproject.parentFolderrelationship rather than reconstructed by scanning folders.folderPathlists ancestors root-first so repeated folder names stay distinguishable, and is depth-capped so a malformed parent cycle cannot spin. A project at the library root reportsnullfor all three.A
rootOnlyfilter scopes the query server-side to projects whoseparentFolderis null.rootOnlyis part of the projects cache key. Without that, a cached full-catalogue page would satisfy a root-only request and vice versa — returning filed projects to a query asking for unfiled ones, which is exactly the confusion this feature exists to remove. It also enters the query identity, so a cursor issued for one scope cannot resume the other. Both are pinned by tests.Measured against the UAT baseline
On a 392-project database:
rootOnly=trueModel-visible items 74% fewer, payload 68% smaller. Per-call latency is unchanged — the bridge still walks
flattenedProjects, so the filter simply runs there instead of in the model. The win is that the question becomes answerable at all, and answerable compactly; it is not a latency optimisation and is not presented as one.Live verification
rootOnlycount vsfolderID == nullin full cataloguefolderNamefolderPathtail ≠folderIDDirect MCP probes confirmed the same through the wire, including that unsupported field names are rejected (
list_projects.fields contains unsupported field(s): bogusField) — that acceptance criterion turned out to be already satisfied before this work.Validation
Impact:
transport-reliabilityperfocusrelay-dev classify(it flagsBridgeClient, whose only change here is threading arootOnlyparameter).281 tests pass, including 9 new ones covering root/filed/nested/duplicate-name membership, cache keying, and cursor identity.
All semantic gates pass;
validate --impact queryclean.The 10-minute smoke run failed closed, and I am not claiming otherwise: 1 timeout in 66 measured calls (1.52%), on
get_project_counts/project_view_everything,pickupState=bridge_processingat the 45 s deadline.Attribution: this change cannot reach that path. Every JS edit lies between lines 2526-2660, inside the
list_projectsbranch;get_project_countsbegins at line 3177 and is untouched, and the new code is gated behindhasField(...)androotOnly === true. The run's own baseline was already slow — p50 9.4 s, p95 15.1 s, p99 17.4 s — on a machine that had been running benchmarks and heavy OmniFocus work for hours.A re-run is in progress and this PR will be updated with the result. Do not merge on the strength of the failed run alone.
🤖 Generated with Claude Code