Skip to content

fix(admin): extend server-side content search to the content picker and MCP - #752

Open
edrpls wants to merge 7 commits into
emdash-cms:mainfrom
edrpls:fix/content-list-search
Open

fix(admin): extend server-side content search to the content picker and MCP#752
edrpls wants to merge 7 commits into
emdash-cms:mainfrom
edrpls:fix/content-list-search

Conversation

@edrpls

@edrpls edrpls commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Extends the server-side content list search (?q=) — which shipped via #1226 — to two surfaces that still post-filtered in memory:

  • ContentPickerModal (used when linking content from the editor) now pushes its search box to the server (search option → ?q=) instead of filtering only the rows already loaded, so it finds entries anywhere in a large collection. It uses keepPreviousData so the list doesn't flash to empty between keystrokes, and keeps load-more available while searching (results can span multiple pages).
  • MCP content_list tool gains a q parameter, so agents can search a collection server-side rather than post-filtering a single page of results.

This branch was rebased onto main after #1226 merged an independent implementation of the core feature. The overlapping core-search work (handler, repository, schemas, ContentList, router, API client, index migration) is now provided by #1226; this PR keeps only the picker + MCP extensions on top of it.

Related: #1219, #1226

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). Do not include messages.po changes except in translation PRs — a workflow extracts catalogs on merge to main.
  • I have added a changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/...

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool: Claude Opus 4.8 (Claude Code)

Screenshots / test output

MCP integration tests pass, including a new case asserting content_list filters by q:

Test Files  1 passed (1)
     Tests  27 passed (27)

@changeset-bot

changeset-bot Bot commented Apr 24, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 36e9e66

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages
Name Type
emdash Patch
@emdash-cms/admin Patch
@emdash-cms/cloudflare Patch
@emdash-cms/sandbox-workerd Patch
@emdash-cms/plugin-mcp-smoke Patch
@emdash-cms/fixture-perf-site Patch
@emdash-cms/perf-demo-site Patch
@emdash-cms/cache-demo-site Patch
@emdash-cms/do-demo-site Patch
@emdash-cms/do-solo-demo-site Patch
@emdash-cms/auth Patch
@emdash-cms/blocks Patch
@emdash-cms/gutenberg-to-portable-text Patch
@emdash-cms/x402 Patch
create-emdash Patch
@emdash-cms/auth-atproto Patch
@emdash-cms/plugin-embeds Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 1,912 lines across 29 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@github-actions

Copy link
Copy Markdown
Contributor

Lunaria Status Overview

🌕 This pull request will trigger status changes.

Learn more

By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.

You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File Note
packages/admin/src/locales/ar/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/de/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/en/messages.po Source changed, localizations will be marked as outdated.
packages/admin/src/locales/es-419/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/eu/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/fa/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/fr/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/ja/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/ko/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/pseudo/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/pt-BR/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/zh-CN/messages.po Localization changed, will be marked as complete.
packages/admin/src/locales/zh-TW/messages.po Localization changed, will be marked as complete.
Warnings reference
Icon Description
🔄️ The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

@pkg-pr-new

pkg-pr-new Bot commented Apr 24, 2026

Copy link
Copy Markdown

Open in StackBlitz

@emdash-cms/admin

npm i https://pkg.pr.new/@emdash-cms/admin@752

@emdash-cms/auth

npm i https://pkg.pr.new/@emdash-cms/auth@752

@emdash-cms/auth-atproto

npm i https://pkg.pr.new/@emdash-cms/auth-atproto@752

@emdash-cms/blocks

npm i https://pkg.pr.new/@emdash-cms/blocks@752

@emdash-cms/cloudflare

npm i https://pkg.pr.new/@emdash-cms/cloudflare@752

@emdash-cms/contentful-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/contentful-to-portable-text@752

emdash

npm i https://pkg.pr.new/emdash@752

create-emdash

npm i https://pkg.pr.new/create-emdash@752

@emdash-cms/gutenberg-to-portable-text

npm i https://pkg.pr.new/@emdash-cms/gutenberg-to-portable-text@752

@emdash-cms/plugin-cli

npm i https://pkg.pr.new/@emdash-cms/plugin-cli@752

@emdash-cms/plugin-types

npm i https://pkg.pr.new/@emdash-cms/plugin-types@752

@emdash-cms/registry-client

npm i https://pkg.pr.new/@emdash-cms/registry-client@752

@emdash-cms/registry-lexicons

npm i https://pkg.pr.new/@emdash-cms/registry-lexicons@752

@emdash-cms/registry-verification

npm i https://pkg.pr.new/@emdash-cms/registry-verification@752

@emdash-cms/sandbox-workerd

npm i https://pkg.pr.new/@emdash-cms/sandbox-workerd@752

@emdash-cms/x402

npm i https://pkg.pr.new/@emdash-cms/x402@752

@emdash-cms/plugin-ai-moderation

npm i https://pkg.pr.new/@emdash-cms/plugin-ai-moderation@752

@emdash-cms/plugin-atproto

npm i https://pkg.pr.new/@emdash-cms/plugin-atproto@752

@emdash-cms/plugin-audit-log

npm i https://pkg.pr.new/@emdash-cms/plugin-audit-log@752

@emdash-cms/plugin-color

npm i https://pkg.pr.new/@emdash-cms/plugin-color@752

@emdash-cms/plugin-embeds

npm i https://pkg.pr.new/@emdash-cms/plugin-embeds@752

@emdash-cms/plugin-field-kit

npm i https://pkg.pr.new/@emdash-cms/plugin-field-kit@752

@emdash-cms/plugin-forms

npm i https://pkg.pr.new/@emdash-cms/plugin-forms@752

@emdash-cms/plugin-webhook-notifier

npm i https://pkg.pr.new/@emdash-cms/plugin-webhook-notifier@752

commit: 36e9e66

@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented May 8, 2026

Copy link
Copy Markdown
Contributor

This PR has been inactive for 14 days. It will be closed automatically in 7 days if there is no further activity.

If you're still working on this, please push an update or leave a comment.

@edrpls

edrpls commented May 16, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on upstream main (f28eb7b). Resolved orthogonal conflicts with #750 (sort headers, since merged) — <ContentList> now takes both sort/onSortChange and searchQuery/onSearchChange props; router threads sort + debounced search through the same TanStack Query key; fetchContentList accepts both orderBy/order and q; describe("orderBy"), describe("search"), and describe("sortable headers") test blocks coexist. Locale .po files reset to upstream for Lunaria re-extraction.

Local test runs after rebase, on top of f28eb7b:

  • packages/core/tests/database/repositories/content.test.ts: 51/51 pass (includes 6 new search tests)
  • packages/core/tests/unit/api/content-handlers.test.ts: 33/33 pass
  • packages/admin/tests/components/ContentList.test.tsx: 38/38 pass

Bumping out of stale-warning territory.

@edrpls
edrpls force-pushed the fix/content-list-search branch from 37749f3 to 59d9e82 Compare June 2, 2026 00:49
edrpls added a commit to edrpls/emdash that referenced this pull request Jun 2, 2026
Addresses review feedback on PR emdash-cms#752 where server-side search changed
the meaning of `items` from "loaded pages" to "filtered result set",
breaking pagination and creating a zero-result UX trap.

- ContentList denominator now trusts the server `total` in server-side
  search mode (it already reflects the filtered count) instead of
  collapsing to the loaded page size while a query is active.
- The search input stays mounted when a server-side search returns zero
  results, so the user can still edit or clear the query.
- A zero-result server-side search shows the "no results" message rather
  than the "empty collection / create your first one" CTA.
- ContentPickerModal keeps the load-more button visible during search,
  since server-side search results are paginated too.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@edrpls edrpls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated high-effort review (/code-review high) of the server-side search change. 9 findings below as inline comments, ranked by severity: 2 correctness gaps that matter for an i18n CMS, a count/denominator mismatch, two picker state races, a perf regression on D1, a Postgres dialect-parity issue, a debounce UI flash, and one reuse cleanup. None block the core fix; the non-ASCII and custom-field ones are the most worth addressing before merge.

Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/admin/src/components/ContentList.tsx
Comment thread packages/admin/src/components/ContentPickerModal.tsx Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
Comment thread packages/core/src/database/dialect-helpers.ts Outdated
Comment thread packages/admin/src/components/ContentPickerModal.tsx Outdated
Comment thread packages/admin/src/components/ContentList.tsx Outdated
Comment thread packages/core/src/database/repositories/content.ts Outdated
edrpls added a commit to edrpls/emdash that referenced this pull request Jun 2, 2026
…spec once

Two fixes from the high-effort review of PR emdash-cms#752:

- ContentList: in server-side search mode the row-count line above
  pagination reported the loaded-items count, not the server's filtered
  `total`, so it contradicted the pagination denominator until every page
  was fetched. renderItemCount now uses `total` for the match count in
  server-side mode (client-side mode still uses the locally-filtered count,
  since `total` there reflects the unfiltered set).

- ContentRepository: a searched list resolved the search predicate twice —
  once in findMany and once in count — each running a table-column
  introspection query, doubling round-trips on D1. count now accepts an
  optional pre-resolved searchSpec; findMany passes its own through, so the
  introspection runs once per request.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@edrpls

edrpls commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

@emdashbot — ran a high-effort multi-agent review on this branch and pushed fixes for the two low-risk findings. Summary below, with a couple of design calls that need a maintainer decision before I go further.

✅ Fixed in d087bc86

  • Search count line contradicted the pagination denominator. In server-side mode renderItemCount reported the loaded item count ("20 items matching…") while the pager used the server total ("1 / 8"). Now uses total for the match count in server-side mode; client-side mode still uses the local filtered count. Test added.
  • Double table-column introspection per search. findMany resolved the search predicate and then count resolved it again, so each searched request ran listTableColumns twice (4 D1 round-trips instead of 2). count now takes an optional pre-resolved searchSpec and findMany passes its own through — introspection runs once.

❓ Need a maintainer decision (design altitude)

  1. Non-ASCII case-insensitive search is broken on SQLite. The pattern is lowercased in JS (full Unicode) but the column uses SQLite's LOWER(), which folds ASCII only (no ICU/COLLATE configured). A row titled ÉCOLE won't match école. For a CMS that ships i18n/RTL as a headline feature this seems significant — but the fix (FTS-backed search, or ICU/unaccent on PG + a documented SQLite caveat) is a bigger call than I want to make unilaterally. Worth blocking the merge, or acceptable as a known limitation + follow-up?
  2. Search only targets hardcoded ["title","name","slug"], ignoring the existing per-field _emdash_fields.searchable flag (read elsewhere via FTSManager.getSearchableFields). A collection whose primary text field is headline/body is silently unsearchable. Should the list q filter derive searchable columns from the field schema instead of a constant? (This also overlaps conceptually with the FTS layer — want them unified or kept separate?)

🟡 Lower-priority findings (left for your call — happy to fix any in this PR)

  • Postgres dialect parity: listTableColumns hardcodes table_schema = 'public' while sibling columnExists/indexExists use current_schema() (migration 038 documents this footgun). On a non-public-schema PG deployment, search silently returns the entire unfiltered collection. One-line fix; flagging since tableExists/listTablesLike share the hardcode (mixed precedent).
  • ContentPickerModal: load-more results wiped by a background refetch (no staleTime; refocus/invalidation re-settles to page 1 and the accumulator-reset effect discards appended rows). Cleaner as useInfiniteQuery.
  • ContentPickerModal: load-more cursor/query mismatch during the 300ms debounce window (old cursor + new q).
  • ContentList: search-box unmount + empty-collection CTA flash for ~300ms after clearing a zero-result query (router passes the immediate searchQuery while items lag on the debounce). Cosmetic.
  • Cleanup: escapeLike is duplicated verbatim in media.ts and options.ts (+ regex equivalents in redirect.ts/comment.ts) — worth one shared helper.

Full details are in the inline comments on review #4405665999. Let me know how you'd like to handle #1 and #2 and I'll proceed.

@edrpls
edrpls force-pushed the fix/content-list-search branch from d087bc8 to 349a6f2 Compare June 2, 2026 19:19
@edrpls edrpls changed the title fix(admin): make content list search work across the whole collection fix(admin): extend server-side content search to the content picker and MCP Jun 2, 2026
@github-actions github-actions Bot added size/M and removed size/L labels Jun 2, 2026
@github-actions github-actions Bot mentioned this pull request Jun 2, 2026
18 tasks
@edrpls

edrpls commented Jun 2, 2026

Copy link
Copy Markdown
Contributor Author

Update — scope narrowed after rebase. Since my review note above, #1226 merged an independent implementation of the core server-side content search (?q=), including the locale-aware list index migration. I've rebased this branch onto main and dropped the overlapping core work; this PR is now just the two surfaces #1226 didn't cover:

  • ContentPickerModal — pushes its search box to the server instead of in-memory filtering.
  • MCP content_list — gains a q parameter so agents search server-side.

That moots most of the items above. Re-mapping each:

Design calls — now #1226's territory

  1. Non-ASCII case-insensitive search on SQLite — no longer in this PR; the core search ships via fix: server-side content list search + locale-aware list indexes #1226. If the ASCII-only LOWER() folding is still worth addressing, it's a follow-up against fix: server-side content list search + locale-aware list indexes #1226's implementation rather than a blocker here. Happy to open a separate issue.
  2. Hardcoded ["title","name","slug"] vs the field schemafix: server-side content list search + locale-aware list indexes #1226 partly addresses this: its resolveSearchColumns derives columns from _emdash_fields (adds title/name only when the collection defines them) rather than a flat constant. Whether to further key off the searchable flag / unify with the FTS layer is best tracked against that code.

Lower-priority findings

  • Postgres table_schema = 'public' hardcode — moot; the listTableColumns helper that had it was part of the discarded implementation and is gone.
  • ContentPickerModal load-more wiped by background refetchfixed in c4cb0e99 — the picker now derives its list from useInfiniteQuery pages instead of a manual accumulator, so a refocus/invalidation can't drop loaded pages.
  • ContentPickerModal load-more cursor/query mismatchfixed in the same commit — search lives in the query key, so a search change starts a fresh page chain rather than firing load-more with a stale cursor.
  • ContentList zero-result CTA flash — moot; that component's search now ships via fix: server-side content list search + locale-aware list indexes #1226.
  • escapeLike duplication — moot here (the copy was in the discarded code); still a reasonable standalone cleanup if anyone wants to pick it up.

All inline threads have been resolved accordingly. No maintainer decision is blocking this PR anymore — it's a focused follow-up on top of #1226. 🤖 update drafted with Claude Code.

@github-actions github-actions Bot added stale and removed stale labels Jun 27, 2026
@ascorbic ascorbic removed the bot:review Trigger an emdashbot code review on this PR label Jul 1, 2026
@github-actions github-actions Bot added the stale label Jul 15, 2026
edrpls and others added 2 commits July 15, 2026 14:48
The core server-side content search (?q=) landed via emdash-cms#1226. Extend it to
two surfaces that still post-filtered in memory:

- ContentPickerModal now pushes its search box to the server (search
  option -> ?q=), so it finds entries anywhere in a large collection
  instead of only the rows already loaded. Uses keepPreviousData to
  avoid flashing to empty between keystrokes and keeps load-more
  available while searching.
- MCP content_list gains a q parameter, so agents search server-side
  rather than post-filtering a page of results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the manual accumulator (allItems/nextCursor mirrored from a
useQuery) with useInfiniteQuery, matching the ContentList pattern. The
accumulator could lose loaded pages on any background refetch (window
focus, cache invalidation) and, with keepPreviousData, could fire
load-more with a stale cursor against a freshly-changed search. Deriving
the list straight from query pages removes both windows; search stays in
the query key so changing it starts a fresh page chain.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@edrpls
edrpls force-pushed the fix/content-list-search branch from c4cb0e9 to e45bf3b Compare July 15, 2026 20:53
@ascorbic ascorbic added the bot:review Trigger an emdashbot code review on this PR label Jul 20, 2026

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right fix for a real bug. Moving the content picker and MCP content_list to the server-side q search introduced in #1226 is the correct architecture, and the implementation fits EmDash's patterns. The picker now uses useInfiniteQuery with keepPreviousData to avoid flashing empty lists between keystrokes, and it correctly keeps the load-more button available while searching. The MCP tool exposes q and passes it through to the handler layer, reusing the same SQL-safe repository path as the REST route.

I checked the picker refactor, the MCP schema and tool wiring, the new MCP integration test, the underlying handleContentList/ContentRepository.findMany search implementation, and the current ContentList component. The four findings from the prior review are addressed: the three ContentList issues (trusting the server total in search mode, keeping the search input visible, and showing a "no results" state instead of the empty-collection prompt for zero-result searches) are already fixed on this branch, and this PR fixes the picker's load-more-during-search issue.

Headline conclusion: The PR is clean and ready to land after addressing two minor nits — an MCP schema inconsistency and a test organization issue. No blocking logic, security, or convention problems.

.describe("Filter by locale (e.g. 'en', 'fr'). Only relevant when i18n is enabled."),
q: z
.string()
.trim()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] The q schema here accepts empty/whitespace strings, while the REST contentListQuery schema in packages/core/src/api/schemas/content.ts adds .min(1). An empty q is silently treated as "no search" by the handler (params.q?.trim()), but the two interfaces should reject/accept the same inputs. Add .min(1) after .trim() to match the REST contract.

Suggested change
.trim()
q: z
.string()
.trim()
.min(1)
.max(200)
.optional()
.describe("Case-insensitive substring search across title, name, and slug."),

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in 5c80fee — the MCP q schema now carries .min(1) after .trim(), matching contentListQuery.

Following up in 37040d3 with the regression test the constraint was missing: content_list with q: " " now asserts an error response. Verified it fails without the .min(1) (the blank q is silently accepted and treated as "no search"), so the two interfaces can't drift apart again unnoticed.

expect(items[0]?.id).toBe(extractJson<{ item: { id: string } }>(b).item.id);
});

it("content_list filters by the q search parameter", async () => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[suggestion] This new test is nested inside describe("soft-delete visibility", () => { ... }), but it has nothing to do with soft-delete or trash visibility. Move it to a dedicated describe("content_list search", () => { ... }) block (or another appropriate top-level/grouped describe) so test failures and test reports accurately describe what behavior is being verified.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Applied in 5c80fee — the search test moved out of soft-delete visibility into its own describe("content_list search") block with its own harness setup, so a failure names the behavior it covers. The file's header comment lists it alongside the other groups.

37040d3 adds a second case to that block for the blank-q rejection.

@emdashbot emdashbot Bot removed the bot:review Trigger an emdashbot code review on this PR label Jul 20, 2026
edrpls and others added 5 commits July 20, 2026 16:49
…h test

Address review feedback on emdash-cms#752:

- MCP `content_list` `q` now uses `.trim().min(1).max(200)`, matching the
  REST `contentListQuery` contract. It previously accepted empty/whitespace
  `q` and silently ignored it; it now rejects it, so the MCP and REST
  interfaces accept and reject the same inputs.
- Move the `content_list` search test out of the unrelated
  "soft-delete visibility" describe into a dedicated "content_list search"
  block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The q schema gained min(1) to match the REST contract, but nothing
asserted it — the constraint could be dropped without a test noticing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@edrpls

edrpls commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

Merged current main (36e9e66) — the branch was 2 commits behind after the last sync.

On the two red E2E checks from the previous run: they're the invite-flow spec (user appears in the users listtext=invited-user@example.com not visible within 15s), which this PR cannot plausibly affect. The diff against main is 4 files — ContentPickerModal.tsx, mcp/server.ts, the MCP test, and the changeset — and touches nothing in the users, invite, or auth path. Main's own CI is green on 11 of its last 12 runs, and the same E2E suite passes on my other open PRs, so this reads as a flaky/timing failure rather than a regression. I don't have rerun rights on the repo, so this merge doubles as a fresh CI trigger — if the shard goes green on this run that settles it.

Local verification on the merged head: full packages/core suite green (5094 passing), MCP integration suite 321 passing, lint and typecheck clean for both emdash and @emdash-cms/admin.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants