feat(tools): SDK-level cross-source memory deduplication by Dhravya · Pull Request #1531 · supermemoryai/supermemory

Dhravya · 2026-08-18T15:16:39Z

Stack Context

This stack moves memory deduplication out of the playground UI and into the SDKs themselves, so every integration injects a single, deduplicated, self-replacing memory block. Three PRs:

sdk-dedup/tools-ts (this PR) — TypeScript SDK core + integrations
sdk-dedup/python — Python SDKs
sdk-dedup/playground — playground debug view reflects the SDK-owned block

What?

Move profile deduplication into the SDK middleware for the TypeScript tools package.

Facts are normalized (strip leading [YYYY-MM-DD], trim, collapse whitespace, casefold) and deduplicated in static > dynamic > search priority within a single request.
The result is injected as one owned <supermemory> block that replaces the previous block instead of accumulating a new one each turn.
Dedup is mode-aware: in query mode, search results are not dropped against a profile that isn't being injected.
Deduplication is request-local — no global/browser Set. Safe for multiple users, concurrent requests, and Cloudflare Worker isolates.

Covers AI SDK, OpenAI (Chat + Responses), Mastra, and VoltAgent. New shared/memory-context.ts owns the block-replacement logic.

Why?

The earlier "conversation-scoped deduplication" was only a playground browser Set — a UI debug affordance that did not change what the SDK sent to the model, and would have been unsafe as server-side global state. Real cross-source dedup belongs in the SDK, applied fresh per stateless model request.

Testing

bun run test in packages/tools: 145 passed (the one failing suite, claude-memory.test.ts, is a pre-existing broken import unrelated to this change).

🤖 Generated with Claude Code

Note

Medium Risk
Changes how system prompts and instructions are built across all TypeScript integrations; behavior is well-covered by unit tests but incorrect strip/replace logic could drop or duplicate context in production prompts.

Overview
Moves cross-source memory deduplication and owned prompt injection into @supermemory/tools so every integration sends one deduplicated memory block per request instead of growing context each turn.

Deduplication: Facts are normalized via normalizeMemoryFact (strip [YYYY-MM-DD], trim, collapse whitespace, lowercase) and deduplicated with static → dynamic → search priority. deduplicateMemoriesForMode keeps search hits in query mode when the profile is not injected.

Owned <supermemory> block: New shared/memory-context.ts wraps memories in <supermemory context="user-memories" readonly>, strips stale blocks, and replaces prior SDK context while preserving caller system instructions. Applied in AI SDK (injectMemoriesIntoParams), OpenAI Chat/Responses middleware, Mastra input processor (wrapMemoryContext), and VoltAgent hooks.

Tests: Unit coverage for block replacement (with-supermemory, OpenAI, VoltAgent), Mastra wrapper tag assertion, normalized dedup variants, and concurrent containerTag isolation.

^{Reviewed by Cursor Bugbot for commit 2fa2e0d. Bugbot is set up for automated code reviews on this repo. Configure here.}

claude · 2026-08-18T15:17:04Z

Claude finished @Dhravya's task in 1m 35s —— View job

Code Review Complete

Fetch PR diff and existing review comments
Analyze deduplication logic for correctness
Review context block replacement for edge cases
Check for security issues (injection, cross-user leakage)
Examine test coverage for critical paths
Post review summary

Overview: Moves memory deduplication from the playground UI into SDK middleware, implementing request-scoped context block replacement across all TypeScript integrations (AI SDK, OpenAI Chat/Responses, Mastra, VoltAgent).

Issues found: None — this looks good to ship.

Details:

The implementation is solid:

Request isolation is correct — The seenMemories Set in deduplicateMemories() is created fresh per call, and the memoryCache in each middleware context is per-request/per-user. No global state that could cause cross-user leakage in Cloudflare Worker isolates or concurrent requests.
Context block regex is safe — The pattern /<supermemory context="user-memories" readonly>[\s\S]*?<\/supermemory>/g uses non-greedy matching with fixed anchors, avoiding ReDoS. The specific attribute match ensures user-authored supermemory tags with different attributes won't be stripped.
Tag injection prevented — escapeMemoryContextDelimiters() in memory-context.ts:11-14 escapes < and > in memory content, preventing injection attacks where memory text containing </supermemory> could break out of the SDK-owned block.
Mode-aware deduplication handles edge cases — In "query" mode, the profile isn't injected, so search results correctly skip deduplication against the profile (tools-shared.ts:430-441). A memory appearing in both profile and search results survives in search results, preventing legitimate context from disappearing.
Normalization logic is sound — normalizeMemoryFact() strips [YYYY-MM-DD] date prefixes, [recent] tags, collapses whitespace, and lowercases for comparison while preserving the original memory string for output.
Good test coverage — Tests verify block replacement (stale → fresh), no accumulation (single <supermemory> tag), cross-user isolation, normalized deduplication variants, and query mode search result preservation.

Score: 10/10

Dhravya · 2026-08-18T15:18:42Z

How to use the Graphite Merge Queue

Add the label Main to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

_{An organization admin has enabled the Graphite Merge Queue in this repository.} _{Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.}

This stack of pull requests is managed by Graphite. Learn more about stacking.

claude

Overview: Moves memory deduplication from the playground UI into the SDK middleware, implementing request-scoped context block replacement across all TypeScript SDK integrations (AI SDK, OpenAI, Mastra, VoltAgent).

Issues found: None — this looks good to ship.

Details:

The implementation is solid:

Request isolation is correct — The seenMemories Set in deduplicateMemories() is created fresh per call, and the injected flag in the middleware map() callbacks is closure-scoped per request. No global state that could cause cross-user leakage.
Context block regex is safe — The pattern /<supermemory context="user-memories" readonly>[\s\S]*?<\/supermemory>/ uses non-greedy matching with fixed anchors, avoiding ReDoS concerns. The specific attribute match (context="user-memories" readonly) ensures user-authored supermemory tags with different attributes won't be stripped.
Normalization logic is sound — normalizeMemoryFact() strips date prefixes, collapses whitespace, and lowercases for comparison while preserving the original memory string for output. The priority order (static > dynamic > search) is maintained correctly.
Edge cases handled — Empty memories return early without injecting content. Multiple system messages get the context injected into only the first one, with stale blocks stripped from subsequent ones.
Good test coverage — Tests verify context replacement (not accumulation), cross-user isolation, and normalized deduplication.

Score: 10/10

cloudflare-workers-and-pages · 2026-08-18T15:33:26Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	supermemory-mcp	`d0f53b0`	Sep 01 2026, 06:19 AM

cloudflare-workers-and-pages · 2026-08-18T15:55:55Z

Deploying with Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	supermemory-app	`d0f53b0`	Commit Preview URL Branch Preview URL	Sep 01 2026, 06:22 AM

claude

Overview: Moves memory deduplication from the playground UI into SDK middleware, implementing request-scoped context block replacement across all TypeScript integrations (AI SDK, OpenAI Chat/Responses, Mastra, VoltAgent).

Issues found: None — this looks good to ship.

Details:

The implementation is solid:

Request isolation is correct — The seenMemories Set in deduplicateMemories() is created fresh per call, and the memoryCache in each middleware context is per-request/per-user. No global state that could cause cross-user leakage in Cloudflare Worker isolates or concurrent requests.
Context block regex is safe — The pattern /<supermemory context="user-memories" readonly>[\s\S]*?<\/supermemory>/g uses non-greedy matching with fixed anchors, avoiding ReDoS. The specific attribute match ensures user-authored supermemory tags with different attributes won't be stripped.
Tag injection prevented — escapeMemoryContextDelimiters() escapes < and > in memory content, preventing injection attacks where memory text containing </supermemory> could break out of the SDK-owned block.
Mode-aware deduplication handles edge cases — In "query" mode, the profile isn't injected, so search results correctly skip deduplication against the profile. A memory appearing in both profile and search results survives in search results, preventing legitimate context from disappearing.
Normalization logic is sound — normalizeMemoryFact() strips [YYYY-MM-DD] date prefixes, [recent] tags, collapses whitespace, and lowercases for comparison while preserving the original memory string for output.
Good test coverage — Tests verify block replacement (stale → fresh), no accumulation (single <supermemory> tag), cross-user isolation, normalized deduplication variants, and query mode search result preservation.

Score: 10/10

graphite-app · 2026-09-01T04:33:34Z

Merge activity

Sep 1, 4:33 AM UTC: Dhravya added this pull request to the Graphite merge queue.
Sep 1, 4:34 AM UTC: The Graphite merge queue removed this pull request due to downstack failures on PR #1431.
Sep 1, 4:34 AM UTC: The Graphite merge queue removed this pull request due to downstack failures on PR #1431.
Sep 1, 5:59 AM UTC: Dhravya added this pull request to the Graphite merge queue.
Sep 1, 6:20 AM UTC: Merged by the Graphite merge queue.

## Stack Context This stack moves memory deduplication **out of the playground UI and into the SDKs themselves**, so every integration injects a single, deduplicated, self-replacing memory block. Three PRs: 1. **`sdk-dedup/tools-ts`** (this PR) — TypeScript SDK core + integrations 2. `sdk-dedup/python` — Python SDKs 3. `sdk-dedup/playground` — playground debug view reflects the SDK-owned block ## What? Move profile deduplication into the SDK middleware for the TypeScript tools package. - Facts are normalized (strip leading `[YYYY-MM-DD]`, trim, collapse whitespace, casefold) and deduplicated in **`static > dynamic > search`** priority within a single request. - The result is injected as one **owned `<supermemory>` block** that *replaces* the previous block instead of accumulating a new one each turn. - Dedup is **mode-aware**: in query mode, search results are not dropped against a profile that isn't being injected. - Deduplication is **request-local** — no global/browser `Set`. Safe for multiple users, concurrent requests, and Cloudflare Worker isolates. Covers AI SDK, OpenAI (Chat + Responses), Mastra, and VoltAgent. New `shared/memory-context.ts` owns the block-replacement logic. ## Why? The earlier "conversation-scoped deduplication" was only a playground browser `Set` — a UI debug affordance that did not change what the SDK sent to the model, and would have been unsafe as server-side global state. Real cross-source dedup belongs in the SDK, applied fresh per stateless model request. ## Testing - `bun run test` in `packages/tools`: 145 passed (the one failing suite, `claude-memory.test.ts`, is a pre-existing broken import unrelated to this change). 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- > [!NOTE] > **Medium Risk** > Changes how system prompts and instructions are built across all TypeScript integrations; behavior is well-covered by unit tests but incorrect strip/replace logic could drop or duplicate context in production prompts. > > **Overview** > Moves **cross-source memory deduplication** and **owned prompt injection** into `@supermemory/tools` so every integration sends one deduplicated memory block per request instead of growing context each turn. > > **Deduplication:** Facts are normalized via `normalizeMemoryFact` (strip `[YYYY-MM-DD]`, trim, collapse whitespace, lowercase) and deduplicated with **static → dynamic → search** priority. `deduplicateMemoriesForMode` keeps search hits in **query** mode when the profile is not injected. > > **Owned `<supermemory>` block:** New `shared/memory-context.ts` wraps memories in `<supermemory context="user-memories" readonly>`, strips stale blocks, and **replaces** prior SDK context while preserving caller system instructions. Applied in AI SDK (`injectMemoriesIntoParams`), OpenAI Chat/Responses middleware, Mastra input processor (`wrapMemoryContext`), and VoltAgent hooks. > > **Tests:** Unit coverage for block replacement (with-supermemory, OpenAI, VoltAgent), Mastra wrapper tag assertion, normalized dedup variants, and concurrent `containerTag` isolation. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 2fa2e0d. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>

The base branch was changed.

graphite-app Bot assigned Dhravya Aug 18, 2026

Dhravya mentioned this pull request Aug 18, 2026

feat(sdk-playground): interactive SDK chat playground #1437

Merged

3 tasks

claude Bot reviewed Aug 18, 2026

View reviewed changes

This was referenced Aug 18, 2026

feat(python-sdks): SDK-level cross-source memory deduplication #1532

Merged

feat(sdk-playground): reflect SDK-owned memory block in debug view #1533

Merged

Dhravya assigned ved015 Aug 18, 2026

Dhravya requested a review from MaheshtheDev August 18, 2026 15:20

ved015 force-pushed the feat/sdk-playground branch from 93f92cd to 4e04a0d Compare August 24, 2026 14:43

ved015 force-pushed the sdk-dedup/tools-ts branch from 2fa2e0d to be97748 Compare August 24, 2026 16:21

ved015 previously approved these changes Aug 24, 2026

View reviewed changes

claude Bot reviewed Aug 24, 2026

View reviewed changes

Dhravya force-pushed the sdk-dedup/tools-ts branch from be97748 to ec6c159 Compare September 1, 2026 05:58

Dhravya force-pushed the feat/sdk-playground branch from 8a78b11 to e1c3122 Compare September 1, 2026 05:58

graphite-app Bot force-pushed the feat/sdk-playground branch from e1c3122 to ff2d0cf Compare September 1, 2026 06:02

graphite-app Bot force-pushed the sdk-dedup/tools-ts branch from ec6c159 to d035932 Compare September 1, 2026 06:03

graphite-app Bot force-pushed the feat/sdk-playground branch from ff2d0cf to b01d2b6 Compare September 1, 2026 06:12

graphite-app Bot force-pushed the sdk-dedup/tools-ts branch from d035932 to d0f53b0 Compare September 1, 2026 06:12

graphite-app Bot changed the base branch from feat/sdk-playground to main September 1, 2026 06:19

graphite-app Bot merged commit d0f53b0 into main Sep 1, 2026
6 of 7 checks passed

Navigation Menu

Uh oh!

feat(tools): SDK-level cross-source memory deduplication - #1531

feat(tools): SDK-level cross-source memory deduplication#1531
graphite-app[bot] merged 1 commit into
mainfrom
sdk-dedup/tools-ts

Dhravya commented Aug 18, 2026 •

edited by cursor Bot

Loading

Uh oh!

claude Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

Dhravya commented Aug 18, 2026 •

edited

Loading

Uh oh!

claude Bot left a comment

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 18, 2026 •

edited

Loading

Uh oh!

claude Bot left a comment

Uh oh!

graphite-app Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Dhravya commented Aug 18, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Stack Context

What?

Why?

Testing

Uh oh!

claude Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review Complete

Uh oh!

Dhravya commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

How to use the Graphite Merge Queue

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

cloudflare-workers-and-pages Bot commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Deploying with Cloudflare Workers

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Uh oh!

graphite-app Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge activity

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Dhravya commented Aug 18, 2026 •

edited by cursor Bot

Loading

claude Bot commented Aug 18, 2026 •

edited

Loading

Dhravya commented Aug 18, 2026 •

edited

Loading

cloudflare-workers-and-pages Bot commented Aug 18, 2026 •

edited

Loading

cloudflare-workers-and-pages Bot commented Aug 18, 2026 •

edited

Loading

graphite-app Bot commented Sep 1, 2026 •

edited

Loading