Skip to content

feat: audit duplicate claims CLI command - #176

Open
TanishValesha wants to merge 3 commits into
Autoloops:mainfrom
TanishValesha:feat/audit-duplicate-claims
Open

feat: audit duplicate claims CLI command #176
TanishValesha wants to merge 3 commits into
Autoloops:mainfrom
TanishValesha:feat/audit-duplicate-claims

Conversation

@TanishValesha

Copy link
Copy Markdown
Contributor

Fixes #169

Summary

Implements a new graph audit duplicates CLI command that detects and reports duplicate claims in the knowledge graph using cosine similarity on claim embedding vectors.

Files Changed

  • apps/cli/main.ts — Registers the graph audit duplicates route and implements its handler
  • libs/knowledge-graph/provider.ts — Adds auditDuplicateClaims method signature to the provider interface
  • libs/knowledge-graph/service.ts — Implements the core duplicate detection logic with auditDuplicateClaims method and adds DuplicateAuditGroup / DuplicateAuditResult types
  • libs/knowledge-graph/local-provider.ts — Wires the local provider implementation to delegate to the service
  • libs/knowledge-graph/managed-client.ts — Wires the managed (remote) client implementation via HTTP request

Description

Background

As the knowledge graph grows, duplicate or near-duplicate claims can accumulate across sessions, diluting retrieval quality and wasting embedding storage. Users needed a way to inspect and quantify these duplicates before deciding whether to deduplicate.

Implementation

The new greplica graph audit duplicates command performs the following:

  1. Reads the full graph for the current repository from the database
  2. Filters out superseded claims, keeping only active ones
  3. Loads existing embedding vectors from the database for each active claim
  4. Generates embeddings on-the-fly for any active claims missing vector data
  5. Performs pairwise cosine similarity comparison between all active claim vectors using findSimilarClaims
  6. Groups duplicate claims — a source claim and all its similar matches above the configured similarity threshold (default 0.75)
  7. Prints a summary showing total active claims checked and total duplicate pairs found, followed by a detailed per-group listing with claim IDs, truncated text, and similarity scores

Output

Duplicate claims audit:
Total active claims checked: 447
Total duplicate pairs found: 263
- "claim.cli_install_flags_enabled_disabled_20260629" ("`greplica install` now accepts...")
  → "claim.install_flags_hooks_auto_memory_20260629" (0.9609) "..."
  → "claim.install_flags_default_behavior" (0.8912) "..."
  ...

@TanishValesha

Copy link
Copy Markdown
Contributor Author

@kushalpatil07 have a look at it whenever possible.
Thankyou!

@an1ojha an1ojha closed this Jul 24, 2026
@an1ojha an1ojha reopened this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a CLI command to audit duplicate claims

3 participants