Skip to content

Repository files navigation

sub-checker

PyPI version Python versions CI License: MIT

繁體中文 | English

Pre-submission manuscript checker powered by Claude agents with a Plan-Execute-Verify harness. Each check is performed by a specialized AI agent, then validated by deterministic checks and a reviewer agent to eliminate false positives.

What it checks

Agent What it does
typo_grammar Spelling, grammar, awkward phrasing (skips reference list)
figure_table Figure/table references exist, numbering is sequential, files present
citation_exist In-text citations match the reference list (deterministic pre-scan + agent)
citation_format Reference list follows target journal's citation style (APA, Vancouver, AMA, etc.)
journal_guidelines Word count, required sections, abstract format, required statements (COI, ethics, data availability)
logic Contradictions, unsupported claims, methods-results mismatches
citation_claim Multi-source verification (PubMed + Semantic Scholar + Crossref), then verifies claims against abstracts

v2 reliability guards

  • Bounded manuscript reads: oversized sections and reference lists stop at a safe context boundary and produce an explicit partial-coverage notice. They are never silently treated as a complete check.
  • Persistent, expiring web cache: public journal-guideline pages and search results are reused for 30 days, reducing latency and repeated requests without caching failed fetches.
  • No private corpus required: all public tests use generated or synthetic manuscripts.

Demo

🎥 Watch the 60-second demo  ·  vertical version for mobile

How it works

Architecture

How a reference is verified

Reference verification

Install

pip install sub-checker

Setup

You need an Anthropic API key:

export ANTHROPIC_API_KEY=sk-ant-...

Or create a .env file in your working directory:

ANTHROPIC_API_KEY=sk-ant-...

Usage

CLI

# Full check with target journal
sub-check paper.docx -j "The Lancet"

# Chinese report output
sub-check paper.docx -j "Nature Medicine" --lang zh-TW

# Only run specific checkers (cheaper & faster)
sub-check paper.docx --only figure,citation

# Skip expensive checkers
sub-check paper.docx --skip claim,logic

# Output as styled HTML report (includes COT viewer + confidence scores)
sub-check paper.docx -o html --output-file report.html

# Output as JSON (for programmatic use)
sub-check paper.docx -o json --output-file report.json

# Dry run (just parse, no agents)
sub-check paper.docx --dry-run

Web GUI

The GUI needs the source repo (the React frontend is not shipped on PyPI), plus the optional web backend dependencies:

git clone https://github.com/odafeng/sub-checker.git
cd sub-checker
pip install -e ".[web]"          # backend: FastAPI + uvicorn

# Start backend
uvicorn sub_checker.api:app --reload

# Start frontend (in another terminal)
cd frontend && npm install && npm run dev

Open http://localhost:5173 — upload a .docx, pick a journal, run, and view the report with confidence badges and filtered false positives.

CLI options

sub-check [OPTIONS] MANUSCRIPT_PATH

Arguments:
  MANUSCRIPT_PATH    Path to .docx file or directory containing one (optional with --init)

Options:
  -j, --journal      Target journal name (e.g. "The Lancet")
  -c, --config       Path to a config file (default: ./.sub-checker.yaml if present)
  -o, --output       terminal | json | markdown | html (default: terminal)
  --output-file      Write report to file
  --lang             Report language: en (default) or zh-TW
  --only             Comma-separated: typo,logic,figure,citation,format,guidelines,claim
  --skip             Comma-separated checkers to skip
  -v, --verbose      Show agent tool calls in real-time
  --dry-run          Only parse .docx, don't run agents
  --init             Generate default .sub-checker.yaml

Pipeline (Plan-Execute-Verify)

Checkers   │  7 agents run concurrently (global concurrency cap, default 3 at a time)
Validate   │  Deterministic post-validation (date math, citation cross-check)
Review     │  Reviewer agent scores every surviving finding → confidence
Dedup      │  Same issue from multiple checkers collapsed to the highest-confidence one
  • Pre-execution: deterministic citation pre-scan + multi-source reference verification
  • Post-validation: false positives filtered, remaining findings get confidence scores (0-100%), cross-checker duplicates merged
  • See harness-architecture.md for full technical details

HTML report features

  • Dark-themed styled report with severity badges
  • Confidence scores — each finding shows reviewer-assigned confidence (%)
  • False positive filtering — deterministic + reviewer agent removes incorrect findings
  • Chain of Thought viewer — expand to see every API call, tool use, and reasoning step
  • Model display — shows which Claude model generated the report
  • i18n support (English / Traditional Chinese)

Cost estimate

By default judgment-heavy checkers (logic, citation_claim, journal_guidelines) and the reviewer run on Claude Opus 4.8, while mechanical checkers (typo, figure, citation_exist, citation_format) run on the cheaper Claude Sonnet 4.6. Per-checker effort is tuned to keep token use low. Measured on a real ~5,000-word, 25-reference manuscript:

Scope Agents Time Cost
Quick check --only figure,citation ~3 min ~$1
Standard --skip claim ~6 min ~$2–3
Full check all 7 agents + harness ~10–12 min ~$3–5

Cost scales with manuscript length and reference count (citation_claim verifies each reference against PubMed/Semantic Scholar/Crossref). Override any model in .sub-checker.yaml (e.g. set everything to claude-sonnet-4-6 for cheaper runs).

Logging

All logs are stored in ~/.sub-checker/:

  • logs/sub-checker.log — application log (auto-rotated, 10MB x 5)
  • logs/sub-checker.error.log — errors only
  • cot/ — agent chain-of-thought JSON logs (every tool call, every response)

Set cot_dir: "disabled" in .sub-checker.yaml to turn off COT file logging (entries still appear in HTML reports).

Public journal-guideline pages and search results are cached in ~/.cache/sub-checker/web.json for 30 days. Configure or disable this in .sub-checker.yaml:

web_cache_dir: "~/.cache/sub-checker"  # null disables persistent web caching
web_cache_max_age_days: 30

Architecture

  • Plan-Execute-Verify harness: concurrent checkers → deterministic validation → reviewer → dedup (ADR-0010)
  • 7 agents + reviewer agent, each with system prompt + curated tools + agentic loop (ADR-0002)
  • Parser provides raw data; agents judge document structure (ADR-0009)
  • Multi-source citation verification: PubMed + Semantic Scholar + Crossref (ADR-0005)
  • FastAPI + React + TypeScript GUI (ADR-0006)
  • Benchmark comparison | Harness architecture

License

MIT

About

Pre-submission manuscript checker with specialized Claude agents and a Plan-Execute-Verify harness.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages