Skip to content

ocr: add hpd-parsing-server.py (HPD-Parsing 1B, vendor vLLM fork, server-only) - #95

Open
davanstrien wants to merge 3 commits into
mainfrom
ocr/hpd-parsing-server
Open

ocr: add hpd-parsing-server.py (HPD-Parsing 1B, vendor vLLM fork, server-only)#95
davanstrien wants to merge 3 commits into
mainfrom
ocr/hpd-parsing-server

Conversation

@davanstrien

Copy link
Copy Markdown
Owner

Adds ocr/hpd-parsing-server.py for PaddlePaddle/HPD-Parsing (1B, Apache-2.0) — a document parser that forks a child decoding branch per page region, with a medusa P-MTP head drafting inside each branch. 94.91 OmniDocBench v1.6 at 4,752 TPS.

It earns a slot on layout output + throughput, not accuracy — 94.91 sits below ovis-ocr2 (96.58) and paddleocr-vl-1.6 (96.33). What it gives that those don't is its native <BLOCK> <type> [bbox] <CHILD> stream: block types, bboxes and reading order in one pass, kept in an hpd_blocks column alongside the markdown.

Two things reviewers should look at first

1. First non-Docker-Hub image in the collection. The model only runs on Paddle's vLLM fork (0.17.1+hpdparsing), which ships in their own registry image. It pulls anonymously (verified) and HF Jobs can fetch it, but it's ~11 GB and every job re-pulls it. build_serve_args() is the image's own /home/hpd/entrypoint.sh, not our translation.

2. Server-only by design, no offline sibling. Forking is a scheduler feature, so an offline llm.generate loop forfeits the point of the model — the vendor agrees, their image entrypoint is the serve command.

Sharp edges (all in ocr/CLAUDE.md)

  • Vendor image ships no uv, so hf jobs uv run --image fails; the job command bootstraps it
  • The entrypoint's speculative path PaddlePaddle/HPD-Parsing/P-MTP is neither a repo id nor a directory — vLLM rejects it outside the image's offline mode; resolve_spec_model() materializes it locally
  • --enable-prefix-caching stays on, against SERVING.md's usual OCR pattern, because forked children share the parent prefix KV (52.9% hit rate measured)
  • bboxes are 0–1000 normalized, not pixels (unlike surya_blocks), stamped in inference_info.bbox_space

Testing

Set Result
ufo-ColPali 5 + shuffled 10 15/15, 0 errors, 0 tag leaks, 0 out-of-range bboxes; bbox space confirmed by overlay render
OmniDocBench 12 shuffled 12/12, 0 errors; covers tables (HTML w/ rowspan/colspan) + equations (LaTeX)
Chronicling America 12 (1774–1809) 10/12 excellent — within 1% of LoC's own OCR length and cleaner — 2/12 repetition collapse

Known failure mode, documented not fixed

On dense 1809 advertisement pages that fragment into 279–340 blocks, output degenerates into repetition loops; one page emitted 726,706 chars against 24,907 in the reference OCR. Structural cause: --max-tokens bounds one branch, not the page, so hundreds of forked children can each spend the full budget. Nothing errors and nothing truncates. A per-block repetition guard + repetition flag is the follow-up (pairs with the deferred ocr_error column); flagging rather than silently trimming, since upstream's own postprocessing does no repetition cleanup.

Also unexercised: chart and seal blocks, the two types whose text is deliberately suppressed.

🤖 Generated with Claude Code

davanstrien and others added 3 commits July 27, 2026 13:20
…ver-only)

PaddlePaddle/HPD-Parsing parses a page with hierarchical parallel decoding: a
layout branch forks a child branch per region (children reuse the parent's
prefix KV) with a medusa P-MTP head drafting inside each branch. 94.91
OmniDocBench v1.6 at 4,752 TPS.

Server-only by design — forking is a scheduler feature of Paddle's vLLM fork
(0.17.1+hpdparsing), so an offline generate loop forfeits the point of the
model. The fork ships only in the vendor image, whose own entrypoint is the
serve command that build_serve_args() reproduces.

Recipe writes `markdown` plus `hpd_blocks` (type, bbox, text per region, in
reading order); bboxes are 0-1000 normalized, recorded in inference_info so
downstream code can't guess wrong. `--keep-raw` retains the tag stream.

Four sharp edges documented in ocr/CLAUDE.md:
- vendor image has no uv, so `hf jobs uv run --image` fails; bootstrap uv in
  the job command (first non-Docker-Hub image in the collection, ~11 GB pull)
- the entrypoint's speculative path `PaddlePaddle/HPD-Parsing/P-MTP` is neither
  a repo id nor a directory; resolve_spec_model() materializes it locally
- `--enable-prefix-caching` stays on, against the usual OCR serve pattern,
  because forked children share the parent prefix KV
- bboxes are normalized, not pixels, unlike surya_blocks

Smoke-tested on l4x1 / davanstrien/ufo-ColPali: 5/5 pages, 0 errors, 1.00
img/s, spec-decode mean acceptance length 4.01 @ 50.1% draft acceptance, 52.9%
prefix-cache hit rate; bbox space confirmed by overlay render.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tions)

Third smoke run on 12 shuffled opendatalab/OmniDocBench pages - the model's own
eval set - closes the gap the ufo-ColPali runs left: 6 table blocks (HTML with
rowspan/colspan) and 4 equation blocks (LaTeX) now exercised, 12/12, 0 errors.
chart/seal remain unseen.

Also records two benign log lines so they don't get re-debugged: the P-MTP
'Error retrieving safetensors' (head loads regardless - spec decoding reports
up to 96% draft acceptance) and the tokenizer sequence-length warning on a
61.7 MPx page (no request errored; --max-pixels / --max-model-len are the knobs
if it ever does).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
12 Chronicling America pages (1774-1809, LoC via IIIF): 10 excellent - output
length within 1% of LoC's own OCR and visibly cleaner - but 2 collapsed into
degenerate repetition, both dense 1809 ad pages that fragmented into 279-340
blocks. One emitted 726,706 chars against 24,907 in the reference OCR.

Root cause is structural: hierarchical decoding gives each forked child branch
its own --max-tokens budget, so a page fragmenting into hundreds of regions can
emit ~100x the per-branch limit. Nothing errors, nothing truncates, throughput
craters (0.05 img/s vs 0.48 on OmniDocBench).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant