feat: request and persist OCR 4 blocks + confidence scores - #27
Merged
Conversation
Adds --include-blocks/--no-include-blocks (INCLUDE_BLOCKS) and
--confidence-scores-granularity {page,block,word}
(CONFIDENCE_SCORES_GRANULARITY), forwards both to the OCR API, and writes
the returned payload to a blocks.json sidecar in the doc dir.
Both default OFF. The sidecar sits beside metadata.json, never inside it
(DocMetadata is a closed field set) and never under figures/ (the only
directory the conformance harness enumerates).
Fingerprint: the new keys are added to the extra dict ONLY when enabled.
run_fingerprint hashes the whole dict, so emitting include_blocks=False
would change every fingerprint the moment this shipped and force a full
re-OCR of existing corpora at $4/1k pages for a flag nobody turned on.
Omitting at default keeps an untouched run fingerprint-identical to a
pre-feature run while still forcing re-OCR when a flag is toggled.
Sidecar page keys are the globally re-indexed 1-based numbers used by the
## Page N headers, so it joins to the body across chunked PDFs. Base64 is
stripped: figures are already written to figures/, and the API's image ids
are request-local. Turning blocks off unlinks any stale sidecar. The write
happens inside save_results so a sidecar failure is caught by _persist's
isolation rather than leaving status=completed beside a truncated file.
Item 5 of the issue (record the resolved response.model) is dropped: the
API echoes back whichever model string it was sent, so there is nothing to
resolve. Documented in the README instead.
Closes #23
|
Warning Review limit reached
Next review available in: 47 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Summary by QodoAdd optional OCR4 blocks + confidence capture with blocks.json sidecar
AI Description
Diagram
High-Level Assessment
Files changed (5)
|
Code Review by Qodo
1. Block confidence scores dropped
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Comment on lines
+603
to
+606
| if self.config.include_blocks: | ||
| blocks = getattr(page, "blocks", None) | ||
| if blocks: | ||
| payload["blocks"] = self._jsonable(blocks) |
There was a problem hiding this comment.
1. Block confidence scores dropped 🐞 Bug ≡ Correctness
Selecting block granularity without --include-blocks is accepted, but _page_block_payload discards page.blocks unless the separate blocks flag is true. Because Mistral stores block-level scores on each block, this combination silently writes only page-level confidence data and loses the requested per-block scores.
Agent Prompt
## Issue description
Block confidence scores are nested in `page.blocks`, but those blocks are persisted only when `include_blocks` is independently enabled. A user selecting block granularity alone therefore loses the requested block-level scores.
## Issue Context
Either automatically enable/request and persist blocks for block granularity, or reject that option combination with a clear configuration error. Add coverage for block granularity without an explicit blocks flag using the real response shape.
## Fix Focus Areas
- mistral_ocr/processor.py[324-327]
- mistral_ocr/processor.py[603-610]
- mistral_ocr/cli.py[91-98]
- tests/test_blocks.py[80-87]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
+324
to
+325
| if self.config.include_blocks: | ||
| ocr_kwargs["include_blocks"] = True |
There was a problem hiding this comment.
2. Sdk floor rejects blocks 🐞 Bug ≡ Correctness
The project still allows mistralai>=2.9, although older allowed 2.x clients do not define the newly passed include_blocks argument. Enabling blocks in such an existing environment fails at runtime instead of being rejected during dependency resolution.
Agent Prompt
## Issue description
The new request keyword is not supported by every `mistralai` version permitted by the current dependency range, causing `--include-blocks` to fail at runtime in valid installations.
## Issue Context
Determine the first 2.x SDK release that exposes `include_blocks` on `ocr.process` and raise the lower dependency bound to that version. Alternatively, add an explicit compatibility check that fails early with a clear message.
## Fix Focus Areas
- pyproject.toml[26-29]
- mistral_ocr/processor.py[324-327]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| for page_no in sorted(result.page_blocks) | ||
| ], | ||
| } | ||
| sidecar.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding="utf-8") |
There was a problem hiding this comment.
3. Missing sidecar remains cached 🐞 Bug ☼ Reliability
A previously written blocks.json is not represented in completion metadata or checked by resume logic. If that sidecar is later deleted or corrupted, an ordinary rerun with the same fingerprint skips processing and leaves the completed document without its requested auxiliary payload.
Agent Prompt
## Issue description
Resume validation checks the canonical markdown output but has no state describing whether a blocks sidecar was produced. A sidecar that existed after a successful run can disappear or become invalid while the document continues to be treated as complete.
## Issue Context
Persist enough sidecar state to distinguish an intentionally absent payload from a produced sidecar, then verify its existence and JSON validity before accepting a cache hit. Force reprocessing when the recorded auxiliary output is missing or invalid.
## Fix Focus Areas
- mistral_ocr/processor.py[693-722]
- mistral_ocr/processor.py[832-849]
- mistral_ocr/processor.py[1008-1021]
- mistral_ocr/processor.py[1075-1083]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Comment on lines
687
to
+688
| markdown_path.write_text(body, encoding="utf-8") | ||
| self._write_blocks_sidecar(result, doc_dir) |
There was a problem hiding this comment.
4. Sidecar write not covered by _persist isolation as claimed 🐞 Bug ☼ Reliability
_write_blocks_sidecar is called directly inside save_results after the markdown is already written; if it raises, the exception propagates up through save_results into _persist's try block so the document is recorded status=failed even though the OCR itself succeeded and a valid markdown file was already written to disk. This means a transient sidecar I/O error (e.g. disk full) discards a perfectly good, already-persisted markdown result and marks the whole document as failed, contradicting the intended “markdown succeeds independently” design.
Agent Prompt
## Issue description
A failure while writing the `blocks.json` sidecar in `_write_blocks_sidecar` currently propagates and causes the entire document to be recorded as `status=failed`, even though the markdown body OCR succeeded and was already written to disk.
## Issue Context
`save_results` writes markdown, then unconditionally calls `_write_blocks_sidecar`. Both run inside `_persist`'s try/except, so any exception in the sidecar write (disk full, permission error, non-JSON-serializable payload) discards the successful markdown result.
## Fix Focus Areas
- mistral_ocr/processor.py[684-691]
- mistral_ocr/processor.py[693-722]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #23. Builds on #26 (mistralai 2.x), now merged.
Summary
Adds
--include-blocks/--no-include-blocks(INCLUDE_BLOCKS) and--confidence-scores-granularity {page,block,word}(
CONFIDENCE_SCORES_GRANULARITY), forwards both to the OCR API, and persists thereturned payload to a
blocks.jsonsidecar in the doc dir.The two open questions, decided
Defaults: both off. Existing scripts get zero new files, zero new cost, and —
critically — an unchanged fingerprint.
Fingerprint: the new keys enter
extraonly when enabled. This is the oneplace I departed from the issue text, and it is deliberate.
run_fingerprinthashes the whole
extradict, so emittinginclude_blocks: Falsewould changeevery fingerprint the moment this shipped, invalidating every completed document
and forcing a full re-OCR of existing corpora at $4/1k pages — for a flag nobody
turned on. Omitting at default keeps an untouched run fingerprint-identical to a
pre-feature run, while toggling a flag still forces re-OCR (adding or removing a
key changes the hash).
test_defaults_do_not_change_the_fingerprintpins theexact pre-feature digest so this can never regress silently.
Item 5 is dropped. The API echoes back whichever model string it was sent
(evidence in the issue thread), so there is no resolved id to record. Documented
in the README instead: pin
--model mistral-ocr-4-1if you need the version inmetadata.json.Design notes
metadata.json— not inside it (DocMetadatais a closedfield set) and not under
figures/(the only directory conformance enumerates).## Page Nheaders, so the sidecar joins to the body even for chunked PDFs.
figures/, and the API'simage ids are request-local, so embedded payloads would bloat the file with
bytes that resolve to nothing.
save_results, so a sidecar failure is caught by_persist's isolation rather than leavingstatus=completedbeside atruncated file.
from one fingerprint next to metadata from another.
Test plan — observed results
uv run pytest→ 161 passed (141 existing, unmodified + 20 new intests/test_blocks.py).ruff check/ruff format --checkclean. mypy 18errors, unchanged from main — no new ones.
New tests cover kwargs construction, all three granularities, the five
fingerprint behaviours, sidecar shape, page-number alignment, base64 stripping,
stale-sidecar removal, the no-blocks-returned case, and body byte-identity.
Live API, 2-page PDF, OCR 4.1:
--include-blocks --confidence-scores-granularity blockproducedblocks.jsonwith 3 blocks on page 1, keys
type / top_left_x / top_left_y / bottom_right_x / bottom_right_y / content / confidence_scores, and page-level scores..mdis byte-identical (cmpclean) to the no-blocks run, and containsno
bbox/confidence/typetokens.fp:1c85ec03…vs withoutfp:e1ece341…— differs, sotoggling forces re-OCR.
fp:e1ece341…, the same digest asbefore the feature existed.
leaving the body identical.
One shape correction worth recording: the issue assumed a
bboxfield. The liveAPI returns four scalar corner fields instead. Test fixtures match the real shape.
Out of scope
socr consuming the sidecar (r-uben/socr, after a bake-off). Also unaddressed, and
worth its own ticket:
_is_retryablehas two dead limbs on both SDKs, noted in#26.
Summary by cubic
Adds optional OCR 4 per-page blocks and confidence scores and persists them to a blocks.json sidecar. Previously we did not request or store these; now, when enabled, we forward flags to the API, write the sidecar, and keep the markdown body and default fingerprints unchanged.
Rollout
Written for commit bc32d86. Summary will update on new commits.