Feature/cpu optimizations - #562
Open
OligerMan wants to merge 20 commits into
Open
Conversation
Collaborator
|
NastyBoget
force-pushed
the
feature/cpu-optimizations
branch
from
July 31, 2026 08:57
1535a03 to
567e660
Compare
NastyBoget
reviewed
Jul 31, 2026
| not reusable | ||
| and start == 1 # noqa W503 | ||
| and get_param_pdf_with_txt_layer(parameters) == "auto_tabby" # noqa W503 | ||
| and not get_param_with_attachments(parameters) # noqa W503 |
Collaborator
There was a problem hiding this comment.
Над переиспользованием attachments я бы еще подумала - они в большинстве случаев нужны. Можно на уровне auto reader создавать временную директорию и сохранять аттачи туда при первом чтении, тогда они не потеряются при втором чтении
NastyBoget
force-pushed
the
feature/cpu-optimizations
branch
5 times, most recently
from
July 31, 2026 10:03
1afbe1f to
8b7a7e2
Compare
NastyBoget
reviewed
Jul 31, 2026
Collaborator
There was a problem hiding this comment.
я не могу одобрить мерж, пока исходный код tabby с изменениями не будет запушен и одобрен Андреем Михайловым
NastyBoget
reviewed
Jul 31, 2026
| self.logger.warning(f"pypdfium2 render failed ({error}); falling back to pdf2image") | ||
| yield from self._split_pdftoppm(path, page_from, page_to) | ||
|
|
||
| def _split_pdfium(self, path: str, page_from: int, page_to: int) -> Iterator[ndarray]: |
Collaborator
There was a problem hiding this comment.
Альтернативный способ рендеринга полезен, возможно даже его настраивать лучше не через переменные окружения, а через конфиг. Но нужно написать для него тест
NastyBoget
reviewed
Jul 31, 2026
| parameters_copy["pages"] = "1:8" # two batches for pdf_txtlayer_reader | ||
| parameters_copy["need_pdf_table_analysis"] = "false" | ||
| if pages_reusable: | ||
| parameters_copy["__tabby_raw_pages_out"] = detected_pages |
Collaborator
There was a problem hiding this comment.
Мне не нравится этот способ передачи данных между tabby и детектором текстового слоя из-за его неявности. Еще мне не нравится то, что в tabby мы зашиваем логику от детектора текстового слоя, tabby о ней не должен знать. Я бы сейчас не стала оптимизировать конкретно эту проблему (когда мы читаем первые 8 страниц tabby и не хотим их потом читать снова), потому что в будущем это решение всё равно поменяем, и это не кажется мне очень большой проблемой сейчас. Единственное, что кажется можно сделать попроще - не читать второй раз короткие документы (<8 страниц)
NastyBoget
reviewed
Aug 3, 2026
|
|
||
| # p = fraction of columns with an ink transition, s = ink density. __get_rid_spaces used to sit between | ||
| # base_line_image and s, but its `len(not_space) > 3` guard is the column count (always > 3 for a correct | ||
| # bbox), so it never stripped anything and only wasted a mean(0) -- dropping it is a no-op. (p_img > 0).mean() |
Collaborator
There was a problem hiding this comment.
возможно, в функции __get_rid_spaces был баг, и нужно было проверять not_space.sum()
прежде чем менять код, лучше потестировать старое поведение с исправлением потенциального бага, потому что текущая реализация определения жирности шрифта далеко не всегда работала хорошо
NastyBoget
reviewed
Aug 3, 2026
| crossings) at ~1/50th the detector's cost. Text has horizontal runs but no crossing vertical rules, so a page | ||
| below the threshold has no bordered table the detector could find. (A plain Otsu + long-kernel version missed 12% | ||
| of real tables -- do not simplify further.)""" | ||
| gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) if image.ndim == 3 else image |
Collaborator
There was a problem hiding this comment.
эту строку можно вынести в вызывающую функцию, чтобы не делать то же самое потом еще в __rec_tables_from_img
The bold classifier's ValleyEmphasisBinarizer.__get_threshold was ~37 ms/page, dominated by np.histogram (slow on uint8) plus a 254-iteration Python loop with a per-step np.sum. Replace with cv2.calcHist (per-value, SIMD) rebinned to the same 255-bin edges, and compute the cumulative omega/mu via cumsum and the neighbour-window sum via a cumsum difference. ~47x faster on the histogram (36.5 -> 0.77 ms) and the threshold is bit-identical (verified: same counts, zero threshold diff over 20 pages), so the bold output is unchanged. 297-page DAE report: byte-identical structure (nodes 3657, same text_sha), wall ~56.5 -> ~53.6 s. Now that the CPU pool is the wall bottleneck (per the per-process profile), this CPU cut registers on the wall.
…rizer After vectorizing __get_threshold, the two full-page boolean masks+assigns (image[<=t]=0; image[>t]=1, ~4 passes, 6.9 ms/page) became the dominant cost of ValleyEmphasisBinarizer.binarize. Replace with a single cv2.threshold (THRESH_BINARY, SIMD) -> ~0.5 ms. Byte-identical output (297-page DAE report unchanged: nodes 3657, same text_sha). The binarizer is now threshold (cv2.calcHist + cumsum) + binarize (cv2.threshold), with only the necessary BGR->GRAY cvtColor (0.38 ms) left.
__get_color_annotation built the non-white mask with 5 full-slice numpy passes (3 comparisons + 2 ANDs) and took the per-channel mean with 3 separate boolean-index gathers (image_slice[mask, i].mean() for i in range(3)) -- the gathers dominated at ~20 ms/page. Replace with SIMD cv2: cv2.inRange for the mask, then cv2.sumElems(bitwise_and)/countNonZero for the means. 24.6 -> 5.3 ms/page over 67 lines/page. Bit-identical: the uint8 sum is exact in float64 regardless of order, so channel_sum/count matches the old .mean() exactly (verified max |diff| = 0 over 1334 real line-bboxes; 297-page DAE report unchanged: nodes 3657, same text_sha).
…eval
__evaluation_one_bbox_image called __get_rid_spaces on the baseline band before
computing ink density, but its `len(not_space) > 3` guard tests the column count
(= width, always > 3 for a correct bbox), so it never stripped a column and only
burned a wasted mean(0) (~2.75 ms/page). Drop it and use base_line_image
directly. Also fold the transition count `p_img[abs>0]=1; p_img[<0]=0; mean()`
into a single (p_img > 0).mean() -- algebraically identical on the uint8 {0,1}
baseline. 11.6 -> 7.3 ms/page (-37%). Bit-identical: verified max |eval diff| = 0
over 1373 real word-slices; 297-page DAE report unchanged (nodes 3657, same
text_sha). Baseline-band top-2 loop left as-is (vectorizing it saved only
0.15 ms/page and risked tie-break divergence).
HoughLinesP was ~79% of table detection cost (~540 ms/page). It only needs the line angle (scale-invariant) and to draw gap-filling lines, so run it on a downscaled copy (length/gap params scaled) and upscale the line mask back to full resolution; cell OCR and contours stay full-res. New config key table_hough_scale (default 0.5) halves the table stage (534 -> 210 s over the 297-page doc) with tables and text preserved.
recognize_tables_from_image had no early-exit: it ran full contour/Hough detection on every page. Add a ~7 ms/page line-crossing signal that reproduces the detector's OWN line-detection parameters (fixed-225 threshold, short h/v morphology kernels floored at TableTree.min_w/h_cell) and count grid crossings; skip the detector when a page has fewer than table_line_gate_min_cross (default 2) crossings. Recall preserved -- gen_tables cell_recall/tables-per-doc identical to baseline, DAE tables=6 preserved, and gate ON vs OFF is byte- identical over 60 pages (0/69827 annotation diff: on a tableless page the gate returns the original image, which equals the detector's np.copy). Set table_line_gate_min_cross / DEDOC_TABLE_MIN_CROSS to 0 to disable.
…wall dedocutils SkewCorrector rotates the full page once per candidate angle over arange(-45, 46, 1) = 91 rotations/page -- the dominant cost of the scanned-image pipeline. FastSkewCorrector (a drop-in subclass, wired into PdfImageReader) brackets the peak with a coarse 3-degree sweep on a <=512 px thumbnail (rotation cost scales with area) then refines +-4 degrees at 1-degree step on a 1000 px image, and skips the final rotation when best_angle == 0 (rotate_image(x, 0) is an identity warp, so bit-identical there). Same projection-profile scoring, so the returned angle matches the 91-step whenever the grid brackets the same peak: 29/30 exact on the pdf_profiling/skew set (the lone divergence is a >28-degree page where the peak is inherently ambiguous), exact on realistic skew <=12 deg. DAE 297 pages: wall 1339 -> 1093 s (-18%), body word-bag F1 vs the 91-step output 0.99993, table cells identical, tables=6. DEDOC-style config-free.
In-process PDFium rendering is ~7% faster end-to-end than pdf2image/pdftoppm (no poppler subprocess, no per-batch PDF re-parse; document cached in memory, 2x2 erode to thicken glyphs). Added as _split_pdfium with a dispatcher and a pdf2image fallback. Kept OPT-IN (default DEDOC_RENDER=pdftoppm) because on master's Tesseract path it is NOT quality-neutral: PDFium's thinner glyph anti-aliasing shifts Tesseract's output even with the erode (which was tuned for the hybrid recognizer), measured -0.38% gen_texts word-bag F1 and 0.988 body word-bag F1 vs poppler on the 297-page DAE report. The default path (_split_pdftoppm) is the original code, byte-identical to before.
Report for feature/cpu-optimizations: 4 default fixes (metadata, HoughLinesP, table-gate, deskew) = -28.0% on the 297-page DAE report, all bit-identical or quality-validated; opt-in pdfium render (-7.5% more but degrades Tesseract OCR).
auto_tabby ran the tabby Java subprocess twice per document: once in TxtLayerDetector.__classify_all_pages (first 8 pages, no tables) to feed the text-layer classifier, then again in PdfAutoReader.__parse_document for the real extraction. __parse_document already reuses a detection-provided document (TxtLayerResult.document), but __classify_all_pages never populated it. When the whole document fits inside the 8-page detection window, extract it once with the full parameters (all pages + tables, exactly __parse_document's own read) and pass it through for reuse -- eliminating the second JVM startup + re-extraction. Bit-identical output (verified: identical node/table counts and text SHA on prospectus/VVP_6_tables/short_lines/example), ~2x faster on small text-layer documents (5.93->3.36 s, 2.77->1.36 s, ...). Documents larger than the detection window keep the cheap 8-page detection and are unchanged.
The tabby JAR is invoked as a single java subprocess that uses only ~2.25 of 16 cores (measured: 26 s CPU over 11.6 s wall on the 145-page riscv spec), so a large document leaves the machine mostly idle. Split the page range into contiguous chunks run as concurrent tabby subprocesses (the jar already accepts -sp/-ep) and merge. This is bit-identical: tabby numbers pages absolutely and its per-page output is page-local (verified 145/145 pages byte-identical to the single call), so merging is just concatenating the per-chunk `pages` lists; multi-page tables spanning a chunk boundary are still assembled downstream by convert_to_multipages_tables from the page fragments. riscv 145 pages: full parse 14.15 -> 11.30 s (-20%), identical reader output (node/table counts + text/cell SHA); the tabby extraction step alone goes ~11.6 -> ~6.4 s. Each chunk JVM runs -XX:+UseSerialGC -Xmx1024m to keep N concurrent processes from saturating memory bandwidth with parallel GC threads. Default: documents >= 40 pages split into 2-4 chunks (min 20 pages/chunk, cap 4); smaller docs, the GOST-frame path and open-ended ranges keep the single call. Tunables: tabby_parallel_chunks / DEDOC_TABBY_CHUNKS (cap, 1 disables), tabby_parallel_min_pages_per_chunk, DEDOC_TABBY_JVM_ARGS. Also fixes the data.json read to specify encoding=utf-8 (RU-locale Windows cp1251 bug).
BBoxAnnotation.__init__ called json.dumps(bbox.to_relative_dict(...)) on every construction -- once per line bounding box, tens of thousands of times per document (46919 on the 145-page riscv spec, where the json encoder was the single largest post-processing cost, ~0.57 s). Build the fixed 6-key JSON string directly with an f-string instead: Python's str(float) equals repr and json's float encoding, and int str equals json's, so the output is byte-identical to json.dumps (verified over 122820 bbox values). Micro-benchmark: the whole new constructor (3.02 us) is faster than the old json.dumps call alone (4.78 us).
…an + compact JSON Rebuild the tabby table-extraction engine (ispras_tbl_extr.jar) with CPU optimizations. Engine time on a 145-page document drops ~11.8s -> ~3.7s; end-to-end auto_tabby through dedoc ~13.9s -> ~7.4s (~1.9x). - ruling extraction: the per-page ruling detection (a pixel scan of the rendered page) now runs across all available cores; the page render itself stays serialized, so the result does not depend on scheduling order - ruling scan: read the pixel band value directly instead of allocating an int[] for every pixel -- billions of short-lived allocations on a large document - text extraction: drop a per-character list that was built for every glyph and never read by anything - JSON output: build and serialize each page in parallel and emit compact JSON instead of a single pretty-printed pass; data.json shrinks ~45%, which also speeds up the json.load on the Python side The first three keep data.json byte-identical to the previous jar. Compact JSON changes the bytes but not the parsed content (data.json is consumed with json.load, which is whitespace-insensitive): verified parse-identical across all of tests/data/pdf_with_text_layer, and dedoc's final output (structure, tables, annotations) verified identical end-to-end.
…pying Cell splitting (CellSplitter._merge_close_borders / __split_one_cell) copies a cell per cell and per split sub-cell only to rewrite its geometry and flags -- it never touches the line contents. The lines (text + annotations) are by far the heaviest part of a cell, so deep-copying them was pure overhead. Share the line objects (each copy still gets its own list, so list-level edits stay independent) and deep-copy only the small geometry (bbox, contour_coord), which the splitter and Cell.shift do mutate. -0.21s on a 145-page document (auto_tabby, warm). Verified: dedoc's final output (structure + tables + annotations) is identical on the tabby path (riscv-spec-v2.2, VVP_6_tables, VVP_global_table, big_table_with_merged_cells, Document635) and on the image-reader path (three scanned table images compared against the previous deepcopy behaviour).
…remaining pages The textual layer detection already runs a complete tabby extraction of the first 8 pages -- the tabby reader ignores need_pdf_table_analysis, so tables are extracted too -- and then threw that extraction away, leaving the subsequent full read to extract those pages a second time. Hand tabby's raw per-page output over (TxtLayerResult.detected_pages) so PdfTabbyReader extracts only the pages the detection did not cover. The pages are merged at the raw ``pages`` level, *before* the per-page processing, which is what keeps cross-page merging intact: this is the same invariant __process_pdf_parallel already relies on (absolute page numbers, contiguous disjoint ranges). Splitting the work into separate reads per page range instead tears paragraphs that span the boundary, so it is not done that way. Restricted to auto_tabby (under "auto" the remaining pages are read by pdf_txtlayer_reader, which cannot consume tabby's pages), to reads starting at page 1, and to runs without attachments (extracted image files live in the detection read's temporary directory, which is gone by then). Measured (auto_tabby, warm): article (25p) 7.78s -> 6.03s (-23%), riscv-spec-v2.2 (145p) ~-0.2s; the gain scales with the content density of the first 8 pages. Verified identical dedoc output (structure + tables + annotations) on multipage (the 8/9 boundary case), with_changed_header_footer, mongolo, article, riscv-spec-v2.2, and on example.pdf as an unaffected control.
…ction The chunking existed to work around the old jar using only ~2 of the machine's cores: splitting a large document into contiguous page ranges and running several tabby JVMs concurrently was a net win back then. The rebuilt jar parallelizes internally across all cores, so those extra JVMs now only contend with it -- measured on a 145-page document, auto_tabby warm: 8.25s with the default 4 chunks vs 7.36s with chunking off. Reverts the machinery added in 9be9f63 (__parallel_chunk_count, __process_pdf_parallel, __run's jvm_args) together with its tunables (tabby_parallel_chunks, tabby_parallel_min_pages_per_chunk, DEDOC_TABBY_CHUNKS, DEDOC_TABBY_MIN_PAGES_PER_CHUNK, DEDOC_TABBY_JVM_ARGS), restoring the single-call extraction. The raw-pages reuse from the textual layer detection is unaffected: it merges the ``pages`` lists on the same invariant the chunking relied on (page-local output, absolute page numbers). Output verified identical (dedoc's final structure, tables and annotations) on the 145-page document.
Internal working report, not needed in the pull request.
NastyBoget
force-pushed
the
feature/cpu-optimizations
branch
from
August 3, 2026 13:21
8b7a7e2 to
b0d845d
Compare
NastyBoget
reviewed
Aug 3, 2026
NastyBoget
requested changes
Aug 3, 2026
Collaborator
|
skew corrector нужно будет поправить - я бы предложила искать итоговый точный угол на исходной картинке (без ее уменьшения) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I reworked tabby pipeline to be more optimal for CPU inference.
I decompiled jar and reworked it with some optimizations + some parts of java code became parallel. Can provide decompiled code, didn't push it here because there is no source code for previous version.
Also added some CPU-oriented optimizations, like vectorization for bold classifier, two-staged skew correction and tested pdftoppm vs pdfium(so currently we have choice between faster version without perfect match and older one which is definitely compatible with everything)
Please check what fixes are relevant for merging to master, on my system it made more than x2 speed improvement for documents with correct text layer