feat(thumbnail): render PDF thumbnails with pypdfium2 instead of pdf2image#5108
feat(thumbnail): render PDF thumbnails with pypdfium2 instead of pdf2image#5108Austin-s-h wants to merge 2 commits into
Conversation
…image Replace the pdf2image/Poppler subprocess path with pypdfium2 native bindings in a dedicated pdf_thumbnail helper module: - Simpler, cross-platform dependency (no Poppler system binary required); a pdftoppm/pdfinfo fast path is still used when those binaries are present, falling back to pypdfium2 otherwise. - Fix a per-render file-handle/mmap leak by closing the PdfDocument and page in try/finally. - Configurable render DPI via PDF_PREVIEW_DPI (default 300, clamped 72..600); resize with LANCZOS and report pdf_render_dpi / pdf_resize_filter in the response info. - Guard pptx_to_pdf against a missing libreoffice binary with a clean PDFThumbError. Swaps the pdf2image dependency for pypdfium2 in pyproject.toml; all other pins are unchanged. Adds tests/test_pdf_thumbnail.py (11 tests). Touches thumbnail/__init__.py alongside sibling PR feat/pdf-preview-2048; a trivial merge-order conflict is expected and resolvable by taking both hunks. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5108 +/- ##
==========================================
+ Coverage 49.46% 49.55% +0.09%
==========================================
Files 843 844 +1
Lines 34411 34476 +65
Branches 5826 5826
==========================================
+ Hits 17020 17085 +65
Misses 15502 15502
Partials 1889 1889
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| pdf_page = document[page_index] | ||
| try: | ||
| bitmap = pdf_page.render(scale=dpi / 72) | ||
| return bitmap.to_pil() | ||
| finally: | ||
| pdf_page.close() |
There was a problem hiding this comment.
The
PdfBitmap returned by pdf_page.render() is not closed before the function returns. While CPython's reference counting usually frees the native FPDF_BITMAP allocation immediately, it's better to mirror the pattern already used for pdf_page and document — especially under memory pressure in Lambda where GC timing matters.
| pdf_page = document[page_index] | |
| try: | |
| bitmap = pdf_page.render(scale=dpi / 72) | |
| return bitmap.to_pil() | |
| finally: | |
| pdf_page.close() | |
| pdf_page = document[page_index] | |
| try: | |
| bitmap = pdf_page.render(scale=dpi / 72) | |
| try: | |
| return bitmap.to_pil() | |
| finally: | |
| bitmap.close() | |
| finally: | |
| pdf_page.close() |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| DEFAULT_PDF_RENDER_DPI = 300 | ||
| MAX_PDF_RENDER_DPI = 600 |
There was a problem hiding this comment.
shutil.which() is called on every invocation of render_pdf_page and count_pdf_pages. In a warm Lambda the binary layout never changes between requests, so re-probing PATH on every call is wasted work. Caching at module load time (using None as a sentinel for "binary absent") avoids the redundant filesystem lookups.
| DEFAULT_PDF_RENDER_DPI = 300 | |
| MAX_PDF_RENDER_DPI = 600 | |
| DEFAULT_PDF_RENDER_DPI = 300 | |
| MAX_PDF_RENDER_DPI = 600 | |
| _PDFTOPPM = shutil.which("pdftoppm") # None when Poppler is not installed | |
| _PDFINFO = shutil.which("pdfinfo") |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
- Close the PdfBitmap returned by page.render() after to_pil() to avoid leaking native buffers, mirroring the existing page/document cleanup. - Resolve pdftoppm/pdfinfo paths once at import (_PDFTOPPM/_PDFINFO) instead of probing PATH on every render/count call; update call sites and tests to patch the cached module attributes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Replaces the
pdf2imagePython dependency in the thumbnail lambda withpypdfium2native bindings, extracted into a dedicatedpdf_thumbnail.pyhelper.pdf2imagelibrary — the only dependency change is swappingpdf2imageforpypdfium2>=4,<5inpyproject.toml; all other pins and thet4-lambda-sharedsource are unchanged.pdftoppm/pdfinfoare present onPATHthey're still used as a fast path; when they aren't,pypdfium2renders natively. This is a compatibility improvement (native fallback), not a removal of the Poppler code path — the subprocess route is retained.PdfDocumentand page are closed intry/finallyon every render and page-count.PDF_PREVIEW_DPI(default 300, clamped to 72–600); pages are resized with LANCZOS, andpdf_render_dpi/pdf_resize_filterare reported in the response info.pptx_to_pdfagainst a missinglibreofficebinary with a cleanPDFThumbError.Adds
tests/test_pdf_thumbnail.py(11 tests, all passing locally viauv run pytest).Note
Touches
lambdas/thumbnail/src/t4_lambda_thumbnail/__init__.pyalongside sibling PR #5107 (raise PDF preview resolution to 2048x1536). A trivial merge-order conflict is expected and resolvable by taking both hunks.🤖 Generated with Claude Code
Greptile Summary
This PR replaces the
pdf2image/Poppler subprocess dependency in the thumbnail lambda withpypdfium2native bindings, extracted into a newpdf_thumbnail.pyhelper. Apdftoppm/pdfinfofast path is preserved when those binaries are present onPATH, withpypdfium2as the fallback.pdf2imageremoved frompyproject.tomlanduv.lock;pypdfium2>=4,<5(locked to 4.30.0) added — no Poppler system binary required in the base Lambda image.PdfDocumentandPdfPageare now closed intry/finallyblocks on every render and page-count, fixing the per-render file-handle/mmap leak mentioned in the PR description. The intermediatePdfBitmapfromrender()is not yet wrapped in its owntry/finally.PDF_PREVIEW_DPI(clamped 72–600, default 300);pdf_render_dpiandpdf_resize_filteradded to response info.pptx_to_pdfnow raisesPDFThumbErroron missinglibreofficerather than lettingFileNotFoundErrorpropagate.Confidence Score: 4/5
Safe to merge; the core rendering logic is sound, error handling is improved over the previous implementation, and the 11 new tests cover all key branches.
The migration from pdf2image to pypdfium2 is clean and well-tested. The only items worth addressing are the PdfBitmap not being explicitly closed after to_pil(), and shutil.which() being probed on every request instead of once at module load. Neither affects correctness in CPython reference-counting GC, but worth tightening before shipping to production.
lambdas/thumbnail/src/t4_lambda_thumbnail/pdf_thumbnail.py — bitmap lifecycle and repeated shutil.which() calls