Skip to content

feat(thumbnail): render PDF thumbnails with pypdfium2 instead of pdf2image#5108

Open
Austin-s-h wants to merge 2 commits into
quiltdata:masterfrom
Austin-s-h:feat/pdf-thumbnail-pypdfium2
Open

feat(thumbnail): render PDF thumbnails with pypdfium2 instead of pdf2image#5108
Austin-s-h wants to merge 2 commits into
quiltdata:masterfrom
Austin-s-h:feat/pdf-thumbnail-pypdfium2

Conversation

@Austin-s-h

@Austin-s-h Austin-s-h commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Replaces the pdf2image Python dependency in the thumbnail lambda with pypdfium2 native bindings, extracted into a dedicated pdf_thumbnail.py helper.

  • Drops the pdf2image library — the only dependency change is swapping pdf2image for pypdfium2>=4,<5 in pyproject.toml; all other pins and the t4-lambda-shared source are unchanged.
  • Poppler is now optional, not required. When pdftoppm/pdfinfo are present on PATH they're still used as a fast path; when they aren't, pypdfium2 renders natively. This is a compatibility improvement (native fallback), not a removal of the Poppler code path — the subprocess route is retained.
  • Fixes a file-handle/mmap leak: the PdfDocument and page are closed in try/finally on every render and page-count.
  • Configurable render DPI via PDF_PREVIEW_DPI (default 300, clamped to 72–600); pages are resized with LANCZOS, and pdf_render_dpi / pdf_resize_filter are reported in the response info.
  • Guards pptx_to_pdf against a missing libreoffice binary with a clean PDFThumbError.

Adds tests/test_pdf_thumbnail.py (11 tests, all passing locally via uv run pytest).

Note

Touches lambdas/thumbnail/src/t4_lambda_thumbnail/__init__.py alongside sibling PR #5107 (raise PDF preview resolution to 2048x1536). A trivial merge-order conflict is expected and resolvable by taking both hunks.

🤖 Generated with Claude Code

Greptile Summary

This PR replaces the pdf2image/Poppler subprocess dependency in the thumbnail lambda with pypdfium2 native bindings, extracted into a new pdf_thumbnail.py helper. A pdftoppm/pdfinfo fast path is preserved when those binaries are present on PATH, with pypdfium2 as the fallback.

  • Dependency swap: pdf2image removed from pyproject.toml and uv.lock; pypdfium2>=4,<5 (locked to 4.30.0) added — no Poppler system binary required in the base Lambda image.
  • Resource management: PdfDocument and PdfPage are now closed in try/finally blocks on every render and page-count, fixing the per-render file-handle/mmap leak mentioned in the PR description. The intermediate PdfBitmap from render() is not yet wrapped in its own try/finally.
  • New features: Configurable render DPI via PDF_PREVIEW_DPI (clamped 72–600, default 300); pdf_render_dpi and pdf_resize_filter added to response info. pptx_to_pdf now raises PDFThumbError on missing libreoffice rather than letting FileNotFoundError propagate.

Confidence Score: 4/5

Safe to merge; the core rendering logic is sound, error handling is improved over the previous implementation, and the 11 new tests cover all key branches.

The migration from pdf2image to pypdfium2 is clean and well-tested. The only items worth addressing are the PdfBitmap not being explicitly closed after to_pil(), and shutil.which() being probed on every request instead of once at module load. Neither affects correctness in CPython reference-counting GC, but worth tightening before shipping to production.

lambdas/thumbnail/src/t4_lambda_thumbnail/pdf_thumbnail.py — bitmap lifecycle and repeated shutil.which() calls

…image

Replace the pdf2image/Poppler subprocess path with pypdfium2 native
bindings in a dedicated pdf_thumbnail helper module:

- Simpler, cross-platform dependency (no Poppler system binary required);
  a pdftoppm/pdfinfo fast path is still used when those binaries are
  present, falling back to pypdfium2 otherwise.
- Fix a per-render file-handle/mmap leak by closing the PdfDocument and
  page in try/finally.
- Configurable render DPI via PDF_PREVIEW_DPI (default 300, clamped
  72..600); resize with LANCZOS and report pdf_render_dpi /
  pdf_resize_filter in the response info.
- Guard pptx_to_pdf against a missing libreoffice binary with a clean
  PDFThumbError.

Swaps the pdf2image dependency for pypdfium2 in pyproject.toml; all other
pins are unchanged. Adds tests/test_pdf_thumbnail.py (11 tests).

Touches thumbnail/__init__.py alongside sibling PR feat/pdf-preview-2048;
a trivial merge-order conflict is expected and resolvable by taking both
hunks.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.55%. Comparing base (3d20021) to head (d4ce2f5).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5108      +/-   ##
==========================================
+ Coverage   49.46%   49.55%   +0.09%     
==========================================
  Files         843      844       +1     
  Lines       34411    34476      +65     
  Branches     5826     5826              
==========================================
+ Hits        17020    17085      +65     
  Misses      15502    15502              
  Partials     1889     1889              
Flag Coverage Δ
api-python 93.14% <ø> (ø)
catalog 25.08% <ø> (ø)
lambda 97.15% <100.00%> (+0.06%) ⬆️
py-shared 98.02% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment on lines +36 to +41
pdf_page = document[page_index]
try:
bitmap = pdf_page.render(scale=dpi / 72)
return bitmap.to_pil()
finally:
pdf_page.close()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The PdfBitmap returned by pdf_page.render() is not closed before the function returns. While CPython's reference counting usually frees the native FPDF_BITMAP allocation immediately, it's better to mirror the pattern already used for pdf_page and document — especially under memory pressure in Lambda where GC timing matters.

Suggested change
pdf_page = document[page_index]
try:
bitmap = pdf_page.render(scale=dpi / 72)
return bitmap.to_pil()
finally:
pdf_page.close()
pdf_page = document[page_index]
try:
bitmap = pdf_page.render(scale=dpi / 72)
try:
return bitmap.to_pil()
finally:
bitmap.close()
finally:
pdf_page.close()

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Comment on lines +12 to +13
DEFAULT_PDF_RENDER_DPI = 300
MAX_PDF_RENDER_DPI = 600

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 shutil.which() is called on every invocation of render_pdf_page and count_pdf_pages. In a warm Lambda the binary layout never changes between requests, so re-probing PATH on every call is wasted work. Caching at module load time (using None as a sentinel for "binary absent") avoids the redundant filesystem lookups.

Suggested change
DEFAULT_PDF_RENDER_DPI = 300
MAX_PDF_RENDER_DPI = 600
DEFAULT_PDF_RENDER_DPI = 300
MAX_PDF_RENDER_DPI = 600
_PDFTOPPM = shutil.which("pdftoppm") # None when Poppler is not installed
_PDFINFO = shutil.which("pdfinfo")

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@Austin-s-h Austin-s-h changed the title thumbnail: render PDF thumbnails with pypdfium2 instead of pdf2image feat(thumbnail): render PDF thumbnails with pypdfium2 instead of pdf2image Jul 7, 2026
- Close the PdfBitmap returned by page.render() after to_pil() to avoid
  leaking native buffers, mirroring the existing page/document cleanup.
- Resolve pdftoppm/pdfinfo paths once at import (_PDFTOPPM/_PDFINFO)
  instead of probing PATH on every render/count call; update call sites
  and tests to patch the cached module attributes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant