Skip to content

fix: extract valid encoded images from PDFs - #10795

Open
AIdafashi wants to merge 2 commits into
AstrBotDevs:masterfrom
AIdafashi:fix/pdf-image-encoding
Open

AIdafashi wants to merge 2 commits into
AstrBotDevs:masterfrom
AIdafashi:fix/pdf-image-encoding

Conversation

@AIdafashi

@AIdafashi AIdafashi commented Oct 26, 2026 •

Copy link
Copy Markdown

PDFParser saved decoded pixel buffers as image files and guessed MIME types from a single PDF filter. Flate images were unreadable, filter-chained JPEGs were mislabeled, and nested images were omitted.

Use pypdf's public page.images API to preserve encoded image bytes and derive the actual extension/MIME type. Isolate image and page failures so remaining images and text still load. Failed extraction now logs the file, page, image (when known), and exception details, addressing the review feedback.

Validation

Latest update, Windows / Python 3.12:

  • python -m pytest tests/test_pdf_parser.py -q: 8 passed.
  • Real PDF coverage includes PNG/JPEG, transparency, nested Form XObjects, multiple pages, malformed images, page image-enumeration failure, and text preservation. The failure tests also verify contextual warnings.
  • Ruff format/check and git diff --check: passed.

The original patch reproduced five failing cases before the fix and passed its seven-test suite with both pypdf 6.21.2 and the minimum supported 6.1.1. The added logging test was run with the current environment.

No new dependencies or API changes. Existing stored media are not rewritten; affected documents must be parsed again. The full repository suite was not run. Upstream Actions require maintainer approval; local results are not a claim that upstream CI passed.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/knowledge_base/parsers/pdf_parser.py" line_range="73" />
<code_context>
                         )
                     except Exception:
-                        # 单个图片提取失败不影响整体
+                        # A broken image must not hide other images on the page.
                         continue
             except Exception:
</code_context>
<issue_to_address>
**Image failures go unnoticed**

When pypdf or Pillow raises while retrieving or encoding an image, the per-image `except Exception` discards the failed image and returns the remaining media without recording the exception, so the upload succeeds with missing media and operators have no indication extraction failed.

Log the image/page context and exception before continuing so extraction remains isolated but failures are observable.
</issue_to_address>

Sourcery assessment

Approval pending. 1 finding to address first.

Blocking findings: astrbot/core/knowledge_base/parsers/pdf_parser.py:73


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread astrbot/core/knowledge_base/parsers/pdf_parser.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants