Skip to content

fix: isolate list-form converter metadata - #13779

Merged
anakin87 merged 5 commits into
deepset-ai:mainfrom
lll-hhh:fix/list-meta-isolation-v2
Nov 20, 2026
Merged

anakin87 merged 5 commits into
deepset-ai:mainfrom
lll-hhh:fix/list-meta-isolation-v2

Conversation

@lll-hhh

@lll-hhh lll-hhh commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

  • Deep-copy each entry returned by normalize_metadata() for list-form metadata, so converter output no longer aliases the caller's dictionaries.
  • Copy repeated entries independently, covering [shared, shared] where copying the list once would still leave both documents sharing nested values.
  • Add unit and converter integration regression tests plus a release note.

How did you test it?

  • python -m pytest test/components/converters/test_utils.py test/components/converters/test_textfile_to_document.py test/components/converters/test_multi_file_converter.py -m "not integration" -q -> 28 passed, 11 deselected
  • uvx --from ruff==0.16.0 ruff check ... -> passed
  • uvx --from ruff==0.16.0 ruff format --check ... -> passed
  • uvx mypy --python-executable <venv-python> ... -> no issues in 3 source files
  • uvx pre-commit run --files ... -> passed

The reproduction fails before the fix (3 failed) and the converter regression test passes after it. On this Windows host, two unrelated integration cases classify CSV as application/vnd.ms-excel, so the integration marker was deselected.

Notes for the reviewer

The list path previously returned its input unchanged. A single deepcopy(meta) would still preserve aliases between equal dictionary objects from the caller, so the fix copies per source with [deepcopy(item) for item in meta].

This PR was prepared with an AI assistant under the account owner's authorization. The assistant ran the tests listed above; no separate manual line-by-line review is claimed.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have added unit tests and updated the docstring.
  • I've used a conventional commit type for the PR title.
  • I have added a release note file.
  • I have run pre-commit hooks and fixed any issue.
  • Allow edits and access to secrets by maintainers is enabled.

List-form converter metadata keeps aliases to each caller dictionary, including repeated entries. Pin the expected per-source copy behavior before changing normalize_metadata().
Deep-copy each list entry so converter output cannot mutate caller metadata, and repeated dictionary entries receive independent copies.
@lll-hhh
lll-hhh requested a review from a team as a code owner November 16, 2026 18:54
@lll-hhh
lll-hhh requested review from anakin87 and removed request for a team November 16, 2026 18:54
@vercel

vercel Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

@lll-hhh is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Nov 16, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions Bot added topic:tests P3 Low priority, leave it in the backlog type:documentation Improvements on the docs labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/components/converters
  utils.py
  haystack/core/pipeline
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

@anakin87 anakin87 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

I pushed a minor simplification

@anakin87
anakin87 enabled auto-merge (squash) November 20, 2026 16:20
@anakin87
anakin87 merged commit 32067b9 into deepset-ai:main Nov 20, 2026
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P3 Low priority, leave it in the backlog topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

List-form converter metadata remains shared with caller and other documents

3 participants