Skip to content

fix: let Pipeline.connect accept a Literal output for an input of its value type - #13642

Merged
davidsbatista merged 4 commits into
deepset-ai:mainfrom
aryamangodara:fix/literal-output-to-base-type-connection
Nov 20, 2026
Merged

davidsbatista merged 4 commits into
deepset-ai:mainfrom
aryamangodara:fix/literal-output-to-base-type-connection

Conversation

@aryamangodara

@aryamangodara aryamangodara commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

Pipeline.connect() rejected a Literal output whenever the receiver wasn't itself a Literal, even though every value the sender can produce is valid for the receiver:

from typing import Literal
from haystack import Pipeline, component

@component
class Classifier:
    @component.output_types(label=Literal["positive", "negative"])
    def run(self, text: str):
        return {"label": "positive"}

@component
class Echo:
    @component.output_types(out=str)
    def run(self, text: str):
        return {"out": text.upper()}

p = Pipeline()
p.add_component("clf", Classifier())
p.add_component("echo", Echo())
p.connect("clf.label", "echo.text")
# PipelineConnectError: Cannot connect 'clf.label' with 'echo.text': their declared input and output types do not match.

Cause: _strict_types_are_compatible only handled Literal → Literal (by membership). For any other receiver, the sender's origin (Literal) differed from the receiver's (str), so the check returned False. Literal["a"] -> str | None, Literal[None] -> str | None, Literal[Enum.X] -> Enum and Literal["a"] -> list[str] (wrap) all failed the same way.

Fix: a Literal sender is now treated as the set of its values. Each value has to fit the receiver:

  • by its type (type(value) checked against the receiver with the existing strict rules), or
  • when the receiver is a Union, by being a member of one of its Literal arguments or matching one of its other arguments. That way mixed cases like Literal["a", 1] -> Literal["a"] | int work, and Literal["a", "x"] -> Literal["a"] | int is still rejected.

Literal -> Literal keeps its membership rule, the reverse direction (str -> Literal[...]) stays incompatible, and bare Literal stays incompatible.

How did you test it?

  • Added compatible cases to the asymmetric strict cases in test/core/test_type_utils.py. That list also asserts the reverse direction is incompatible. The new cases cover str, int, enum, Optional/PEP 604 unions, mixed literal to a union of types or literals, and list[Literal[...]] -> list[str].
  • Added incompatible cases: a literal to a different primitive, a mixed literal to a type covering only some of its values, and a literal that only partly matches a union.
  • Added test_connect_literal_output_to_str_input to test/core/pipeline/test_pipeline_base.py.
  • hatch run test:unit test/core test/components/routers test/components/converters/test_output_adapter.py: 2154 passed. hatch run fmt and hatch run test:types are clean.

Notes for the reviewer

The change is a single early branch in _strict_types_are_compatible, placed before the generic Union receiver branch so mixed literals can be split per value. A side effect: Literal[...] -> list[T] now gets the existing WRAP conversion, because wrapping reuses the strict check on the inner type.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

🤖 Generated with Claude Code

… value type

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aryamangodara
aryamangodara requested a review from a team as a code owner November 9, 2026 14:31
@aryamangodara
aryamangodara requested review from davidsbatista and removed request for a team November 9, 2026 14:31
@vercel

vercel Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

@aryamangodara is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/core
  type_utils.py
  haystack/core/pipeline
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

@lxcxjxhx lxcxjxhx left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new branch is right, and it is right for a reason worth naming: it compares type(value), not value.

return all(_strict_types_are_compatible(type(value), receiver) for value in sender_args)

That matters for Literal[True] and Literal[1]. Python says True == 1 and True in (1,), so a membership test cannot tell them apart — but type(True) is bool and type(1) is int can, and bool/int then go through the existing issubclass path. If the branch had been written as all(value in receiver_args ...) or by comparing values directly, Literal[True] would have been accepted for an int input that cannot actually take a bool safely in the other direction.

One inconsistency this leaves behind, which I am flagging rather than asking you to fix here. A few lines down, the Literal-vs-Literal case still uses membership:

if sender_origin is receiver_origin is Literal:
    return all(value in receiver_args for value in sender_args)

So Literal[True] → Literal[1] passes that branch (True in (1,)), while your new branch would reject it. The two now disagree about the same pair of types depending on whether the receiver is a Literal. That is pre-existing, not something this PR introduces — but since this PR is exactly about how a Literal relates to a non-Literal, it is the natural place to note it.

If you want them consistent, the Literal-vs-Literal branch could compare (type(v), v) pairs instead of bare values. If you would rather keep this PR narrow, a comment on the membership line saying "values compare by equality, so True matches Literal[1]" would at least record the decision.

The Union recursion is also sound: _strict_types_are_compatible(Literal[value], union_arg) re-enters this same branch when union_arg is a plain type, and the inner call gets type(value), which is no longer a Literal — so it terminates.

@aryamangodara

Copy link
Copy Markdown
Contributor Author

Thanks for the review, @lxcxjxhx! Agreed: the True/1 equality behavior in the Literal → Literal membership check predates this PR, so I have kept the scope narrow here. Happy to follow up separately if the maintainers want it changed.

@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
haystack-docs Ready Ready Preview Nov 20, 2026 3:41pm UTC

Request Review

@davidsbatista

Copy link
Copy Markdown
Contributor

Thanks! Can you also add a test for Literal["a"] can now feed list[str] ?

@DevNeto07 DevNeto07 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed fbc958d against its main merge parent bbf5b2b.

Following the maintainer's question about Literal["a"] feeding list[str], I exercised a real two-component Pipeline with that connection:

  • Pipeline.run({}) and Pipeline.run_async({}) both deliver ["a"] to the list[str] consumer.
  • The scalar Literal["a"] to list[list[str]] connection remains rejected.

This uses the existing WRAP conversion: the new strict Literal-to-str compatibility enables the list receiver through _get_conversion_strategy, rather than treating the scalar as already being a list. A regression for the connection's runtime output in both execution modes would pin that distinction.

The two existing type-utils and pipeline-base unit modules passed with 1,294 tests and 2 integration tests deselected. Commands were run through a separate Hatch environment with the project core dependencies and focused test dependencies. External-provider, document-converter integrations, and the full project suite were outside this validation.

No introduced correctness defect found in the reviewed diff.

Review assisted by Codex and Claude Code.

@davidsbatista

Copy link
Copy Markdown
Contributor

@DevNeto07 please don't post unsolicited automated reviews on PRs in this repo.

Reviews like this just add noise by repeating what's already in the thread. If you'd like to contribute, the most helpful thing is see open issues.

Also, note that if automated agent reviews like this keep appearing, we'll block the account from the repository.

@DevNeto07

Copy link
Copy Markdown

Understood. I apologize for the unsolicited automated reviews, including the later review posted after your warning. I will respect your request and will not post further unsolicited automated reviews in this repository.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@aryamangodara

Copy link
Copy Markdown
Contributor Author

Thanks! Added it: Literal["a"] to list[str] is now covered at the type level and with an end-to-end pipeline test (sync and async) asserting the receiver gets ["a"].

@github-actions github-actions Bot added the type:documentation Improvements on the docs label Nov 20, 2026

@davidsbatista davidsbatista left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍🏽

@davidsbatista
davidsbatista merged commit a597d31 into deepset-ai:main Nov 20, 2026
26 checks passed
@aryamangodara
aryamangodara deleted the fix/literal-output-to-base-type-connection branch November 20, 2026 16:10

This branch was successfully deployed

1 active deployment
Preview — ddc1d8ad Deployed Nov 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline.connect rejects a Literal output connected to an input of its value type (e.g. Literal["a", "b"] -> str)

4 participants