Skip to content

fix: keep streamed tool calls apart when an OpenAI-compatible server omits index - #13848

Open
Mattbusel wants to merge 1 commit into
deepset-ai:mainfrom
Mattbusel:fix/openai-stream-tool-call-index
Open

Mattbusel wants to merge 1 commit into
deepset-ai:mainfrom
Mattbusel:fix/openai-stream-tool-call-index

Conversation

@Mattbusel

Copy link
Copy Markdown

Related Issues

None open. Two crashes and a lost id in OpenAIChatGenerator streaming with tools against OpenAI-compatible servers.

Proposed Changes

  • _convert_chat_completion_chunk_to_streaming_chunk passes index=tool_call.index straight into ToolCallDelta (haystack/components/generators/chat/openai.py:824). Gemini's OpenAI-compatible endpoint and Mistral stream tool calls with an id and no index, so StreamingChunk raises ValueError: If tool_call, tool_call_result or reasoning is set, index must also be set. (haystack/dataclasses/streaming_chunk.py:153) and the run fails. This happens on this recorded Gemini response (two parallel calls, no index). The index is now resolved from the previous chunks: a delta with an id not seen yet starts a new tool call, one without an id continues the latest.
  • _convert_streaming_chunks_to_chat_message overwrites the saved id whenever a chunk has id is not None (haystack/components/generators/utils.py:147). Alibaba Cloud Model Studio sends "id": "" on every chunk after the first (recorded qwen3-max stream), so the reply's ToolCall.id came back as "" and the next turn cannot be matched to the call. An empty id no longer overwrites.

How did you test it?

Two new unit tests in TestChatCompletionChunkConversion that feed the recorded chunks through the converter and _convert_streaming_chunks_to_chat_message; both fail on main and pass here. pytest test/components/generators -m "not integration": 323 passed (test_openai_responses.py was left out locally because it needs the flaky plugin). ruff format and ruff check clean on the changed files.

Notes for the reviewer

Release note added. OpenAI's own streams always carry index, so their path is unchanged.

Found by replaying recorded provider streams through the framework with llm-stream-check. Disclosure: I maintain llm-stream-check.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I added unit tests and updated the docstrings.
  • I used a conventional commit type in the PR title (fix:).
  • I added a release note.

I used an AI coding assistant to help write this change. I read every line, and I ran the tests listed above myself.

…omits index

Gemini's OpenAI-compatible endpoint and Mistral stream tool calls without the
tool call index. OpenAIChatGenerator passed index=None into ToolCallDelta,
so a streaming run with tools raised ValueError. Resolve the index from the
previous chunks: a delta with an id not seen yet starts a new tool call, one
without an id continues the latest.

Also keep the first chunk's tool call id when later chunks of the same call
send an empty id (Alibaba Cloud Model Studio), instead of overwriting it
with an empty string.
@Mattbusel
Mattbusel requested a review from a team as a code owner November 20, 2026 19:39
@Mattbusel
Mattbusel requested review from davidsbatista and removed request for a team November 20, 2026 19:39
@vercel

vercel Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

@Mattbusel is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Nov 20, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @Mattbusel, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

Please note that PRs without a signed CLA are automatically closed 48 hours after they were opened.

@HaystackBot
HaystackBot removed the request for review from davidsbatista November 20, 2026 20:51
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Nov 20, 2026
@HaystackBot
HaystackBot marked this pull request as draft November 20, 2026 20:51
@HaystackBot
HaystackBot marked this pull request as ready for review November 21, 2026 00:26
@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Nov 21, 2026
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @Mattbusel! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants