Repository navigation
Conversation
…omits index Gemini's OpenAI-compatible endpoint and Mistral stream tool calls without the tool call index. OpenAIChatGenerator passed index=None into ToolCallDelta, so a streaming run with tools raised ValueError. Resolve the index from the previous chunks: a delta with an id not seen yet starts a new tool call, one without an id continues the latest. Also keep the first chunk's tool call id when later chunks of the same call send an empty id (Alibaba Cloud Model Studio), instead of overwriting it with an empty string.
|
@Mattbusel is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @Mattbusel, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the Please note that PRs without a signed CLA are automatically closed 48 hours after they were opened. |
|
Thanks for signing the CLA, @Mattbusel! 🎉 This PR is now ready for review again and the reviewer has been re-assigned. |
Related Issues
None open. Two crashes and a lost id in
OpenAIChatGeneratorstreaming with tools against OpenAI-compatible servers.Proposed Changes
_convert_chat_completion_chunk_to_streaming_chunkpassesindex=tool_call.indexstraight intoToolCallDelta(haystack/components/generators/chat/openai.py:824). Gemini's OpenAI-compatible endpoint and Mistral stream tool calls with anidand noindex, soStreamingChunkraisesValueError: If tool_call, tool_call_result or reasoning is set, index must also be set.(haystack/dataclasses/streaming_chunk.py:153) and the run fails. This happens on this recorded Gemini response (two parallel calls, noindex). The index is now resolved from the previous chunks: a delta with an id not seen yet starts a new tool call, one without an id continues the latest._convert_streaming_chunks_to_chat_messageoverwrites the saved id whenever a chunk hasid is not None(haystack/components/generators/utils.py:147). Alibaba Cloud Model Studio sends"id": ""on every chunk after the first (recorded qwen3-max stream), so the reply'sToolCall.idcame back as""and the next turn cannot be matched to the call. An empty id no longer overwrites.How did you test it?
Two new unit tests in
TestChatCompletionChunkConversionthat feed the recorded chunks through the converter and_convert_streaming_chunks_to_chat_message; both fail onmainand pass here.pytest test/components/generators -m "not integration": 323 passed (test_openai_responses.pywas left out locally because it needs theflakyplugin).ruff formatandruff checkclean on the changed files.Notes for the reviewer
Release note added. OpenAI's own streams always carry
index, so their path is unchanged.Found by replaying recorded provider streams through the framework with llm-stream-check. Disclosure: I maintain llm-stream-check.
Checklist
fix:).I used an AI coding assistant to help write this change. I read every line, and I ran the tests listed above myself.