Repository navigation
Python: fix(bedrock): report the requested model on non-streaming responses - #9399
Somay kousis (Somay-kousis) wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
The focused, backward-compatible fix covers both reported failure cases with no unresolved findings.
0 open findings
What changed in this PR
Fixes #9398 by making non-streaming Bedrock responses report the requested model, correcting model attribution in telemetry.
Changes:
- Passes the resolved request model into response processing while preserving existing callers.
- Adds regression coverage for per-call overrides with and without a client default model.
| File | Description |
|---|---|
| python/packages/bedrock/tests/test_bedrock_client.py | Tests model reporting using a realistic Converse response. |
| python/packages/bedrock/agent_framework_bedrock/_chat_client.py | Uses the requested model before falling back to the client default. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
|
@microsoft-github-policy-service agree |
|
Eduard van Valkenburg (@eavanvalkenburg) The red Python Tests run is one failure outside this change: Could you re-run the failed jobs? |
|
Eduard van Valkenburg (@eavanvalkenburg) could you re-run the Python Tests here when you get a chance? The only failure was the foundry_hosting cancellation timeout on Windows 3.14, which this bedrock change doesn't touch. Happy to rebase on main first if you'd rather. |
Bicheng (Kenneth) (BichengWang)
left a comment
There was a problem hiding this comment.
Reviewed 1610dff922e65fedbb4eab39f69a96720d73898c. I found no correctness blocker in the model-reporting change. Passing the resolved request model into the non-streaming response conversion matches the streaming path, and the new keyword-only argument preserves existing calls to _process_converse_response(response) and (response, options).
Offline validation on macOS arm64 / CPython 3.14.7:
- All 82 Bedrock unit tests passed; one integration test was deselected. The isolated invocation emitted an existing unregistered
flakymarker warning. - Both added regression cases fail with the parent implementation: the response reports the client default or
Noneinstead of the requested override. Both pass at this head. - An additional 10-case harness covered default models, overrides with and without a client default, and
None/empty override fallback, in both streaming modes. It checked the request model, response model, text, usage, andgen_ai.response.modelattribute extraction. Three direct-call compatibility cases also passed. - Ruff 0.17.0 lint and format checks passed across the Bedrock source and tests. Cached botocore's Converse response shape confirms that the service does not echo
modelId.
A useful follow-up test would retain one end-to-end streaming/non-streaming model-parity case or a telemetry attribute assertion; those paths worked in the additional harness but are not pinned by the new committed cases.
I checked existing reviews and inline feedback before reviewing. I did not run live AWS requests, the full workspace suite, type checks, or other Python versions. Maintainer agreement on #9398 remains a separate issue-first prerequisite.
1610dff to
c9af528
Compare
|
Bicheng (Kenneth) (@BichengWang) thanks for the review. Added the parity case in c9af528: one stub that answers both Converse and ConverseStream without a Breaking either path fails only its own case:
Also rebased on main. 92 Bedrock unit tests pass, ruff 0.17.0 lint and format clean. |
Motivation & Context
BedrockChatClientsends a per-calloptions["model"]to Converse, but the non-streamingChatResponse.modelis the client's default model, orNoneif the client has no default. Converse does not return amodelId, so_process_converse_responsealways falls back toself.model. Telemetry readsgen_ai.response.modelfrom that value, so traces name the wrong model too. Streaming already reports the request's model.Description & Review Guide
request["modelId"]into_process_converse_responseas a new keyword-onlymodelargument, and uses it beforeself.model. This matches what the streaming path already does withrequest["modelId"]. Added a test with a stub that returns the real Converse response shape (nomodelId), for a client with and without a default model.response.modelon non-streaming Bedrock responses now names the model the request went to. No public API change. Existing callers of_process_converse_response(response)and(response, options)keep working.response.get("modelId")/output.get("modelId")lookups. Converse never returns them, but I left them in to keep the change small.Related Issue
Fixes #9398
I think this is an obvious fix: the streaming path already reports
request["modelId"], and #9139 did the same for Gemini. Happy to wait forready-for-implementationif you'd rather triage #9398 first.AI Assistance
AI assistance details: AI tools helped find the bug and write the code, the test and this description. I reviewed all of it and ran the repro, the bedrock tests, and the lint and typing checks myself.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.