Skip to content

fix(advanced): don't force tool_choice on Opus/Sonnet 5.5, bump 0.19.10 [PC-5488] - #1206

Open
tudormatei1 wants to merge 2 commits into
mainfrom
fix/advanced-unforced-tool-choice
Open

tudormatei1 wants to merge 2 commits into
mainfrom
fix/advanced-unforced-tool-choice

Conversation

@tudormatei1

@tudormatei1 tudormatei1 commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Advanced agents on Claude Opus 5.5 and Sonnet 5.5 fail on their first model call with a 400: tool_choice: type "tool" and "any" are not supported for this model. Standard agents were fixed in #1121, but advanced agents never read the shouldSkipForcedToolChoice discovery flag. Government regions are moving to the 5.5 Claude models only, so advanced agents there have no working Claude model besides Haiku.

With the new unforced_tool_choice argument on, for those two models the main agent's calls go out on tool_choice="auto" with end_execution bound as a normal tool, and the middleware builds the same structured_response and ToolMessage that langchain's ToolStrategy path builds. Every other model keeps the forced path unchanged. The argument is off by default; the agents runtime turns it on per tenant with the EnableAdvancedAgentUnforcedToolChoice flag (UiPath/uipath-agents-python#880, flag in UiPath/gitops-centralized-cluster#616795). A turn that ends in plain text is caught by the existing nudge retry from #1181.

Changes

  • agent/advanced/tool_choice.py: new ToolChoiceMiddleware. It removes response_format from the request, which is the only input that stops create_agent from forcing any (factory.py#L1423). It adds end_execution first in the tool list and rebuilds the ToolStrategy result from the call.
  • agent/advanced/agent.py: create_advanced_agent and create_advanced_agent_graph take unforced_tool_choice: bool = False. When it's on, the middleware goes last on the main agent, inside _PayloadHandlerMiddleware, so the nudge retry still sees the ToolStrategy. The stall error no longer tells users to check for forced tool choice support.
  • should_skip_forced_tool_choice moved from llm_node.py to no_forced_tool_choice.py so standard and advanced agents share it. Mechanical.
  • Version 0.19.10.

Note

The middleware depends on create_agent routing that skips output-tool calls registered from the build-time ToolStrategy, so the graph must still be built with it. A langchain upgrade that changes this routing would break the 5.5 path. The graph-level tests cover it.

Why not native structured output (ProviderStrategy): it passed most cases, but Sonnet 5.5 sometimes returned its plan text as the final answer without doing any work (5 of 6 runs across two prompts), and the run still reported success. On this approach Sonnet 5.5 did the work in 12 of 12 runs of the same prompts.

Testing

  • 11 tests (9 in tests/agent/advanced/test_tool_choice.py, 2 in test_create_advanced_agent_graph.py), covering the argument on, off and passed through. Each one fails against at least one deliberate break: never unforce, unforce every model, drop the agent's own tools, wrong middleware order, lose the tool cache tag, mishandle two output calls. One test uses a real ChatAnthropic and checks the request body.
  • Live on alpha with a local harness (not linkable): 86 runs. All 77 Opus/Sonnet 5.5 runs gave the expected result, covering nested and typed outputs, file outputs, subagents, code interpreter, tool errors, max iterations and truncation. The 8 agents on other models match main; bedrock-sonnet-4-5-thinking still fails with budget_tokens thinking, as it does on main.
  • Not checked: Fable 5.1, a Bedrock-hosted 5.5 model (alpha has none), and conversational agents with custom output fields. Their output extractor still forces any; that's a separate follow-up.

🤖 Generated with Claude Code

tudormatei1 and others added 2 commits November 11, 2026 17:26
Advanced agents on Claude Opus 5.5 and Sonnet 5.5 failed on the first
model call with a 400: create_agent forces tool_choice "any" whenever a
ToolStrategy is set, and those models reject a forced tool choice.

For models whose discovery details set shouldSkipForcedToolChoice, the
new ToolChoiceMiddleware drops response_format per call, binds
end_execution as a normal tool on "auto" and builds the ToolStrategy
result itself. It sits inside _PayloadHandlerMiddleware so the
missed-output-tool nudge still fires. Other models are unchanged.

Native structured output (ProviderStrategy) was rejected: Sonnet 5.5
sometimes returned its plan as the final answer without doing the work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agents runtime wants to roll this out behind a feature flag, so
create_advanced_agent and create_advanced_agent_graph take an
unforced_tool_choice argument, off by default. Callers that don't pass
it keep today's forced path on every model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tudormatei1
tudormatei1 marked this pull request as ready for review November 12, 2026 10:46
Copilot AI balanced review requested due to automatic review settings November 12, 2026 10:47
@sonarqubecloud

Copy link
Copy Markdown

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unforced tool-choice branch ignores ToolStrategy’s error-handling policy, retrying when handle_errors=False and disregarding configured error messages or callbacks.

1 open issue
What changed in this PR

Adds opt-in support for advanced agents using Claude Opus/Sonnet 5.5 models that reject forced tool choice, extending the discovery-based behavior already available to standard agents.

Changes:

  • Adds middleware that uses automatic tool choice while preserving structured output and the existing plain-text retry.
  • Passes the new option through both advanced-agent builders, shares the discovery helper, and adds regression tests.
  • Bumps the package version to 0.19.10.
File Description
uv.lock Updates the locked package version to 0.19.10.
tests/​agent/​advanced/​test_tool_choice.py Tests unforced requests, structured output, retries, defaults, and cache tagging.
tests/​agent/​advanced/​test_create_advanced_agent_graph.py Tests propagation of the opt-in flag.
src/​uipath_langchain/​agent/​react/​no_forced_tool_choice.py Hosts the shared discovery-flag helper.
src/​uipath_langchain/​agent/​react/​llm_node.py Uses the relocated helper without changing standard-agent behavior.
src/​uipath_langchain/​agent/​advanced/​tool_choice.py Adds unforced tool-choice and structured-response middleware.
src/​uipath_langchain/​agent/​advanced/​agent.py Wires the opt-in middleware and updates the stall error message.
pyproject.toml Sets the release version to 0.19.10.

Review effort: Balanced

return response
if len(calls) > 1:
error = MultipleStructuredOutputsError([c["name"] for c in calls], output)
content = STRUCTURED_OUTPUT_ERROR_TEMPLATE.format(error=str(error))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving this as is. The only caller builds ToolStrategy(output_json_schema) with the default handle_errors=True (graph.py#L504), and for that default this sends the same message langchain does. The module docstring says only the default is mirrored (tool_choice.py#L10-L11). This is a stopgap until langchain lets ToolStrategy skip the forced tool choice; if a caller needs a custom handle_errors before then, I'll add it.

@tudormatei1 tudormatei1 changed the title fix(advanced): don't force tool_choice on Opus/Sonnet 5.5, bump 0.19.10 fix(advanced): don't force tool_choice on Opus/Sonnet 5.5, bump 0.19.10 [PC-5488] Nov 12, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants