Skip to content

fix: Handle tools without parameters in OpenAI frontend tool_choice - #9056

Open
paolo-perrone wants to merge 2 commits into
triton-inference-server:mainfrom
paolo-perrone:fix/openai-tool-without-parameters
Open

paolo-perrone wants to merge 2 commits into
triton-inference-server:mainfrom
paolo-perrone:fix/openai-tool-without-parameters

Conversation

@paolo-perrone

Copy link
Copy Markdown

What does the PR do?

If a chat request forces a tool that has no "parameters" field, the OpenAI frontend returns a 500. OpenAI lets you leave "parameters" out for a function that takes no arguments, and our schema allows it too (FunctionObject.parameters is Optional), but _get_guided_json_from_tool calls tool.parameters.model_dump_json() without checking for None. Both the vLLM and the TRT-LLM request converters call it, so this hits every backend.

Repro: tools=[{"type": "function", "function": {"name": "get_current_time"}}] with tool_choice={"type": "function", "function": {"name": "get_current_time"}} (or tool_choice="required"). The response is a 500 with 'NoneType' object has no attribute 'model_dump_json'.

The fix returns an empty object schema, {"type": "object", "properties": {}}, when parameters is missing, so the forced call comes back with {} as its arguments. That's the same thing vLLM does for "required" when a tool has no parameters.

Checklist

  • I have read the Contribution guidelines and signed the Contributor License Agreement (CLA is being sent to triton-cla@nvidia.com)
  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Added test plan and verified test passes.
  • I ran pre-commit locally
  • Verified copyright is correct on all changed files.

Commit Type:

  • fix

Where should the reviewer start?

python/openai/openai_frontend/engine/utils/triton.py, _get_guided_json_from_tool.

Test plan:

Added test_guided_json_for_tool_without_parameters to tests/test_tool_calling.py. It covers a named tool_choice and "required" with a tool that has no parameters. It fails on main with the AttributeError and passes with the fix. Tools that do have parameters get exactly the same schema as before.

Caveats:

Another option is to return None and skip guided decoding for these tools, which is what vLLM does for a named tool_choice. I went with the empty object schema so the arguments always come back as valid JSON. Happy to switch if you'd rather.

pre-commit's flake8 flags three existing lines in the same file (!= True twice and an unused except ... as e), which this PR doesn't touch. I left them alone to keep this to one change.

@greptile-apps

greptile-apps Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous finding is fixed and no new actionable issues remain.

Summary

Fixes forced tool calls when a function omits parameters.

  • Forced calls to tools without parameters now use an empty-object schema.

Reviews (2) · Last reviewed commit: "fix: Forbid extra keys in the no-paramet..." · Reviewed by Greptile

@yinggeh
yinggeh self-requested a review November 4, 2026 16:09
@yinggeh yinggeh assigned paolo-perrone and unassigned yinggeh Nov 4, 2026
@yinggeh yinggeh added the bug Something isn't working label Nov 4, 2026
@yinggeh

yinggeh commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

LGTM. Please respond to the review comment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Development

Successfully merging this pull request may close these issues.

2 participants