jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

API: Default tool call ID and type

Browse source 
Doing this helps reduce the model's burden of generating the tool
call ID and type (which is always "function"). Follow mistral's spec
for tool call IDs by using a 9 character alphanumeric string.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-07-11 01:11:09 -04:00
parent 5b1db3ad83
commit 707d005aad
3 changed files with 8 additions and 11 deletions
Download patch file Download diff file Expand all files Collapse all files

8
endpoints/OAI/utils/chat_completion.py
View file

@ -70,12 +70,11 @@ def _create_response(
logprob_response = ChatCompletionLogprobs(content=collected_token_probs)
# Initialize finish_reason with a default value or from generation data
finish_reason = generation.get("finish_reason", "stop")
# If a tool call is present, mark the finish reason as such
# Set finish reason
if message.tool_calls:
finish_reason = "tool_calls"
else:
finish_reason = generation.get("finish_reason", "stop")
choice = ChatCompletionRespChoice(
index=index,
@ -152,7 +151,6 @@ def _create_stream_chunk(
choice.finish_reason = "tool_calls"
choices.append(choice)
else:
message = ChatCompletionMessage(
role="assistant", content=unwrap(generation.get("text"), "")