jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Sampling: Make add_bos_token override concise

Browse source 
Also set the default to None so text completions follows the same
pattern.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-05-10 19:07:35 -04:00
parent 656af41b5d
commit 6379081dd8
3 changed files with 3 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

2
endpoints/OAI/types/chat_completion.py
View file

@ -84,7 +84,7 @@ class ChatCompletionRequest(CommonCompletionRequest):
# Chat completions requests do not have a BOS token preference. Backend
# respects the tokenization config for the individual model.
add_bos_token: Optional[bool] = Field(default=None)
add_bos_token: Optional[bool] = None
@field_validator("add_bos_token", mode="after")
def force_bos_token(cls, v):