jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

API: Add chat_template_kwargs alias for template_vars

Browse source 
This key is used in VLLM and SGLang.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-05-12 15:33:34 -04:00
parent c9dc0b2aa4
commit 7900b72848
2 changed files with 9 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

3
docs/04.-Chat-Completions.md
View file

@ -23,6 +23,9 @@ A chat completions request to TabbyAPI also supports custom template variables i
}
```
> [!NOTE]
> To preserve compatibility with other standards, `chat_template_kwargs` can be used instead of `template_vars`
Now let's pass the custom var in the following template:
```jinja2
I'm going to say {{ test_var }}

8
endpoints/OAI/types/chat_completion.py
View file

@ -1,4 +1,4 @@
from pydantic import BaseModel, Field, field_validator
from pydantic import AliasChoices, BaseModel, Field, field_validator
from pydantic.json_schema import SkipJsonSchema
from time import time
from typing import Literal, Union, List, Optional, Dict
@ -65,7 +65,11 @@ class ChatCompletionRequest(CommonCompletionRequest):
messages: List[ChatCompletionMessage] = Field(default_factory=list)
prompt_template: Optional[str] = None
add_generation_prompt: Optional[bool] = True
template_vars: Optional[dict] = {}
template_vars: Optional[dict] = Field(
default={},
validation_alias=AliasChoices("template_vars", "chat_template_kwargs"),
description="Aliases: chat_template_kwargs",
)
response_prefix: Optional[str] = None
model: Optional[str] = None