jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Exl3: Simplify add_bos_token handling

Browse source 
Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-05-02 21:44:36 -04:00
parent d8a8ccfc2a
commit 15cc480cb0
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
backends/exllamav3/model.py
View file

@ -728,8 +728,8 @@ class ExllamaV3Container(BaseModelContainer):
prompts = [prompt]
stop_conditions = params.stop
add_bos_token = coalesce(
params.add_bos_token, self.tokenizer_config.add_bos_token, True
add_bos_token = unwrap(
params.add_bos_token, self.tokenizer_config.add_bos_token
)
# Fetch EOS tokens from generation_config if they exist