jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Model: Fix usage stats in non-streaming gens

Browse source 
The wrong key was being returned from the model to the API.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-05-12 22:44:50 -04:00
parent 213430a122
commit e4bb709305
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
backends/exllamav2/model.py
View file

@ -656,7 +656,7 @@ class ExllamaV2Container:
joined_generation["prompt_tokens"] = unwrap(
generations[-1].get("prompt_tokens"), 0
)
joined_generation["generation_tokens"] = unwrap(
joined_generation["generated_tokens"] = unwrap(
generations[-1].get("generated_tokens"), 0
)