jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

API: Fix banned_tokens string when empty

Browse source 
The string should not be parsed and any non-string elements should
be removed as well.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-04-28 12:46:28 -04:00
parent 72dff0b6d5
commit 6114bfd221
1 changed files with 6 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

9
common/sampling.py
View file

@ -249,11 +249,14 @@ class BaseSamplerRequest(BaseModel):
self.validate_params()
# Convert stop to an array of strings
if isinstance(self.stop, str):
if self.stop and isinstance(self.stop, str):
self.stop = [self.stop]
if isinstance(self.banned_tokens, str):
self.banned_tokens = list(map(int, self.banned_tokens.split(",")))
# Convert string banned tokens to an integer list
if self.banned_tokens and isinstance(self.banned_tokens, str):
self.banned_tokens = [
int(x) for x in self.banned_tokens.split(",") if x.isdigit()
]
gen_params = {
"max_tokens": self.max_tokens,