Model: Add min_tokens support

In the form of min_new_tokens. Stopping strings take priority.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-05-24 21:04:08 -04:00 committed by Brian Dashore
parent 5f0fb9c4ff
commit c2d3675408

View file

@ -937,6 +937,7 @@ class ExllamaV2Container:
self.generator,
input_ids=input_ids,
max_new_tokens=max_tokens,
min_new_tokens=min_tokens,
gen_settings=gen_settings,
stop_conditions=stop_conditions,
decode_special_tokens=decode_special_tokens,