API: Add min_tokens

Bans the EOS token until the generation reaches a minimum length. This will not prevent the model from otherwise ending the generation early by outputting other stop conditions.
This commit is contained in:
DocShotgun 2024-05-10 12:30:17 -07:00
parent 643b53e347
commit a1df22668b
3 changed files with 33 additions and 1 deletions

View file

@ -11,6 +11,9 @@
max_tokens:
override: 150
force: false
min_tokens:
override: 0
force: false
stop:
override: []
force: false