API + Model: Add new parameters and clean up documentation

The example JSON fields were changed because of the new sampler
default strategy. Fix these by manually changing the values.

Also add support for fasttensors and expose generate_window to
the API. It's recommended to not adjust generate_window as it's
dynamically scaled based on max_seq_len by default.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-01-25 00:11:30 -05:00 committed by Brian Dashore
parent 90fb41a77a
commit fc4570220c
4 changed files with 45 additions and 10 deletions

View file

@ -18,6 +18,11 @@ token_healing:
override: false
force: false
# Commented out because the default is dynamically scaled
#generate_window:
#override: 512
#force: false
# MARK: Temperature
temperature:
override: 1.0