jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Tree: Unify sampler parameters and add override support

Browse source 
Unify API sampler params into a superclass which should make them
easier to manage and inherit generic functions from.

Not all frontends expose all sampling parameters due to connections
with OAI (that handles sampling themselves with the exception of
a few sliders).

Add the ability for the user to customize fallback parameters from
server-side.

In addition, parameters can be forced to a certain value server-side
in case the repo automatically sets other sampler values in the
background that the user doesn't want.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-01-21 23:34:44 -05:00 • committed by Brian Dashore
parent 78f920eeda
commit 6c30f24c83
7 changed files with 337 additions and 86 deletions
Download patch file Download diff file Expand all files Collapse all files

8
config_sample.yml
View file

@ -27,6 +27,14 @@ logging:
# Enable generation parameter logging (default: False)
generation_params: False
# Options for sampling
sampling:
# Override preset name. Find this in the sampler-overrides folder (default: None)
# This overrides default fallbacks for sampler values that are passed to the API
# Server-side overrides are NOT needed by default
# WARNING: Using this can result in a generation speed penalty
#override_preset:
# Options for model overrides and loading
model:
# Overrides the directory to look for models (default: models)