jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Config: Move sampling higher in the list

Browse source 
This has become a bigger priority with addition of the safe_defaults
noob proofing.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-08-18 22:55:03 -04:00
parent 6fb0c2cdbd
commit 067d63773e
2 changed files with 26 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

34
common/config_models.py
View file

@ -347,6 +347,23 @@ class DraftModelConfig(BaseConfigModel):
)
class SamplingConfig(BaseConfigModel):
"""Options for Sampling"""
override_preset: Optional[str] = Field(
None,
description=(
"Select a sampler override preset (default: None).\n"
"Find this in the sampler-overrides folder.\n"
"This overrides default fallbacks for sampler values "
"that are passed to the API.\n"
"NOTE: safe_defaults preset provides a fallback for frontends "
"that do not pass sampling params.\n"
"Remove it if not necessary."
),
)
class LoraInstanceModel(BaseConfigModel):
"""Model representing an instance of a Lora."""
@ -400,23 +417,6 @@ class EmbeddingsConfig(BaseConfigModel):
)
class SamplingConfig(BaseConfigModel):
"""Options for Sampling"""
override_preset: Optional[str] = Field(
None,
description=(
"Select a sampler override preset (default: None).\n"
"Find this in the sampler-overrides folder.\n"
"This overrides default fallbacks for sampler values "
"that are passed to the API.\n"
"NOTE: safe_defaults preset provides a fallback for frontends "
"that do not pass sampling params.\n"
"Remove it if not necessary."
),
)
class DeveloperConfig(BaseConfigModel):
"""Options for development and experimentation"""

18
config_sample.yml
View file

@ -179,6 +179,15 @@ draft_model:
# If this isn't filled in, the draft model is autosplit.
draft_gpu_split: []
# Options for Sampling
sampling:
# Select a sampler override preset (default: None).
# Find this in the sampler-overrides folder.
# This overrides default fallbacks for sampler values that are passed to the API.
# NOTE: safe_defaults is noob friendly and provides fallbacks for frontends that don't send sampling parameters.
# Remove this for any advanced usage.
override_preset: safe_defaults
# Options for Loras
lora:
# Directory to look for LoRAs (default: loras).
@ -206,15 +215,6 @@ embeddings:
# An initial embedding model to load on the infinity backend.
embedding_model_name:
# Options for Sampling
sampling:
# Select a sampler override preset (default: None).
# Find this in the sampler-overrides folder.
# This overrides default fallbacks for sampler values that are passed to the API.
# NOTE: safe_defaults preset provides a fallback for frontends that do not pass sampling params.
# Remove it if not necessary.
override_preset: safe_defaults
# Options for development and experimentation
developer:
# Skip Exllamav2 version check (default: False).