jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Model: Remove override_base_seq_len

Browse source
This commit is contained in:
DocShotgun 2024-10-30 10:03:08 +08:00
parent 7d18d2e2ca
commit 603760cecb
5 changed files with 0 additions and 33 deletions
Download patch file Download diff file Expand all files Collapse all files

5
config_sample.yml
View file

@ -69,11 +69,6 @@ model:
# Fetched from the model's base sequence length in config.json by default.
max_seq_len:
# Overrides base model context length (default: Empty).
# WARNING: Don't set this unless you know what you're doing!
# Again, do NOT use this for configuring context length, use max_seq_len above ^
override_base_seq_len:
# Load model with tensor parallelism.
# Falls back to autosplit if GPU split isn't provided.
# This ignores the gpu_split_auto value.