jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Config: Switch to YAML and add load progress

Browse source 
YAML is a more flexible format when it comes to configuration. Commandline
arguments are difficult to remember and configure especially for
an API with complicated commandline names. Rather than using half-baked
textfiles, implement a proper config solution.

Also add a progress bar when loading models in the commandline.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-11-12 00:21:16 -05:00
parent 5d32aa02cd
commit a10c14d357
6 changed files with 38 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

8
config_sample.yml Normal file
View file

@ -0,0 +1,8 @@
model_dir: "D:/models"
model_name: "this_is_a_exl2_model"
max_seq_len: 4096
gpu_split: "auto"
rope_scale: 1.0
rope_alpha: 1.0
no_flash_attention: False
low_mem: False