jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Entrypoint: Load loras after model

Browse source 
Prevents an error if the model isn't loaded on startup.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-12-27 23:55:02 -05:00
parent ee84d892b8
commit c5bbfd97b2
1 changed files with 5 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

10
main.py
View file

@ -513,11 +513,11 @@ def entrypoint():
else:
loading_bar.next()
# Load loras
lora_config = get_lora_config()
if "loras" in lora_config:
lora_dir = pathlib.Path(unwrap(lora_config.get("lora_dir"), "loras"))
MODEL_CONTAINER.load_loras(lora_dir.resolve(), **lora_config)
# Load loras after loading the model
lora_config = get_lora_config()
if "loras" in lora_config:
lora_dir = pathlib.Path(unwrap(lora_config.get("lora_dir"), "loras"))
MODEL_CONTAINER.load_loras(lora_dir.resolve(), **lora_config)
uvicorn.run(
app,