jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Model: Fix draft model loading

Browse source 
Use draft_config to find the path instead of kwargs.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-11-19 02:04:02 -05:00
parent 698b0b1976
commit 4cddd0400c
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
model.py
View file

@ -95,7 +95,7 @@ class ModelContainer:
if self.draft_enabled:
self.draft_config = ExLlamaV2Config()
draft_model_path = pathlib.Path(kwargs.get("draft_model_dir") or "models")
draft_model_path = pathlib.Path(draft_config.get("draft_model_dir") or "models")
draft_model_path = draft_model_path / draft_model_name
self.draft_config.model_dir = str(draft_model_path.resolve())