jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Model: Fix no flash attention

Browse source 
Was being called wrong from config.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-12-17 23:31:58 -05:00
parent ad8807a830
commit 95fd0f075e
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
model.py
View file

@ -94,7 +94,7 @@ class ModelContainer:
)
# Turn off flash attention?
self.config.no_flash_attn = unwrap(kwargs.get("no_flash_attn"), False)
self.config.no_flash_attn = unwrap(kwargs.get("no_flash_attention"), False)
# low_mem is currently broken in exllamav2. Don't use it until it's fixed.
"""