jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Dependencies: Update torch, exllamav2, and flash-attn

Browse source 
Torch - 2.6.0
ExllamaV2 - 0.2.8
Flash-attn - 2.7.4.post1

Cuda wheels are now 12.4 instead of 12.1, feature names need to be
migrated over.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-02-09 01:25:47 -05:00
parent dcbf2de9e5
commit 0dcbb7a722
3 changed files with 39 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

2
backends/exllamav2/utils.py
View file

@ -52,8 +52,6 @@ def supports_paged_attn():
"using the following command:\n\n"
"For CUDA 12.1:\n"
"pip install --upgrade .[cu121]\n\n"
"For CUDA 11.8:\n"
"pip install --upgrade .[cu118]\n\n"
"NOTE: Windows users must use CUDA 12.x to use flash-attn."
)