25 lines
1.2 KiB
Text
25 lines
1.2 KiB
Text
# Torch
|
|
--extra-index-url https://download.pytorch.org/whl/cu118
|
|
torch
|
|
|
|
# Exllamav2
|
|
|
|
# Windows
|
|
https://github.com/turboderp/exllamav2/releases/download/v0.0.11/exllamav2-0.0.11+cu118-cp311-cp311-win_amd64.whl; platform_system == "Windows" and python_version == "3.11"
|
|
https://github.com/turboderp/exllamav2/releases/download/v0.0.11/exllamav2-0.0.11+cu118-cp310-cp310-win_amd64.whl; platform_system == "Windows" and python_version == "3.10"
|
|
|
|
# Linux
|
|
https://github.com/turboderp/exllamav2/releases/download/v0.0.11/exllamav2-0.0.11+cu118-cp311-cp311-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.11"
|
|
https://github.com/turboderp/exllamav2/releases/download/v0.0.11/exllamav2-0.0.11+cu118-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.10"
|
|
|
|
# Pip dependencies
|
|
fastapi
|
|
pydantic >= 2.0.0
|
|
PyYAML
|
|
progress
|
|
uvicorn
|
|
jinja2 >= 3.0.0
|
|
colorlog
|
|
|
|
# Linux FA2 from https://github.com/Dao-AILab/flash-attention/releases
|
|
https://github.com/Dao-AILab/flash-attention/releases/download/v2.4.1/flash_attn-2.4.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl; platform_system == "Linux" and platform_machine == "x86_64" and python_version == "3.10"
|