jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README

Browse source 
Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-11-17 22:02:21 -05:00
parent d627d14385
commit 6f2078cbe4
1 changed files with 7 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

8
README.md
View file

@ -18,7 +18,7 @@ To get started, make sure you have the following installed on your system:
- Python 3.x (preferably 3.11) with pip
- CUDA 12.1 or 11.8
- CUDA 12.1 or 11.8 (or ROCm 5.6)
NOTE: For Flash Attention 2 to work on Windows, CUDA 12.1 **must** be installed!
@ -92,6 +92,12 @@ All routes require an API key except for the following which require an **admin*
- `/v1/model/unload`
## Common Issues
- AMD cards will error out with flash attention installed, even if the config option is set to False. Run `pip uninstall flash_attn` to remove the wheel from your system.
- See [#5](https://github.com/theroyallab/tabbyAPI/issues/5)
## Contributing
If you have issues with the project: