DocShotgun 55d979b7a5 Update dependencies, support Python 3.12, update for exl2 0.1.5 (#134) ... * Dependencies: Add wheels for Python 3.12 * Model: Switch fp8 cache to Q8 cache * Model: Add ability to set draft model cache mode * Dependencies: Bump exllamav2 to 0.1.5 * Model: Support Q6 cache * Config: Add Q6 cache and draft_cache_mode to config sample		2024-06-09 17:27:39 +02:00
..
OAI	Update dependencies, support Python 3.12, update for exl2 0.1.5 (#134)	2024-06-09 17:27:39 +02:00
server.py	API: Move OAI to APIRouter	2024-04-06 01:25:31 -04:00