tabbyAPI-ollama/common
kingbri d2c6ae2d35 API: Back to async
According to FastAPI docs, if you're using a generic function, running
it in async will make it more performant (which makes sense since
running def functions for routes will automatically run the caller
through a threadpool).

Tested and everything works fine.

Signed-off-by: kingbri <bdashore3@proton.me>
2024-03-04 23:21:40 -05:00
..
args.py Config: Add experimental torch cuda malloc backend 2024-02-14 21:45:56 -05:00
auth.py API: Back to async 2024-03-04 23:21:40 -05:00
config.py Launch: Make exllamav2 requirement more friendly 2024-02-02 23:36:17 -05:00
gen_logging.py Tree: Refactor code organization 2024-01-25 00:15:40 -05:00
generators.py API: Fix issues with concurrent requests and queueing 2024-03-04 23:21:40 -05:00
logger.py Tree: Refactor code organization 2024-01-25 00:15:40 -05:00
sampling.py Model: Add EBNF grammar support 2024-02-24 23:40:11 -05:00
templating.py API: Add template switching and unload endpoints 2024-01-25 00:15:40 -05:00
utils.py API: Add more methods to semaphore 2024-03-04 23:21:40 -05:00