kingbri 660f9b8432 OAI: Fix request cancellation behavior ... Depending on the day of the week, Starlette can work with a CancelledError or using await request.is_disconnected(). Run the same behavior for both cases and allow cancellation. Streaming requests now set an event to cancel the batched job and break out of the generation loop. Signed-off-by: kingbri <bdashore3@proton.me>		2024-05-26 13:00:33 -04:00
..
OAI	OAI: Fix request cancellation behavior	2024-05-26 13:00:33 -04:00
server.py	API: Move OAI to APIRouter	2024-04-06 01:25:31 -04:00