jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

OAI: Remove double logging if request is cancelled

Browse source 
Uvicorn can log in both the request disconnect handler and the
CancelledError. However, these sometimes don't work and both
need to be checked. But, don't log twice if one works.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-07-22 21:48:59 -04:00
parent 14dfaf600a
commit d1706fb067
2 changed files with 8 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

5
endpoints/OAI/utils/chat_completion.py
View file

@ -311,8 +311,9 @@ async def stream_generate_chat_completion(
except CancelledError:
# Get out if the request gets disconnected
abort_event.set()
handle_request_disconnect("Chat completion generation cancelled by user.")
if not disconnect_task.done():
abort_event.set()
handle_request_disconnect("Chat completion generation cancelled by user.")
except Exception:
yield get_generator_error(
"Chat completion aborted. Please check the server console."

9
endpoints/OAI/utils/completion.py
View file

@ -162,10 +162,11 @@ async def stream_generate_completion(
except CancelledError:
# Get out if the request gets disconnected
abort_event.set()
handle_request_disconnect(
f"Completion generation {request.state.id} cancelled by user."
)
if not disconnect_task.done():
abort_event.set()
handle_request_disconnect(
f"Completion generation {request.state.id} cancelled by user."
)
except Exception:
yield get_generator_error(
f"Completion {request.state.id} aborted. Please check the server console."