OAI: Add finish to completions

OAI spec requires [DONE] to be sent over SSE to signal that a generation
is completed.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-12-25 11:25:38 -05:00
parent cc3229c109
commit 09ae71aa91

View file

@ -366,6 +366,9 @@ async def generate_completion(request: Request, data: CompletionRequest):
)
yield get_sse_packet(response.model_dump_json())
# Yield a finish response on successful generation
yield get_sse_packet("[DONE]")
except CancelledError:
logger.error("Completion request cancelled by user.")
except Exception as exc: