API: Patch kobold generation call

Calling the model requires different args now.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-05-05 22:11:21 -04:00
parent b683545d0e
commit bc0a84241a

View file

@ -53,8 +53,12 @@ async def _stream_collector(data: GenerateRequest, request: Request):
logger.info(f"Received Kobold generation request {data.genkey}")
generator = model.container.stream_generate(
request_id=data.genkey, abort_event=abort_event, **data.model_dump()
request_id=data.genkey,
prompt=data.prompt,
params=data,
abort_event=abort_event,
)
async for generation in generator:
if disconnect_task.done():
abort_event.set()