API: Fix blocking iterator execution

Run these iterators on the background thread. On startup, the API
spawns a background thread as needed to run sync code on without blocking
the event loop.

Use asyncio's run_thread function since it allows for errors to be
propegated.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-03-16 22:31:50 -04:00 committed by Brian Dashore
parent 7fded4f183
commit 2755fd1af0
4 changed files with 56 additions and 24 deletions

View file

@ -72,7 +72,7 @@ async def load_model_gen(model_path: pathlib.Path, **kwargs):
async def load_model(model_path: pathlib.Path, **kwargs):
async for _, _, _ in load_model_gen(model_path, **kwargs):
async for _ in load_model_gen(model_path, **kwargs):
pass