Model: Warn user if terminating jobs

If skip_wait is true, it's best to let the user know that all jobs
will be forcibly cancelled.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-07-15 11:33:44 -04:00
parent 9dae461142
commit 933404c185

View file

@ -449,6 +449,11 @@ class ExllamaV2Container:
# Immediately abort all jobs if asked
if skip_wait:
logger.warning(
"Immediately terminating all jobs. "
"Clients will have their requests cancelled.\n"
)
# Requires a copy to avoid errors during iteration
jobs_copy = self.generator.jobs.copy()
for job in jobs_copy.values():