f070587e9f
Jobs should be started and immediately cleaned up when calling the generation stream. Expose a stream_generate function and append this to the base class since it's more idiomatic than generate_gen. The exl2 container's generate_gen function is now internal. Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| exllamav2 | ||
| infinity | ||
| base_model_container.py | ||