bdc5189a4b
Use the same algorithm for estimating and adjusting cache size based on multiples of 256 and above max seq len. Same applies for chunk size. Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| exllamav2 | ||
| exllamav3 | ||
| infinity | ||
| base_model_container.py | ||