Model: Move calculate_rope_alpha from backend

Makes more sense to use as a utility function. Also clarify how the
vars are set.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-04-20 18:20:19 -04:00
parent 027ffce05d
commit 8e238fa8f6
3 changed files with 33 additions and 27 deletions

View file

@ -112,8 +112,6 @@ async def load_model_gen(model_path: pathlib.Path, **kwargs):
kwargs = {**config.model_defaults, **kwargs}
kwargs = await apply_inline_overrides(model_path, **kwargs)
print(kwargs)
# Create a new container
new_container = await ExllamaV2Container.create(
model_path.resolve(), False, **kwargs