jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Packaging: Add agnostic method to check version of packages

Browse source 
Some packages such as ExllamaV2 and V3 require specific versions for
the latest features. Rather than creating repetitive functions, create
an agnostic function to check the installed package and then report
to the user to upgrade.

This is also sent to requests for loading and unloading, so keep the
error short.

Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
kingbri 2025-05-17 01:04:24 -04:00
parent 084916c04f
commit 17f3dca6fc
3 changed files with 33 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

4
backends/exllamav3/model.py
View file

@ -32,6 +32,7 @@ from common.gen_logging import (
from common.hardware import hardware_supports_flash_attn
from common.health import HealthManager
from common.multimodal import MultimodalEmbeddingWrapper
from common.optional_dependencies import check_package_version
from common.sampling import BaseSamplerRequest
from common.templating import PromptTemplate, find_prompt_template
from common.transformers_utils import HFModel
@ -96,6 +97,9 @@ class ExllamaV3Container(BaseModelContainer):
self = cls()
# Make sure ExllamaV3 is up to date
check_package_version("exllamav3", "0.0.2")
logger.warning(
"ExllamaV3 is currently in an alpha state. "
"Please note that all config options may not work."