Dependencies: Change handling of exllamav2 checks
ExllamaV2 should check for solely exllamav2, otherwise errors don't make sense. Migrate the combined "exl2" computed property to "inference" since those are the required dependencies for minimal inference. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
parent
5380b3fe5e
commit
e0ffa90865
3 changed files with 3 additions and 3 deletions
|
|
@ -15,7 +15,7 @@ from common.networking import handle_request_error
|
|||
from common.tabby_config import config
|
||||
from common.optional_dependencies import dependencies
|
||||
|
||||
if dependencies.exl2:
|
||||
if dependencies.exllamav2:
|
||||
from backends.exllamav2.model import ExllamaV2Container
|
||||
|
||||
# Global model container
|
||||
|
|
|
|||
|
|
@ -25,7 +25,7 @@ class DependenciesModel(BaseModel):
|
|||
|
||||
@computed_field
|
||||
@property
|
||||
def exl2(self) -> bool:
|
||||
def inference(self) -> bool:
|
||||
return self.torch and self.exllamav2 and self.flash_attn
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue