Model: Add vision loading support

Adds the ability to load vision parts of text + image models. Requires
an explicit flag in config because there isn't a way to automatically
determine whether the vision tower should be used.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-11-11 12:04:40 -05:00
parent cc2516790d
commit 69ac0eb8aa
5 changed files with 42 additions and 5 deletions

View file

@ -124,6 +124,9 @@ model:
# NOTE: Only works with chat completion message lists!
prompt_template:
# Enables vision support if the model supports it. (default: False)
vision: false
# Number of experts to use per token.
# Fetched from the model's config.json if empty.
# NOTE: For MoE models only.