jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1051 commits 1 branch 0 tags 1.4 MiB
Commit graph

1051 commits

This branch
This branch
All branches
Author SHA1 Message Date
kingbri
e60c4ba5bc Config: Fix existing value check 
If a sub-field exists in the model provided to the file generator,
use it. Otherwise always fallback to the default factory. This prevents
any subsequent errors from setting None.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 17:51:40 -04:00
kingbri
c715094cdc Config: Add logging config to migration checks 
These keys were changed as well to include a "log_" prefix like the
CLI arguments.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 12:35:33 -04:00
kingbri
81ae461eb8 Config: Allow existing values to get included in generated file 
Allows for generation from an existing config file. Primarily used
for migration purposes.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 12:19:58 -04:00
TerminalMan
7f03003437 rephrase info message 2024-09-16 14:18:54 +01:00
TerminalMan
564bdcf0a8 add legacy config converter 2024-09-16 14:12:47 +01:00
kingbri
b6dd21f737 Config: Handle default factories in config generation 
Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 00:55:46 -04:00
kingbri
3340c3bf2f Config: Rewrite descriptions 
This makes both config.yml and args more descriptive than before.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 00:55:14 -04:00
kingbri
4c8bb42ec1 Config: Reorder models 
It makes sense for the LLM model groups to be clustered around
each other with the least used groups towards the bottom.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 00:55:14 -04:00
kingbri
8ff9f2c6c0 Config: Rewrite docstrings for models 
Adheres to the old config.yml's descriptions and allows for newlines
in generated YAML.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 00:55:14 -04:00
kingbri
250d76f5c6 Config: Alter YAML generator function 
These changes fix the amount and order of newlines to look pleasing
for the user. However, the changes used in here are kind of hacky
and need a proper fix that can contain the same level of efficiency.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-16 00:55:11 -04:00
TerminalMan
92af656705 improve config generation action 2024-09-15 17:50:37 +01:00
kingbri
f05229bce4 Merge branch 'main' into pydantic-config 2024-09-14 22:22:17 -04:00
kingbri
2a41910931 Model: Remove dev wheel setting checks 
Removes TP and DRY sampler checks since those are in stable.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-14 22:14:43 -04:00
Brian Dashore
2d221832fb
Merge pull request #201 from theroyallab/lmfe_fix 
Fix LMFE
 2024-09-14 22:11:11 -04:00
kingbri
5bfa952671 Actions: Format 
Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-14 22:05:11 -04:00
kingbri
d013729b7d Config: Add aliases for logging config 
Config.yml and args take in two different values.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-14 21:56:16 -04:00
kingbri
6f28cfe905 Logging: Remove preferences global 
This is no longer needed because config is a singleton.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-14 21:49:44 -04:00
kingbri
a09dd802c2 Config: Cleanup and organize functions 
Remove access of private attributes and use safer functions. Also
move generalized functions into utils files.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-14 21:48:39 -04:00
TerminalMan
0903f852db add export openAPI to config 2024-09-15 00:17:36 +01:00
TerminalMan
533e7c9119 remove unnecessary code 2024-09-14 22:49:37 +01:00
turboderp
318c425d84 Bump exllamav2 to 0.2.2 2024-09-14 21:43:26 +02:00
turboderp
c66fe8e947 Grammar: Add custom ExLlamaV2TokenEnforcerFilter class 2024-09-14 21:42:53 +02:00
TerminalMan
dc4946b565 make pydantic do all the validation 2024-09-13 10:21:27 +01:00
kingbri
d5b3fde319 Config: Fix descriptions 
Appending lines also requires a space between each one otherwise
they'll squish together.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-12 22:43:30 -04:00
kingbri
21747bf9e4 Args: Switch to use model_field for everything 
Pydantic provides these helpers. Better to use these instead of
the inspect lib.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-12 22:18:20 -04:00
TerminalMan
6e935c565e remove private attributes in args 2024-09-13 00:37:17 +01:00
TerminalMan
eb5f42c845 add error message for invalid use_as_default 2024-09-12 23:48:24 +01:00
TerminalMan
e11d80b285 fix missing rename 2024-09-12 23:32:41 +01:00
TerminalMan
8b48f00271 fix model names 2024-09-12 17:00:07 +01:00
TerminalMan
05f1c3e293 fix line lengths 2024-09-11 21:43:30 +01:00
TerminalMan
c6f9806ec6 remove unused imports 2024-09-11 18:00:29 +01:00
TerminalMan
0d7459191c fix arg parser for dict types 2024-09-11 16:13:31 +01:00
TerminalMan
e8fcecd56a Merge remote-tracking branch 'upstream/main' into HEAD 2024-09-11 15:57:18 +01:00
Brian Dashore
a2b4e3f21f
Merge pull request #192 from SecretiveShell/prune-docker-size 
debloat docker build
 2024-09-11 00:13:16 -04:00
kingbri
e00eb09ef3 OAI: Add cancellation with inline load 
When the request is cancelled, cancel the load task. In addition,
when checking if a model container exists, also check if the model
is fully loaded.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-11 00:08:55 -04:00
kingbri
b9e5693c1b API + Model: Apply config.yml defaults for all load paths 
There are two ways to load a model:
1. Via the load endpoint
2. Inline with a completion

The defaults were not applying on the inline load, so rewrite to fix
that. However, while doing this, set up a defaults dictionary rather
than comparing it at runtime and remove the pydantic default lambda
on all the model load fields.

This makes the code cleaner and establishes a clear config tree for
loading models.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 23:35:35 -04:00
kingbri
7baef05b49 Transformers Utils: Fix file read 
Use asynchronous JSON reading

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 22:41:39 -04:00
kingbri
62beb2b1c8 Config: Fetch the correct dict for draft_model and lora 
Fixed fetching from the merged config instead of the sub-config

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 21:30:53 -04:00
kingbri
aa832b8627 Tree: Format 
Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 20:57:13 -04:00
kingbri
5e8ff9a004 Tree: Fix classmethod usage 
Instead of self, use cls which passes a type of the class.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 20:52:29 -04:00
kingbri
2c3bc71afa Tree: Switch to asynchronous file handling 
Using aiofiles, there's no longer a possiblity of blocking file operations
that can hang up the event loop. In addition, partially migrate
classes to use asynchronous init instead of the normal python magic method.

The only exception is config, since that's handled in the synchonous
init before the event loop starts.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 16:45:14 -04:00
kingbri
54bfb770af API: Fix template switch endpoint 
Forwards a Path instead of a string and adheres to the new pathfinding
system.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 12:22:07 -04:00
kingbri
810cd40016 Start: Broadcast start_options only on first-time run 
Prevents the save from occurring multiple times for no reason.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-10 12:19:54 -04:00
Brian Dashore
0586fc17cc
Merge pull request #197 from atisharma/issue_196 
Fix tabby_config.py _from_file
 2024-09-10 09:14:44 -04:00
Ati Sharma
a370aeb15f
Fix tabby_config.py _from_file 
Update tabby_config.py to fix issue #196
 2024-09-09 09:19:12 +01:00
Brian Dashore
c11461e22f
Merge pull request #195 from Cohee1207/fix-config-name 
Properly specify config "inline_model_loading" value in the error message
 2024-09-08 22:52:42 -04:00
kingbri
cf97113868 Dependencies: Update Exllamav2 
v0.2.1

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-08 21:12:31 -04:00
Cohee
63476041d1 Properly specify config value in the error message 2024-09-08 22:02:49 +03:00
kingbri
d6ad17097c Templates: Remove whitespace from metadata 
Apparently setting variables also adds extraneous whitespace before
the template itself.

Doing {%- set stop_strings = ["string1"] -%} fixes this issue.

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-08 12:36:36 -04:00
kingbri
776bfd817d Templates: Migrate tool calling templates to folder 
Mirrors the llm-prompt-templates repo

Signed-off-by: kingbri <bdashore3@proton.me>
 2024-09-08 12:28:38 -04:00