Add SillyTavern instructions

Temporary until proper support is added in.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2023-11-16 17:33:23 -05:00
parent b20e71dcd4
commit 2cf93c092b

28
SillyTavern.md Normal file
View file

@ -0,0 +1,28 @@
# Running on SillyTavern
To run TabbyAPI on SillyTavern, a few more configuration changes are required on SillyTavern's side.
Make sure to get your TabbyAPI **api key** from `api_config.yml`. The admin key is not useable on SillyTavern.
Inside SillyTavern's `config.conf`, add the following under `const requestOverrides`:
```json
{
hosts: ["127.0.0.1:5000"],
headers: {
Authorization: "Bearer <Your API key>"
}
}
```
This configuration assumes you're using the default config.yml. If you have customized it, make sure to replace `127.0.0.1:5000` with whatever you set the IP and port to in your config.
What this does is intercepts all your oobabooga requests and appends this API key on top of them for security purposes.
Now, inside SillyTavern, just connect to your TabbyAPI URL using oobabooga's text completion API option!
The fetching of models isn't accurate since ooba uses the top-most model in tge model directory (I am not sure why this is the case) to decide which model is loaded. Please make sure to **load a model** beforehand!
Completions work without an issue, just know that all samplers in oobabooga's slider panel won't work in TabbyAPI yet.
Hopefully, Cohee and Ross (or myself) will add full support for TabbyAPI in the not too distant future.