jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Config: Add option to force streaming off

Browse source 
Many APIs automatically ask for request streaming without giving
the user the option to turn it off. Therefore, give the user more
freedom by giving a server-side kill switch.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-02-07 21:08:21 -05:00
parent d0027bce32
commit 58590a6c57
3 changed files with 19 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

5
common/args.py
View file

@ -135,3 +135,8 @@ def add_developer_args(parser: argparse.ArgumentParser):
developer_group.add_argument(
"--unsafe-launch", type=str_to_bool, help="Skip Exllamav2 version check"
)
developer_group.add_argument(
"--disable-request-streaming",
type=str_to_bool,
help="Disables API request streaming",
)