Config: Add experimental torch cuda malloc backend
This option saves some VRAM, but does have the chance to error out. Add this in the experimental config section. Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
parent
664e2c417e
commit
949248fb94
3 changed files with 16 additions and 1 deletions
|
|
@ -140,3 +140,8 @@ def add_developer_args(parser: argparse.ArgumentParser):
|
|||
type=str_to_bool,
|
||||
help="Disables API request streaming",
|
||||
)
|
||||
developer_group.add_argument(
|
||||
"--cuda-malloc-backend",
|
||||
type=str_to_bool,
|
||||
help="Disables API request streaming",
|
||||
)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue