API + Model: Add support for JSON schema constraints

Add the ability to constrain the return value of a model to be JSON.
Built using the JSON schema standard to define the properties of what
the model should return.

This feature should be more accurate than using GBNF/EBNF to yield
the same results due to the use of lmformatenforcer.

GBNF/EBNF will be added in a different commit/branch.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-02-22 01:36:46 -05:00 committed by Brian Dashore
parent ccd41d720d
commit 57b3d69949
3 changed files with 77 additions and 0 deletions

View file

@ -118,6 +118,10 @@ class BaseSamplerRequest(BaseModel):
default_factory=lambda: get_default_sampler_value("negative_prompt")
)
json_schema: Optional[object] = Field(
default_factory=lambda: get_default_sampler_value("json_schema"),
)
# Aliased variables
typical: Optional[float] = Field(
default_factory=lambda: get_default_sampler_value("typical", 1.0),
@ -261,6 +265,7 @@ class BaseSamplerRequest(BaseModel):
"mirostat_eta": self.mirostat_eta,
"cfg_scale": self.cfg_scale,
"negative_prompt": self.negative_prompt,
"json_schema": self.json_schema,
}
return {**gen_params, **kwargs}