Model: Add EBNF grammar support

Using the Outlines library, add support to supply EBNF strings and
pass them to the library for parsing.

From there, a wrapper is created and a filter is passed to generation.

Replace with an in-house solution at some point that's more flexible.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-02-23 01:36:10 -05:00 committed by Brian Dashore
parent 57b3d69949
commit f6d749c771
3 changed files with 119 additions and 16 deletions

View file

@ -122,6 +122,10 @@ class BaseSamplerRequest(BaseModel):
default_factory=lambda: get_default_sampler_value("json_schema"),
)
grammar_string: Optional[str] = Field(
default_factory=lambda: get_default_sampler_value("grammar_string"),
)
# Aliased variables
typical: Optional[float] = Field(
default_factory=lambda: get_default_sampler_value("typical", 1.0),
@ -266,6 +270,7 @@ class BaseSamplerRequest(BaseModel):
"cfg_scale": self.cfg_scale,
"negative_prompt": self.negative_prompt,
"json_schema": self.json_schema,
"grammar_string": self.grammar_string,
}
return {**gen_params, **kwargs}