jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Samplers: Expose skew sampling

Browse source 
Skew is an extra unused sampler in ExllamaV2. Add it in for coverage.

Signed-off-by: kingbri <bdashore3@proton.me>
This commit is contained in:
kingbri 2024-05-12 01:41:01 -04:00
parent 6f4012d20d
commit c8ec742be9
3 changed files with 10 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
common/sampling.py
View file

@ -75,6 +75,11 @@ class BaseSamplerRequest(BaseModel):
examples=[1.0],
)
skew: Optional[float] = Field(
default_factory=lambda: get_default_sampler_value("skew", 0.0),
examples=[0.0],
)
frequency_penalty: Optional[float] = Field(
default_factory=lambda: get_default_sampler_value("frequency_penalty", 0.0)
)
@ -295,6 +300,7 @@ class BaseSamplerRequest(BaseModel):
"typical": self.typical,
"min_p": self.min_p,
"tfs": self.tfs,
"skew": self.skew,
"frequency_penalty": self.frequency_penalty,
"presence_penalty": self.presence_penalty,
"repetition_penalty": self.repetition_penalty,