# Overview

Clients can send sampler parameters that users are unaware of, or even not send any parameters at all!

Sampler overrides is tabbyAPI's flexible proxy for adding or forcing sampler values before generation. 
# Setting up

All supported samplers are located in `sampler_overrides/sample_preset.yml`. 

> [!NOTE]
> Sampler overrides can also be switched at runtime via the `/v1/sampling/override` endpoint set. Please read the [API docs](https://theroyallab.github.io/tabbyAPI/#operation/list_sampler_overrides_v1_sampling_override_list_get) for more information.

Steps:
1. Create a new YML in the `sampler_overrides` folder or duplicate `sample_preset.yml`
2. Adjust the samplers you want to override using the examples below
3. Save and rename the file (if you haven't already)
4. Open `config.yml` and inside the `sampling` block, set the `sampler_override` key to the preset name from step 3
# Examples

Let's say a client doesn't send a value for `top_p`:

```yml
top_p:
	override: 0.7
	force: False
```

This override will change the default fallback of top_p from a neutral value of 1.0 to 0.7.

Now, let's say a client forces a value that you don't like (ex. `top-k: 30`):

```yml
top_k:
	override: 0
	force: True
```

This override forces top_k to be 0 no matter what the client sends.

Finally, let's say a client sends a list, but you want to add something else:

```yml
stop:
	override: ["a"]
	force: False
	additive: True
```

Now this override appends the given stop string to whatever the client sends.