Model: Remove num_experts_per_token
This shouldn't even be an exposed option since changing it always breaks inference with the model. Let the model's config.json handle it. Signed-off-by: kingbri <8082010+kingbri1@users.noreply.github.com>
This commit is contained in:
parent
698d8339cb
commit
79f9c6e854
6 changed files with 0 additions and 30 deletions
|
|
@ -294,16 +294,6 @@ class ModelConfig(BaseConfigModel):
|
|||
"Enables vision support if the model supports it. (default: False)"
|
||||
),
|
||||
)
|
||||
num_experts_per_token: Optional[int] = Field(
|
||||
None,
|
||||
description=(
|
||||
"Number of experts to use per token.\n"
|
||||
"Fetched from the model's config.json if empty.\n"
|
||||
"NOTE: For MoE models only.\n"
|
||||
"WARNING: Don't set this unless you know what you're doing!"
|
||||
),
|
||||
ge=1,
|
||||
)
|
||||
|
||||
_metadata: Metadata = PrivateAttr(Metadata())
|
||||
model_config = ConfigDict(protected_namespaces=())
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue