jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Templates: Alter chatml_with_headers to fit huggingface spec

Browse source 
The previous template was compatible with Jinja2 in Python, but it
was not cross-platform compatible according to HF's standards.

Signed-off-by: kingbri <8082010+bdashore3@users.noreply.github.com>
This commit is contained in:
kingbri 2024-12-30 14:00:44 -05:00
parent 709493837b
commit 245bd5c008
1 changed files with 15 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

33
templates/tool_calls/chatml_with_headers.jinja
View file

@ -6,7 +6,7 @@
{%- set start_header = "<|start_header_id|>" -%} {%- set start_header = "<|start_header_id|>" -%}
{%- set end_header = "<|end_header_id|>\n" -%} {%- set end_header = "<|end_header_id|>\n" -%}
{%- set example_tool_call -%}[ {%- set example_tool_call = '[
{ {
"id": "tool_id_1342", "id": "tool_id_1342",
"function": { "function": {
@ -23,29 +23,26 @@
}, },
"type": "function" "type": "function"
} }
] ]' -%}
{%- endset -%}
{%- set inital_system_prompt -%}You are an assistant that has access to the following set of tools, to call a tool: {%- set inital_system_prompt = 'You are an assistant that has access to the following set of tools, to call a tool:
1. Prefix calls with '{{ tool_start }}' and end calls with '{{ tool_end }}' 1. Prefix calls with ' + tool_start + ' and end calls with ' + tool_end + '
2. Ensure you use the correct type for arguments. For example, if the argument is a string, ensure it is enclosed in quotes, otherwise, it should not be. 2. Ensure you use the correct type for arguments. For example, if the argument is a string, ensure it is enclosed in quotes, otherwise, it should not be.
3. Generate all calls using the following json tool call format. Here is a multi tool call example: 3. Generate all calls using the following json tool call format. Here is a multi tool call example:
{{ tool_start }}{{ example_tool_call }}{{ tool_end }} ' + tool_start + example_tool_call + tool_end + '
Here are the tools available for you to call: Here are the tools available for you to call:
{{ tools_json }} ' + tools_json -%}
{%- endset -%}
{%- set tool_reminder -%}Available Tools: {%- set tool_reminder = 'Available Tools:
{{ tools_json }} ' + tools_json + '
Tool Call Format Example: Tool Call Format Example:
{{ tool_start }}{{ example_tool_call }} ' + tool_start + example_tool_call + '
Prefix & Suffix: Begin tool calls with {{ tool_start }} and end with {{ tool_end }}. Prefix & Suffix: Begin tool calls with ' + tool_start + ' and end with ' + tool_end + '.
Argument Types: Use correct data types for arguments (e.g., strings in quotes, numbers without). Argument Types: Use correct data types for arguments (e.g., strings in quotes, numbers without).' -%}
{%- endset -%}
{# Template #} {# Template #}
@ -54,15 +51,15 @@ Argument Types: Use correct data types for arguments (e.g., strings in quotes, n
{%- if role not in message_roles -%} {%- if role not in message_roles -%}
{{ raise_exception('Invalid role ' + message['role'] + '. Only ' + message_roles | join(', ') + ' are supported.') }} {{ raise_exception('Invalid role ' + message['role'] + '. Only ' + message_roles | join(', ') + ' are supported.') }}
{%- endif -%} {%- endif -%}
{%- set content = message['content'] | default('', true) | trim -%} {%- set content = message['content'] if message['content'] is defined else '' | trim -%}
{%- if loop.first -%} {%- if loop.first -%}
{{ bos_token }}{{ start_header }}{{ role }}{{ end_header }} {{ bos_token }}{{ start_header }}{{ role }}{{ end_header }}
{{ inital_system_prompt }} {{ inital_system_prompt }}
{{ content }}{{ eos_token }} {{ content }}{{ eos_token }}
{%- endif -%} {%- endif -%}
{%- if not loop.first -%} {%- if not loop.first -%}
{{ start_header }}{{ role }}{{ end_header }} {{ start_header }}{{ role }}{{ end_header }}
{{ content }} {{ content }}
@ -81,4 +78,4 @@ Argument Types: Use correct data types for arguments (e.g., strings in quotes, n
{{ tool_precursor }}{{ tool_start }} {{ tool_precursor }}{{ tool_start }}
{%- else -%} {%- else -%}
{{ start_header }}assistant{{ end_header }} {{ start_header }}assistant{{ end_header }}
{%- endif -%} {%- endif -%}