jalr/tabbyAPI-ollama
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Templates: Alter chatml_with_headers to fit huggingface spec

Browse source 
The previous template was compatible with Jinja2 in Python, but it
was not cross-platform compatible according to HF's standards.

Signed-off-by: kingbri <8082010+bdashore3@users.noreply.github.com>
This commit is contained in:
kingbri 2024-12-30 14:00:44 -05:00
parent 709493837b
commit 245bd5c008
1 changed files with 15 additions and 18 deletions
Download patch file Download diff file Expand all files Collapse all files

33
templates/tool_calls/chatml_with_headers.jinja
View file

@ -6,7 +6,7 @@
{%- set start_header = "<|start_header_id|>" -%}
{%- set end_header = "<|end_header_id|>\n" -%}
{%- set example_tool_call -%}[
{%- set example_tool_call = '[
{
"id": "tool_id_1342",
"function": {
@ -23,29 +23,26 @@
},
"type": "function"
}
]
{%- endset -%}
]' -%}
{%- set inital_system_prompt -%}You are an assistant that has access to the following set of tools, to call a tool:
1. Prefix calls with '{{ tool_start }}' and end calls with '{{ tool_end }}'
{%- set inital_system_prompt = 'You are an assistant that has access to the following set of tools, to call a tool:
1. Prefix calls with ' + tool_start + ' and end calls with ' + tool_end + '
2. Ensure you use the correct type for arguments. For example, if the argument is a string, ensure it is enclosed in quotes, otherwise, it should not be.
3. Generate all calls using the following json tool call format. Here is a multi tool call example:
{{ tool_start }}{{ example_tool_call }}{{ tool_end }}
' + tool_start + example_tool_call + tool_end + '
Here are the tools available for you to call:
{{ tools_json }}
{%- endset -%}
' + tools_json -%}
{%- set tool_reminder -%}Available Tools:
{{ tools_json }}
{%- set tool_reminder = 'Available Tools:
' + tools_json + '
Tool Call Format Example:
{{ tool_start }}{{ example_tool_call }}
' + tool_start + example_tool_call + '
Prefix & Suffix: Begin tool calls with {{ tool_start }} and end with {{ tool_end }}.
Argument Types: Use correct data types for arguments (e.g., strings in quotes, numbers without).
{%- endset -%}
Prefix & Suffix: Begin tool calls with ' + tool_start + ' and end with ' + tool_end + '.
Argument Types: Use correct data types for arguments (e.g., strings in quotes, numbers without).' -%}
{# Template #}
@ -54,15 +51,15 @@ Argument Types: Use correct data types for arguments (e.g., strings in quotes, n
{%- if role not in message_roles -%}
{{ raise_exception('Invalid role ' + message['role'] + '. Only ' + message_roles | join(', ') + ' are supported.') }}
{%- endif -%}
{%- set content = message['content'] | default('', true) | trim -%}
{%- set content = message['content'] if message['content'] is defined else '' | trim -%}
{%- if loop.first -%}
{{ bos_token }}{{ start_header }}{{ role }}{{ end_header }}
{{ inital_system_prompt }}
{{ content }}{{ eos_token }}
{%- endif -%}
{%- if not loop.first -%}
{{ start_header }}{{ role }}{{ end_header }}
{{ content }}
@ -81,4 +78,4 @@ Argument Types: Use correct data types for arguments (e.g., strings in quotes, n
{{ tool_precursor }}{{ tool_start }}
{%- else -%}
{{ start_header }}assistant{{ end_header }}
{%- endif -%}
{%- endif -%}