[FIX] VLLM reasoning with mm: is a part of context
#6
by voves - opened
- chat_template.jinja +7 -8
chat_template.jinja
CHANGED
|
@@ -159,13 +159,12 @@
|
|
| 159 |
{%- endif %}
|
| 160 |
{%- endif %}
|
| 161 |
|
| 162 |
-
{
|
| 163 |
-
|
| 164 |
-
|
| 165 |
-
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
{%- endif -%}
|
| 169 |
|
| 170 |
{%- if content -%}
|
| 171 |
{{- content }}
|
|
@@ -244,4 +243,4 @@
|
|
| 244 |
{%- else -%}
|
| 245 |
{#- adaptive: no prefix, let model decide -#}
|
| 246 |
{%- endif -%}
|
| 247 |
-
{%- endif -%}
|
|
|
|
| 159 |
{%- endif %}
|
| 160 |
{%- endif %}
|
| 161 |
|
| 162 |
+
{#- PATCH: strip reasoning from history to save context.
|
| 163 |
+
Only emit think_end_token prefix so the model knows the
|
| 164 |
+
(omitted) thinking block ended. Original behavior rendered
|
| 165 |
+
<mm:think>...</mm:think> for every assistant turn, which
|
| 166 |
+
caused context bloat in multi-turn conversations. -#}
|
| 167 |
+
{{- think_end_token }}
|
|
|
|
| 168 |
|
| 169 |
{%- if content -%}
|
| 170 |
{{- content }}
|
|
|
|
| 243 |
{%- else -%}
|
| 244 |
{#- adaptive: no prefix, let model decide -#}
|
| 245 |
{%- endif -%}
|
| 246 |
+
{%- endif -%}
|