Fix: Context Retention Fixes After Compression
#40
by ross232323 - opened
Updated the title and description of this thread. What started out as just one fix ended up being many.
Using LM Studio + QwenCode together I have fixed issues where the agent would stall midway through.
UPDATE 1 - 11/06/26: I have also included a few other stall related fixes related to compaction leaving incomplete tool calls. This happens to me a lot due to small context size.
UPDATE 2 - 13/06/26: Additional updates to improve context retention.
I am running 60k context with the following QwenCode settings in my settings.json:
"temperature": 0.3,
"top_p": 0.9,
"top_k": 40,
"presence_penalty": 0.1,
"frequency_penalty": 0.0,
"repetition_penalty": 1.12,
"max_tokens": 16384
Note I raised a bug in QwenCode which got fixed in master. Hard and Auto compression levels on small contexts were the same (viewed with /context). With only a 60k context I needed compression to work reliably to allow full agentic coding.
Here is the fixed version:
{%- set template_version = "qwen3.6-froggeric-v34.6-contextfixes" %}
{%- set image_count = namespace(value=0) %}
{%- set video_count = namespace(value=0) %}
{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
{%- set _preserve_thinking = preserve_thinking if preserve_thinking is defined else false %}
{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
{%- set ns_state = namespace(thinking=enable_thinking) %}
{%- if auto_disable_thinking_with_tools and _has_tools %}
{%- set ns_state.thinking = false %}
{%- endif %}
{%- macro render_content(content, do_vision_count, is_system_content=false) %}
{%- if content is string %}
{%- set is_compaction = ('state_snapshot' in content or 'input token limit' in content or 'compressed context' in content or 'approached the input token limit' in content) %}
{%- if is_compaction and not ('<tool_call>' in content or '<tool_call>' in content) %}
{%- set parts = content.split('✦') %}
{%- set notice_part = parts[0] | trim %}
{%- set history_ns = namespace(text='') %}
{%- for part in parts[1:] %}
{%- set clean_part = part | trim %}
{%- if clean_part %}
{%- set block = clean_part.split('Let me ')[0].split('Now I need ')[0].split('Now I shall ')[0].split("Let's ")[0] | trim %}
{%- if block %}
{%- set history_ns.text = history_ns.text ~ '\n- ' ~ block %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set reconstructed = notice_part %}
{%- if history_ns.text %}
{%- set reconstructed = reconstructed ~ '\n\n### Historical Knowledge Constants (Recovered):' ~ history_ns.text %}
{%- endif %}
{{- reconstructed ~ '\n\n[System Infrastructure Note: Context decompression complete. The above points are validated historical facts. Maintain standard system operation. Continue execution by outlining your immediate technical plan inside <think>, then execute required tools or final resolution.]' }}
{%- else %}
{{- content }}
{%- endif %}
{%- elif content is iterable and content is not mapping %}
{%- for item in content %}
{%- if item is mapping %}
{%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain images.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set image_count.value = image_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Picture ' ~ image_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|image_pad|><|vision_end|>' }}
{%- elif item.type == 'video' or 'video' in item %}
{%- if is_system_content %}
{{- raise_exception('System message cannot contain videos.') }}
{%- endif %}
{%- if do_vision_count %}
{%- set video_count.value = video_count.value + 1 %}
{%- endif %}
{%- if add_vision_id %}
{{- 'Video ' ~ video_count.value ~ ': ' }}
{%- endif %}
{{- '<|vision_start|><|video_pad|><|vision_end|>' }}
{%- elif 'text' in item %}
{%- set item_text_str = item.text | string %}
{%- set is_compaction = ('state_snapshot' in item_text_str or 'input token limit' in item_text_str or 'compressed context' in item_text_str or 'approached the input token limit' in item_text_str) %}
{%- if is_compaction and not ('<tool_call>' in item_text_str or '<tool_call>' in item_text_str) %}
{%- set parts = item_text_str.split('✦') %}
{%- set notice_part = parts[0] | trim %}
{%- set history_ns = namespace(text='') %}
{%- for part in parts[1:] %}
{%- set clean_part = part | trim %}
{%- if clean_part %}
{%- set block = clean_part.split('Let me ')[0].split('Now I need ')[0].split('Now I shall ')[0].split("Let's ")[0] | trim %}
{%- if block %}
{%- set history_ns.text = history_ns.text ~ '\n- ' ~ block %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set reconstructed = notice_part %}
{%- if history_ns.text %}
{%- set reconstructed = reconstructed ~ '\n\n### Historical Knowledge Constants (Recovered):' ~ history_ns.text %}
{%- endif %}
{{- reconstructed ~ '\n\n[System Infrastructure Note: Context decompression complete. The above points are validated historical facts. Maintain standard system operation. Continue execution by outlining your immediate technical plan inside <think>, then execute required tools or final resolution.]' }}
{%- else %}
{{- item.text }}
{%- endif %}
{%- else %}
{{- raise_exception('Unexpected item type in content.') }}
{%- endif %}
{%- else %}
{%- set item_str = item | string %}
{%- set is_compaction = ('state_snapshot' in item_str or 'input token limit' in item_str or 'compressed context' in item_str or 'approached the input token limit' in item_str) %}
{%- if is_compaction and not ('<tool_call>' in item_str or '<tool_call>' in item_str) %}
{%- set parts = item_str.split('✦') %}
{%- set notice_part = parts[0] | trim %}
{%- set history_ns = namespace(text='') %}
{%- for part in parts[1:] %}
{%- set clean_part = part | trim %}
{%- if clean_part %}
{%- set block = clean_part.split('Let me ')[0].split('Now I need ')[0].split('Now I shall ')[0].split("Let's ")[0] | trim %}
{%- if block %}
{%- set history_ns.text = history_ns.text ~ '\n- ' ~ block %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- set reconstructed = notice_part %}
{%- if history_ns.text %}
{%- set reconstructed = reconstructed ~ '\n\n### Historical Knowledge Constants (Recovered):' ~ history_ns.text %}
{%- endif %}
{{- reconstructed ~ '\n\n[System Infrastructure Note: Context decompression complete. The above points are validated historical facts. Maintain standard system operation. Continue execution by outlining your immediate technical plan inside <think>, then execute required tools or final resolution.]' }}
{%- else %}
{{- item_str }}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- elif content is none or content is undefined %}
{{- '' }}
{%- else %}
{{- raise_exception('Unexpected content type.') }}
{%- endif %}
{%- endmacro %}
{%- if not messages %}
{{- raise_exception('No messages provided.') }}
{%- endif %}
{%- set _first_role = messages[0].role %}
{%- if _first_role == 'system' or _first_role == 'developer' %}
{%- set _sys_msg = messages[0] %}
{%- set _msgs = messages[1:] %}
{%- else %}
{%- set _sys_msg = none %}
{%- set _msgs = messages %}
{%- endif %}
{%- set _sc = '' %}
{%- if _sys_msg is not none %}
{%- set _sc = render_content(_sys_msg.content, false, true) | replace('✦ ', '') | replace('✦', '') | trim %}
{%- if '<|think_off|>' in _sc %}
{%- set ns_state.thinking = false %}
{%- set _sc = _sc.split('<|think_off|>') | join('') | trim %}
{%- elif '<|think_on|>' in _sc %}
{%- set ns_state.thinking = true %}
{%- set _sc = _sc.split('<|think_on|>') | join('') | trim %}
{%- endif %}
{%- endif %}
{%- if _has_tools %}
{{- '<|im_start|>system\n' }}
{{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
{%- for tool in tools %}
{{- '\n' }}
{{- tool | tojson }}
{%- endfor %}
{{- '\n</tools>' }}
{{- '\n\n' }}
If you choose to call a function, you MUST output the call immediately using this exact XML structure on a new line with no transitional text or symbols:
<tool_call>
<function=function_name>
<parameter=parameter_name>
value
</parameter>
</function>
</tool_call>
CRITICAL PARSER RULES:
- Do NOT indent any lines or add leading spaces before tags. Every tag must start flush against the absolute left margin of the line.
- If you run an internal monologue first, the closing </think> tag must be followed immediately by the opening <tool_call> tag on the very next line.
- You are strictly forbidden from leaking conversational prose, code scraps, language comments (e.g., HTML comment brackets or code comment slashes //), spaces, or custom symbols outside of your XML blocks.
- The boundary between structural blocks must be entirely pure. Do not bleed code comments across structural transitions.
- If you have all necessary data to fully answer the user's request, provide your final response cleanly without issuing a tool call.
{%- if _sc %}
{{- '\n\n' + _sc }}
{%- endif %}
{{- '<|im_end|>\n' }}
{%- else %}
{%- if _sc %}
{{- '<|im_start|>system\n' + _sc + '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set _last_idx = _msgs | length - 1 %}
{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx, has_snapshot=false) %}
{%- for message in _msgs[::-1] %}
{%- set index = (_msgs | length - 1) - loop.index0 %}
{%- set _rc = render_content(message.content, false) | trim %}
{%- if 'state_snapshot' in _rc or 'input token limit' in _rc or 'compressed context' in _rc %}
{%- set ns.has_snapshot = true %}
{%- endif %}
{%- if ns.multi_step_tool and message.role == 'user' %}
{%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
{%- set ns.multi_step_tool = false %}
{%- set ns.last_query_index = index %}
{%- endif %}
{%- endif %}
{%- endfor %}
{%- if ns.multi_step_tool or ns.has_snapshot %}
{%- if _last_idx > 50 and not ns.has_snapshot %}
{%- set ns.last_query_index = _last_idx %}
{%- else %}
{%- set ns.last_query_index = -1 %}
{%- endif %}
{%- endif %}
{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
{%- for message in _msgs %}
{%- set is_system = (message.role == "system" or message.role == "developer") %}
{%- set content = render_content(message.content, true, is_system) | replace('✦ ', '') | replace('✦', '') | trim %}
{%- if '<|think_off|>' in content %}
{%- set ns_state.thinking = false %}
{%- set content = content.split('<|think_off|>') | join('') | trim %}
{%- elif '<|think_on|>' in content %}
{%- set ns_state.thinking = true %}
{%- set content = content.split('<|think_on|>') | join('') | trim %}
{%- endif %}
{%- if is_system %}
{{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
{%- elif message.role == 'user' %}
{%- set ns2.consecutive_failures = 0 %}
{{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
{%- elif message.role == 'assistant' %}
{%- set reasoning_content = '' %}
{%- set final_content = content %}
{%- if message.reasoning_content is defined and message.reasoning_content is not none %}
{%- set reasoning_content = message.reasoning_content | string | replace('✦ ', '') | replace('✦', '') | trim %}
{%- else %}
{%- set TIGHT = content %}
{%- if '</think>' in TIGHT %}
{%- set parts = TIGHT.split('</think>') %}
{%- set reasoning_content = parts[0].split('<think>')[-1] | trim %}
{%- set final_content = parts[1:] | join('</think>') | trim %}
{%- endif %}
{%- endif %}
{%- if '<tool_call>' in final_content %}
{%- set _tc_parts = final_content.split('<tool_call>') %}
{%- set final_content = '\n<tool_call>' ~ _tc_parts[1:] | join('<tool_call>') %}
{%- endif %}
{%- if (_preserve_thinking or loop.index0 > ns.last_query_index) and reasoning_content %}
{{- '<|im_start|>assistant\n<think>\n' ~ reasoning_content ~ '\n</think>' ~ final_content }}
{%- else %}
{{- '<|im_start|>assistant' ~ final_content }}
{%- endif %}
{%- if loop.last and (('<tool_call>' in content and not '</tool_call>' in content) or ('<think>' in content and not '</think>' in content)) %}
{#- Leave open if truncated -#}
{%- else %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- elif message.role == 'tool' %}
{%- if ns2.prev_role != 'tool' %}
{{- '<|im_start|>user' }}
{%- endif %}
{{- '\n<tool_response>\n' + content + '\n</tool_response>' }}
{%- if loop.last or _msgs[loop.index0 + 1].role != 'tool' %}
{{- '<|im_end|>\n' }}
{%- endif %}
{%- endif %}
{%- set ns2.prev_role = message.role %}
{%- endfor %}
{%- if add_generation_prompt %}
{{- '<|im_start|>assistant\n' }}
{%- if ns_state.thinking %}
{{- '<think>\n' }}
{%- if _msgs | length > 6 %}
{{- "[SYSTEM OPERATIONAL ANCHOR: Deep context threshold exceeded. Beware of structural fatigue and syntax bleed (accidentally outputting HTML comment tags, code notes, or frontend symbols like '✦' or '●' outside your tags). You must keep your structural blocks perfectly clean. State your technical plan inside <think>, close it with </think>, and begin <tool_call> immediately on the next line flush against the left margin. Zero transitional text allowed.]\n" }}
{%- endif %}
{%- endif %}
{%- endif %}
ross232323 changed discussion title from Fix: The system tried to preserve a summary of where we left off (the <state_snapshot> block), but XML tags in that summary are being silently stripped to Fix: Context Retention Fixes After Compression
ross232323 changed discussion status to closed
ross232323 changed discussion status to open
Fails entirely when used with OpenCode. Outputs tool calls in response which isn't parsed by the harness causing the response to just....end.
Backend used:
Llama-cpp build 9632 with vulkan backend
llama flags used:
--jinja --chat-template-file "D:\llms\chat_templatev2.jinja" --chat-template-kwargs "{"preserve_thinking":true}" -kvu -fa on --no-warmup --mlock --cache-type-k q8_0 --cache-type-v q8_0 --metrics --temp 0.6 --top-p 0.95 --min-p 0.0 --repeat-penalty 1.0 --top-k 20 --presence-penalty 0.0 --fit on --fit-ctx 150000 --fit-target 1536 -np 1
Does the same thing in pi, GHCP and Zed agent. Switching to a different template resolves the issue entirely - implying its a template issue
ross232323 changed discussion status to closed