File size: 11,827 Bytes
85f197c 495e0b0 85f197c 495e0b0 85f197c 495e0b0 85f197c 495e0b0 85f197c 73a1200 85f197c 495e0b0 85f197c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | ---
library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE
base_model: Qwen/Qwen3.8-27B
pipeline_tag: text-generation
tags:
- dense
- coding
- agentic
- unimodal
- repackaged
---
<center>
<img style='height: 16em' src='https://cdn-uploads.huggingface.co/production/uploads/63e9a1c4ccae1fe5c622b9f6/zOXMkdhMfeQWxDq6JHj0B.webp'/>
</center>
# Focus Red
repackaged [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B) with a simplified architecture and minor opinionated improvements
- all vision-related components removed
- reduced storage and memory, faster inference
- zero loss of output quality
## comparison
<table>
<tr>
<th></th>
<th>Qwen 3.8 27B</th>
<th>Focus Red</th>
</tr>
<tr>
<td>author</td>
<td><a href='https://huggingface.co/Qwen'>Alibaba Qwen</a></td>
<td><a href='https://huggingface.co/Jaidchen'>Jaid</a></td>
</tr>
<tr>
<td>repository</td>
<td><a href='https://huggingface.co/Qwen/Qwen3.8-27B'>Qwen/Qwen3.8-27B</a></td>
<td><a href='https://huggingface.co/Jaidchen/Focus-Red'>Jaidchen/Focus-Red</a></td>
</tr>
<tr>
<td>architecture</td>
<td><code>qwen3_5</code></td>
<td><code>qwen3_5_text</code></td>
</tr>
<tr>
<td>Transformers handler</td>
<td>
<a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForConditionalGeneration'><code>Qwen3_5ForConditionalGeneration</code></a>
</td>
<td>
<a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForCausalLM'><code>Qwen3_5ForCausalLM</code></a>
</td>
</tr>
<tr>
<td>tensor entries</td>
<td>1199</td>
<td>851</td>
</tr>
<tr style='opacity: 50%'>
<td>tensor type</td>
<td>bf16</td>
<td>bf16</td>
</tr>
<tr>
<td>parameters</td>
<td>27 781 427 952</td>
<td>26 895 998 464</td>
</tr>
<tr style='opacity: 50%'>
<td>active</td>
<td>100%</td>
<td>100%</td>
</tr>
<tr style='opacity: 50%'>
<td>vocabulary size</td>
<td>248 320</td>
<td>248 320</td>
</tr>
<tr style='opacity: 50%'>
<td>context size</td>
<td>262 144</td>
<td>262 144</td>
</tr>
<tr>
<td>MTP</td>
<td>integrated</td>
<td><a href='https://huggingface.co/Jaidchen/Focus-Red-MTP'>Jaidchen/Focus-Red-MTP</a></td>
</tr>
<tr>
<td>sampling strategy</td>
<td>random sampling</td>
<td>greedy/deterministic</td>
</tr>
<tr>
<td>sampling parameters</td>
<td>
<div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
<span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>true</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>1.0</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>20</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>0.95</span></div>
</td>
<td>
<div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
<span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>false</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>0</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span><br>
<span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span></div>
</div>
</td>
</tr>
<tr>
<td>input modality</td>
<td>text, image, video</td>
<td>text</td>
</tr>
<tr>
<td>model size</td>
<td>55 562 855 904</td>
<td>53 791 996 928</td>
</tr>
<tr>
<td>splits</td>
<td>18</td>
<td>none</td>
</tr>
<tr>
<td>Jinja template</td>
<td><a href='https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/chat_template.jinja'>Qwen original</a></td>
<td><a href='https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/chat_template.jinja'>Qwen original</a><br>+ <a href='https://huggingface.co/unsloth/Qwen3.8-27B/blob/main/chat_template.jinja'>Unsloth tweaks</a><br>+ <a href='https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/main/chat_template.jinja'>Froggeric tweaks</a><br>+ unimodality patch<br>+ further custom tweaks</td>
</tr>
</table>
## pros
- reduced storage needs
- reduced loading time
- reduced VRAM occupancy, thus more room for context
- increased inference speed
- simplified architecture, unlocking some further potential for optimizing low-level procedures
## cons
- legally blind
- Pictures and video frames can still be present in the context without crashing, but their contents are no longer interpreted by the model and won’t do anything else than waste space.
- If you occasionally rely on those capabilities, I suggest adding a `consult` tool to your harness that calls a vision-enabled subagent model like [Gemini Flash](https://openrouter.ai/~google/gemini-flash-latest) or [GPT](https://openrouter.ai/~openai/gpt-latest).
- reduced compatibility
- The simplified architecture is handled by the `Qwen3_5ForCausalLM` class which may not be included in your inference engine. In this case you would need to ask your agent or integrate it yourself.
- Inference engines confirmed to have native support:
- [SGLang](https://github.com/sgl-project/sglang) ≥ v0.5.17 (PR [#32401](https://github.com/sgl-project/sglang/pull/32401))
- [vLLM](https://github.com/vllm-project/vllm) ≥ v0.26.0 (PR [#50210](https://github.com/vllm-project/vllm/pull/50210))
- The applied coercions may confuse your inference engine in case it has fixed expectations about the model’s architecture and thus lead to unpredictable behavior.
## caveats
- model file not split, possibly causing issues if intended to be stored on an HDD from the previous century
- random sampling disabled by default, less suitable for long-form writing, entertainment and casual chat
## Jinja template
- base: [Qwen/Qwen3.8-27B original](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja)
- Retains Qwen3.8's native `reasoning_effort` handling (`xhigh` by default, plus `medium` and `low`) and its reasoning instructions. ([lines 45–59](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja#L45-L59))
- Retains injection of those reasoning instructions into the tool/system preamble whenever thinking is enabled. ([lines 60–94](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja#L60-L94))
- Preserves Qwen3.8's default-on historical thinking behavior while the later Froggeric layer avoids synthesizing empty thinking blocks. ([lines 108–120](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja#L108-L120))
- adopted tweaks from [Unsloth Qwen3.8](https://huggingface.co/unsloth/Qwen3.8-27B/blob/3ea932cee0a432ae86e9c7826cbe8aef52323a28/chat_template.jinja)
- commit `3ea932cee0a432ae86e9c7826cbe8aef52323a28`
- Added `developer` role as alias for `system` and merged leading system/developer messages into a single policy message. ([lines 45–59](https://huggingface.co/unsloth/Qwen3.8-27B/blob/3ea932cee0a432ae86e9c7826cbe8aef52323a28/chat_template.jinja#L45-L59), [lines 80–93](https://huggingface.co/unsloth/Qwen3.8-27B/blob/3ea932cee0a432ae86e9c7826cbe8aef52323a28/chat_template.jinja#L80-L93))
- Allowed histories without a normal human query. ([lines 95–107](https://huggingface.co/unsloth/Qwen3.8-27B/blob/3ea932cee0a432ae86e9c7826cbe8aef52323a28/chat_template.jinja#L95-L107))
- retained compatibility tweak from the previous [Unsloth Qwen3.6 layer](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja)
- Made mapping-argument rendering portable by avoiding `|items`; the current Unsloth Qwen3.8 template uses `|items` again, so Focus deliberately keeps the portable implementation. ([lines 122–130](https://huggingface.co/unsloth/Qwen3.6-27B/blob/d6e694245c1d535508bc54062f5e258d7b1e04f5/chat_template.jinja#L122-L130))
- adopted tweaks from [Froggeric Qwen3.8 version](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja)
- commit `9f14778c92c3b5ed3e0738085694c0d3452802dd`
- Retained `preserve_thinking` semantics and avoided rendering empty historical thinking blocks. ([lines 8–15](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L8-L15), [lines 227–233](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L227-L233))
- Retained mid-conversation system/developer messages as Qwen system turns. ([lines 172–183](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L172-L183))
- Allowed string-valued `message.thinking` as fallback for historical reasoning. ([lines 186–199](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L186-L199))
- Applied boundary-aware `</think>` parsing. ([lines 200–226](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L200-L226))
- Supported both wrapped and direct tool calls, but never null-wrapped. ([lines 234–241](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L234-L241))
- Preserved non-empty string tool arguments. ([lines 248–280](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L248-L280))
- Used direct message indexing for tool-response grouping instead of `loop.previtem`/`loop.nextitem`. ([lines 298–317](https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates/blob/9f14778c92c3b5ed3e0738085694c0d3452802dd/chat_template.jinja#L298-L317))
- custom tweaks
- Merged any number of leading system/developer messages, generalizing the Unsloth merge behavior.
- Kept Qwen3.8's reasoning instructions ahead of both merged policy text and the tools preamble.
- Applied boundary-aware `</think>` parsing but excluded Froggeric's malformed-tag recovery.
- Raised an error containing the offending role instead of Qwen's generic unknown-role error. ([lines 160–162](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja#L160-L162))
- Used direct message indexing for tool-response grouping without Froggeric's error-escalation state.
- Guarded undefined `tools`/`tool_calls` and handled non-mapping content items defensively.
- Retained Qwen's proposed `continue_final_message` fix for partial assistant prefills. ([lines 130–132](https://huggingface.co/Qwen/Qwen3.6-27B/blob/0774bb84a25294dac0538c2bfc7abc08ae89d522/chat_template.jinja#L130-L132))
- unimodality patch
- Removed Qwen3.8's vision-token machinery and rendered image/video content as `[image]` and `[video]`. ([lines 1–41](https://huggingface.co/Qwen/Qwen3.8-27B/blob/1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0/chat_template.jinja#L1-L41))
## license
Apache 2.0 – inherited from [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE) |