README.md

---
library_name: transformers
license: apache-2.0
license_link: https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE
base_model: Jaidchen/Focus-Red
pipeline_tag: text-generation
tags:
- dense
- coding
- agentic
- unimodal
- repackaged
- quantized
- compressed-tensors
- int4
---

<center>
<img style='height: 16em' src='https://cdn-uploads.huggingface.co/production/uploads/63e9a1c4ccae1fe5c622b9f6/zOXMkdhMfeQWxDq6JHj0B.webp'/>
</center>

# Focus-Red-Int4

4-bit quantization of [Focus-Red](https://huggingface.co/Jaidchen/Focus-Red), itself a text-only repackaging of [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B).

- W4A16, group size 32, asymmetric weights
- `compressed-tensors` / `pack-quantized` format
- selected linear-attention projections and `lm_head` remain BF16
- single `model.safetensors` file

## comparison

<table>
<tr>
  <th></th>
  <th>Qwen 3.8 27B</th>
  <th>Focus-Red-Int4</th>
</tr>
<tr>
  <td>author</td>
  <td><a href='https://huggingface.co/Qwen'>Alibaba Qwen</a></td>
  <td><a href='https://huggingface.co/Jaidchen'>Jaid</a></td>
</tr>
<tr>
  <td>repository</td>
  <td><a href='https://huggingface.co/Qwen/Qwen3.8-27B'>Qwen/Qwen3.8-27B</a></td>
  <td><a href='https://huggingface.co/Jaidchen/Focus-Red-Int4'>Jaidchen/Focus-Red-Int4</a></td>
</tr>
<tr>
  <td>architecture</td>
  <td><code>qwen3_5</code></td>
  <td><code>qwen3_5_text</code></td>
</tr>
<tr>
  <td>Transformers handler</td>
  <td>
  <a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForConditionalGeneration'><code>Qwen3_5ForConditionalGeneration</code></a>
  </td>
  <td>
  <a href='https://huggingface.co/docs/transformers/model_doc/qwen3_5#transformers.Qwen3_5ForCausalLM'><code>Qwen3_5ForCausalLM</code></a>
  </td>
</tr>
<tr>
  <td>tensor entries</td>
  <td>1199</td>
  <td>2051</td>
</tr>
<tr style='opacity: 50%'>
  <td>tensor type</td>
  <td>bf16</td>
  <td>W4A16 G32 asymmetric + selected BF16</td>
</tr>
<tr>
  <td>parameters</td>
  <td>27 781 427 952</td>
  <td>26 895 998 464</td>
</tr>
<tr style='opacity: 50%'>
  <td>active</td>
  <td>100%</td>
  <td>100%</td>
</tr>
<tr style='opacity: 50%'>
  <td>vocabulary size</td>
  <td>248 320</td>
  <td>248 320</td>
</tr>
<tr style='opacity: 50%'>
  <td>context size</td>
  <td>262 144</td>
  <td>262 144</td>
</tr>
<tr style='opacity: 50%'>
  <td>MTP</td>
  <td>integrated</td>
  <td>detached → <a href='https://huggingface.co/Jaidchen/Focus-Red-MTP'>Focus-Red-MTP</a></td>
</tr>
<tr>
  <td>sampling strategy</td>
  <td>random sampling</td>
  <td>greedy/deterministic</td>
</tr>
<tr>
  <td>sampling parameters</td>
  <td>
    <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
      <span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>true</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>1.0</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>20</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>0.95</span></div>
  </td>
  <td>
    <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
      <span style='color: hsl(from currentColor 0 100% l)'>do_sample</span>: <span style='color: hsl(from currentColor 50 80% l)'>false</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>temperature</span>: <span style='color: hsl(from currentColor 50 80% l)'>0</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>top_k</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span><br>
      <span style='color: hsl(from currentColor 0 100% l)'>top_p</span>: <span style='color: hsl(from currentColor 50 80% l)'>1</span></div>
    </div>
  </td>
</tr>
<tr>
  <td>input modality</td>
  <td>text, image, video</td>
  <td>text</td>
</tr>
<tr>
  <td>model size</td>
  <td>55 562 855 904</td>
  <td>19 202 352 336 bytes on disk</td>
</tr>
<tr>
  <td>splits</td>
  <td>18</td>
  <td>none</td>
</tr>
<tr>
  <td>Jinja template</td>
  <td><a href='https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/chat_template.jinja'>Qwen original</a></td>
  <td><a href='https://github.com/jaidlab/focus-chat-template'>focus-chat-template</a> dist build</td>
</tr>
</table>

## quantization

- method: `compressed-tensors`
- format: `pack-quantized`
- weights: 4-bit integer, group size 32, asymmetric
- activations: unquantized / BF16
- quantization status: `compressed`
- model file: 19,202,352,336 bytes

## pros

- reduced storage needs
- reduced loading time
- reduced VRAM occupancy, thus more room for context
- increased inference speed
- simplified architecture, unlocking some further potential for optimizing low-level procedures

## cons

- legally blind
  - Pictures and video frames can still be present in the context without crashing, but their contents are no longer interpreted by the model and won’t do anything else than waste space.
  - If you occasionally rely on those capabilities, I suggest adding a `consult` tool to your harness that calls a vision-enabled subagent model like [Gemini Flash](https://openrouter.ai/~google/gemini-flash-latest) or [GPT](https://openrouter.ai/~openai/gpt-latest).
- reduced compatibility
  - The simplified architecture is handled by the `Qwen3_5ForCausalLM` class which may not be included in your inference engine. In this case you would need to ask your agent or integrate it yourself.
    - Inference engines confirmed to have native support:
      - [SGLang](https://github.com/sgl-project/sglang) ≥ v0.5.17 (PR [#32401](https://github.com/sgl-project/sglang/pull/32401))
      - [vLLM](https://github.com/vllm-project/vllm) ≥ v0.26.0 (PR [#50210](https://github.com/vllm-project/vllm/pull/50210))
  - The applied coercions may confuse your inference engine in case it has fixed expectations about the model’s architecture and thus lead to unpredictable behavior.

## caveats

- model file not split, possibly causing issues if intended to be stored on an HDD from the previous century
- random sampling disabled by default, less suitable for long-form writing, entertainment and casual chat

## Jinja template

This build uses `dist/chat_template.jinja` from [jaidlab/focus-chat-template](https://github.com/jaidlab/focus-chat-template). The template is reproducibly built from Qwen/Qwen3.8-27B's pinned upstream template at commit `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0` plus the repository's ordered patch stack.

- template SHA-256: `5c381ca45e9538c7a2406331b554ee7d62cf3d0b8c115f17687d4fdd5590a239`
- template size: 9,710 bytes

## license

Apache 2.0 – inherited from [Qwen 3.8 27B](https://huggingface.co/Qwen/Qwen3.8-27B/blob/main/LICENSE)