Hugging Face's logo

Jaidchen
/
Focus

Text Generation
Transformers
Safetensors
qwen3_5_text
dense
coding
agentic
unimodal
repackaged
conversational
Model card Files
xet
 Community
Jaidchen commited on
Commit
2e50e7d
·
verified ·
1 Parent(s): 3339f6d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +33 -7
README.md CHANGED
@@ -8,7 +8,11 @@ pipeline_tag: text-generation
8
 
9
  # Focus
10
 
11
- rebuilt [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplified architecture and minor improvements
 
 
 
 
12
 
13
  ## comparison
14
 
@@ -38,7 +42,7 @@ rebuilt [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplifie
38
  <td>1199</td>
39
  <td>866</td>
40
  </tr>
41
- <tr style='background: #5552; color: rgb(from currentColor r g b / 70%)'>
42
  <td>tensor type</td>
43
  <td>bf16</td>
44
  <td>bf16</td>
@@ -48,17 +52,17 @@ rebuilt [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplifie
48
  <td>27 781 427 952</td>
49
  <td>27 320 697 856</td>
50
  </tr>
51
- <tr style='background: #5552; color: rgb(from currentColor r g b / 70%)'>
52
  <td>vocabulary size</td>
53
  <td>248 320</td>
54
  <td>248 320</td>
55
  </tr>
56
- <tr style='background: #5552; color: rgb(from currentColor r g b / 70%)'>
57
  <td>context size</td>
58
  <td>262 144</td>
59
  <td>262 144</td>
60
  </tr>
61
- <tr style='background: #5552; color: rgb(from currentColor r g b / 70%)'>
62
  <td>MTP</td>
63
  <td>integrated</td>
64
  <td>integrated</td>
@@ -80,10 +84,16 @@ rebuilt [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplifie
80
  <tr>
81
  <td>sampling parameters</td>
82
  <td>
83
- <div style='font-family: Jaidevka Code, JetBrainsMono, monospace'><span style='color: #ff3d6d'>do_sample</span>: <span style='color: #fff570'>true</span><br><span style='color: #ff3d6d'>temperature</span>: <span style='color: #fff570'>0.6</span><br><span style='color: #ff3d6d'>top_k</span>: <span style='color: #fff570'>20</span><br><span style='color: #ff3d6d'>top_p</span>: <span style='color: #fff570'>0.95</span></div>
 
 
 
 
84
  </td>
85
  <td>
86
- <div style='font-family: Jaidevka Code, JetBrainsMono, monospace'><span style='color: #ff3d6d'>do_sample</span>: <span style='color: #fff570'>false</span></div>
 
 
87
  </td>
88
  </tr>
89
  <tr>
@@ -108,6 +118,22 @@ rebuilt [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplifie
108
  </tr>
109
  </table>
110
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111
  ## license
112
 
113
  Apache 2.0, derived from [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/LICENSE)
 
8
 
9
  # Focus
10
 
11
+ repackaged [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B) with a simplified architecture and minor opinionated improvements
12
+
13
+ - all vision-related components removed
14
+ - reduced storage and memory, faster inference
15
+ - zero loss of output quality
16
 
17
  ## comparison
18
 
 
42
  <td>1199</td>
43
  <td>866</td>
44
  </tr>
45
+ <tr style='background: rgb(from currentColor r g b / 10%); color: rgb(from currentColor r g b / 60%)'>
46
  <td>tensor type</td>
47
  <td>bf16</td>
48
  <td>bf16</td>
 
52
  <td>27 781 427 952</td>
53
  <td>27 320 697 856</td>
54
  </tr>
55
+ <tr style='background: rgb(from currentColor r g b / 10%); color: rgb(from currentColor r g b / 60%)'>
56
  <td>vocabulary size</td>
57
  <td>248 320</td>
58
  <td>248 320</td>
59
  </tr>
60
+ <tr style='background: rgb(from currentColor r g b / 10%); color: rgb(from currentColor r g b / 60%)'>
61
  <td>context size</td>
62
  <td>262 144</td>
63
  <td>262 144</td>
64
  </tr>
65
+ <tr style='background: rgb(from currentColor r g b / 10%); color: rgb(from currentColor r g b / 60%)'>
66
  <td>MTP</td>
67
  <td>integrated</td>
68
  <td>integrated</td>
 
84
  <tr>
85
  <td>sampling parameters</td>
86
  <td>
87
+ <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
88
+ <span style='color: #ff3d6d'>do_sample</span>: <span style='color: #fff570'>true</span><br>
89
+ <span style='color: #ff3d6d'>temperature</span>: <span style='color: #fff570'>0.6</span><br>
90
+ <span style='color: #ff3d6d'>top_k</span>: <span style='color: #fff570'>20</span><br>
91
+ <span style='color: #ff3d6d'>top_p</span>: <span style='color: #fff570'>0.95</span></div>
92
  </td>
93
  <td>
94
+ <div style='font-family: Jaidevka Code, JetBrains Mono, monospace; line-height: initial'>
95
+ <span style='color: #ff3d6d'>do_sample</span>: <span style='color: #fff570'>false</span>
96
+ </div>
97
  </td>
98
  </tr>
99
  <tr>
 
118
  </tr>
119
  </table>
120
 
121
+ ## pros
122
+
123
+ - reduced storage needs
124
+ - reduced loading time
125
+ - reduced VRAM occupancy, thus more room for context
126
+ - increased inference speed
127
+ - simplified architecture, unlocking some further potential for optimizing low-level procedures
128
+
129
+ ## cons
130
+
131
+ - legally blind
132
+ - Pictures and video frames can still be present in the context without crashing, but their contents are no longer interpreted by the model and won’t do anything else than waste space.
133
+ - If you occasionally rely on those capabilities, I suggest adding a `consult` tool to your harness that calls a vision-enabled subagent model like [Gemini Flash](https://openrouter.ai/~google/gemini-flash-latest) or [GPT](https://openrouter.ai/~openai/gpt-latest).
134
+ - The applied coercions may confuse your inference engine in case it has fixed expectations about the model’s architecture and thus lead to unpredictable behavior.
135
+ - model file not split, possibly causing issues if intended to be stored on an HDD from the previous century
136
+
137
  ## license
138
 
139
  Apache 2.0, derived from [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.5-27B/blob/main/LICENSE)