AIMindLink commited on
Commit
e5ef0c8
·
verified ·
1 Parent(s): fae7166

Upload README_Qwen3.8_27b.md

Browse files

Added unsloth/Qwen3.8-27B-GGUF model readme

Files changed (1) hide show
  1. model-readme/README_Qwen3.8_27b.md +119 -0
model-readme/README_Qwen3.8_27b.md ADDED
@@ -0,0 +1,119 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - Qwen/Qwen3.8-27B
4
+ license: apache-2.0
5
+ tags:
6
+ - unsloth
7
+ ---
8
+
9
+ # Read our How to [Run Qwen3.8-27B Guide!](https://unsloth.ai/docs/models/qwen3.8)
10
+ <div>
11
+ <p style="margin: 0 0 0px 0; margin-top: 0px;">
12
+ <em><a href="https://unsloth.ai/docs/basics/dynamic-3.0-ggufs">Unsloth Dynamic 3.0</a> achieves superior accuracy & outperforms other leading quants.</em>
13
+ </p>
14
+ <div style="display: flex; gap: 5px; align-items: center; margin-bottom: 0px;">
15
+ <a href="https://github.com/unslothai/unsloth/">
16
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/unsloth%20new%20logo.png" width="133">
17
+ </a>
18
+ <a href="https://discord.gg/unsloth">
19
+ <img src="https://github.com/unslothai/unsloth/raw/main/images/Discord%20button.png" width="173">
20
+ </a>
21
+ <a href="https://unsloth.ai/docs/models/qwen3.8">
22
+ <img src="https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/images/documentation%20green%20button.png" width="143">
23
+ </a>
24
+ </div>
25
+ <ul style="margin: 0;">
26
+ <li>Introducing <a href="https://unsloth.ai/docs/basics/dynamic-3.0-ggufs">Dynamic V3.0</a> GGUFs for SOTA accuracy and quantization performance</li>
27
+ <li>Run and fine-tune Qwen3.8 in <a href="https://unsloth.ai/docs/new/desktop">Unsloth Desktop</a> with <strong>Thinking toggles</strong>. <a href="https://unsloth.ai">Download</a> for Mac, Windows and Linux.</a> <a href="github.com/unslothai/unsloth">GitHub repo</a></li>
28
+ <li>Developer Role Support so Qwen3.8 can work in agentic tools like Codex and more!</li>
29
+ <li>Tool calling improvements: Makes parsing nested objects to make tool calling succeed more.</li>
30
+ <li>See below for 4-bit Qwen3.8-27B run inside of Unsloth Desktop:</li>
31
+ </div>
32
+
33
+ <img width="600" alt="qwen3.8 unsloth desktop" src="https://3215535692-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FSqxs6NjShWrLfRKhDy1m%2Fvolcano%202.gif?alt=media&token=395274a0-b437-403a-8a01-8e8502f9d225" />
34
+
35
+ Analysis of best Qwen3.8 GGUF providers. Unsloth Dynamic v3.0 delivers >10% top-1% better accuracy at the same size compared to every other provider. [Read more](https://unsloth.ai/docs/basics/dynamic-3.0-ggufs)
36
+
37
+ <img width="600" alt="qwen3.8 unsloth desktop" src="https://unsloth.ai/docs/~gitbook/image?url=https%3A%2F%2F3215535692-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252FxhOjnexMCB3dmuQFQ2Zq%252Fuploads%252Fl9piThTmmUsePst5w3F2%252Fimage.png%3Falt%3Dmedia%26token%3D821e88e2-ad4e-40bf-adf2-e13837228e82&width=768&dpr=3&quality=100&sign=dd5f7cb5&sv=2" />
38
+
39
+
40
+ ---
41
+
42
+ # Qwen3.8-27B
43
+
44
+ Following the widespread community adoption of the Qwen3.5 and Qwen3.6 series, we are pleased to introduce Qwen3.8, the most capable generation in the Qwen open-model family to date.
45
+
46
+ Built on the architectural foundation of Qwen3.5, Qwen3.8 delivers substantial gains across coding, professional work, research, and long-horizon agentic tasks. Qwen3.8-27B brings these advances to a compact, deployment-friendly dense model: a native vision-language model that understands images and videos, with flexible thinking control, designed to carry complex, multi-step tasks through to completion with greater reliability.
47
+
48
+ ## Qwen3.8 Highlights
49
+
50
+ Qwen3.8-27B features the following enhancements:
51
+ - **Core Capabilities**: Comprehensive improvements across coding, professional work, research, and long-horizon agentic tasks.
52
+ - **Agent Execution**: Stronger autonomous planning and better handling of environment feedback, leading to more reliable end-to-end task completion.
53
+ - **Downstream Compatibility**: Broader support for popular harnesses and development tools, making it easier to integrate into your existing stack.
54
+ - **Flexible Thinking Control**: Thinking mode is on by default and can be disabled per request; reasoning depth can be tuned with `reasoning_effort`, and reasoning context from historical messages is retained via `preserve_thinking`.
55
+ - **Vision-Language Understanding**: Native support for image and video understanding, from STEM diagrams and documents to hour-scale videos.
56
+
57
+
58
+ ## Model Overview
59
+
60
+ - Type: Causal Language Model with Vision Encoder
61
+ - Training Stage: Pre-training & Post-training
62
+ - Language Model
63
+ - Number of Parameters: 27B
64
+ - Hidden Dimension: 5120
65
+ - Token Embedding: 248,320 (Padded)
66
+ - Number of Layers: 64
67
+ - Hidden Layout: 16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))
68
+ - Gated DeltaNet:
69
+ - Number of Linear Attention Heads: 48 for V and 16 for QK
70
+ - Head Dimension: 128
71
+ - Gated Attention:
72
+ - Number of Attention Heads: 24 for Q and 4 for KV
73
+ - Head Dimension: 256
74
+ - Rotary Position Embedding Dimension: 64
75
+ - Feed Forward Network:
76
+ - Intermediate Dimension: 17,408
77
+ - LM Output: 248,320 (Padded)
78
+ - MTP (Multi-Token Prediction): trained with multiple steps
79
+ - Context Length: 262,144 natively and extensible up to 1,000,000 tokens.
80
+
81
+ ## Best Practices
82
+
83
+ To achieve optimal performance, we recommend the following settings:
84
+
85
+ 1. **Sampling Parameters**: We suggest using the following sets of sampling parameters:
86
+
87
+ - Thinking Mode: `temperature=1.0`, `top_p=0.95`, `top_k=20`, `min_p=0.0`, `presence_penalty=0.0`, `repetition_penalty=1.0`
88
+ - Instruct (or non-thinking) mode: `temperature=0.7`, `top_p=0.80`, `top_k=20`, `min_p=0.0`, `presence_penalty=1.5`, `repetition_penalty=1.0`
89
+
90
+ For supported frameworks, you can adjust the `presence_penalty` parameter between 0 and 2 to reduce endless repetition. However, using a higher value may occasionally result in language mixing and a slight decrease in model performance.
91
+
92
+ 2. **Adequate Output Length**: To optimize performance on agentic tasks, we recommend allocating sufficient output length to allow the model to generate detailed and comprehensive responses. For frameworks that support separate token limits for internal reasoning and final outputs, we suggest the following configuration within the 1M context length:
93
+
94
+ - Reasoning Content: Set the maximum output length to 262,144 tokens.
95
+ - Final Response: Set the maximum output length to 131,072 tokens.
96
+
97
+ These settings provide the necessary capacity for complex reasoning while ensuring ample space for high-quality final deliverables.
98
+
99
+ 3. **Processing Ultra-Long Texts**: Qwen3.8-27B natively supports context lengths of up to 262,144 tokens. For long-horizon tasks where the total length (including both input and output) exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively, e.g., YaRN.
100
+
101
+ 4. **Long Video Understanding**: To optimize inference efficiency for plain text and images, the `size` parameter in the released `video_preprocessor_config.json` is conservatively configured. It is recommended to set the `longest_edge` parameter in the video_preprocessor_config file to 469,762,048 (corresponding to 224k video tokens) to enable higher frame-rate sampling for hour-scale videos and thereby achieve superior performance. For example,
102
+ ```json
103
+ {"longest_edge": 469762048, "shortest_edge": 4096}
104
+ ```
105
+
106
+ ## Citation
107
+
108
+ If you find our work helpful, feel free to give us a cite.
109
+
110
+
111
+ ```bibtex
112
+ @misc{qwen38,
113
+ title = {{Qwen3.8-Max}: A New Bar for Coding and Cowork},
114
+ url = {https://qwen.ai/blog?id=qwen3.8},
115
+ author = {{Qwen Team}},
116
+ month = {August},
117
+ year = {2026}
118
+ }
119
+ ```