Hugging Face's logo

hotdogs
/
Qwen35-Agent-R2A103

Text Generation
Transformers
Safetensors
GGUF
English
Thai
qwen3_5_moe
image-text-to-text
qwen
Mixture of Experts
mixture-of-experts
agent
agent-world
tool-use
tool-calling
reasoning
agents-a1
model-soup
weight-averaging
conversational
Model card Files
xet
 Community
hotdogs commited on
Commit
e9ad2e4
·
verified ·
1 Parent(s): 997862f

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +224 -0
README.md ADDED
@@ -0,0 +1,224 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ - th
6
+ tags:
7
+ - qwen
8
+ - moe
9
+ - mixture-of-experts
10
+ - agent
11
+ - agent-world
12
+ - tool-use
13
+ - tool-calling
14
+ - reasoning
15
+ - agents-a1
16
+ - model-soup
17
+ - weight-averaging
18
+ - transformers
19
+ - text-generation
20
+ base_model:
21
+ - hotdogs/Qwen35B-Agent-R2
22
+ - InternScience/Agents-A1
23
+ library_name: transformers
24
+ pipeline_tag: text-generation
25
+ ---
26
+
27
+ <p align="center">
28
+ <img src="https://img.shields.io/badge/license-Apache--2.0-green">
29
+ <img src="https://img.shields.io/badge/Qwen3.5-35B%20A3B-blue">
30
+ <img src="https://img.shields.io/badge/MoE-256%20experts-orange">
31
+ <img src="https://img.shields.io/badge/Model_Soup-0.7%20R2%20%2B%200.3%20Agents--A1-ff69b4">
32
+ <img src="https://img.shields.io/badge/R2A103-purple">
33
+ </p>
34
+
35
+ <p align="center"><b>🚀 Qwen35-Agent-R2A103 — R2 + Agents-A1 Model Soup (0.7 : 0.3)</b></p>
36
+
37
+ <p align="center"><i>Combining the agentic reasoning of R2 with the multi-domain agent capabilities of Agents-A1.</i></p>
38
+
39
+ ---
40
+
41
+ ## 🧬 How This Model Was Built
42
+
43
+ ```
44
+ ┌──────────────────────────────────────────────────────────────────┐
45
+ │ Qwen35-Agent-R2A103 Construction │
46
+ ├──────────────────────────────────────────────────────────────────┤
47
+ │ │
48
+ │ ┌─────────────────────┐ ┌──────────────────────────┐ │
49
+ │ │ Qwen35B-Agent-R2 │ │ InternScience/Agents-A1│ │
50
+ │ │ (7 LoRAs fused) │ │ (Multi-teacher distilled)│ │
51
+ │ │ - Opus | Fable │ │ - Tool Use | Reasoning │ │
52
+ │ │ - Tool | Routing │ │ - Search | Engineering │ │
53
+ │ │ - Math | Mythos │ │ - Scientific | Instruct │ │
54
+ │ │ - ToolFmt │ │ - Full-domain SFT │ │
55
+ │ └─────────┬───────────┘ └────────────┬──────────────┘ │
56
+ │ │ │ │
57
+ │ └────────── Model Soup ──────────┘ │
58
+ │ │ 0.7 : 0.3 │
59
+ │ ▼ │
60
+ │ ┌──────────────────────┐ │
61
+ │ │ Qwen35-Agent-R2A103 │ │
62
+ │ │ 31,666 tensors │ │
63
+ │ │ 70.2 GB │ │
64
+ │ └──────────────────────┘ │
65
+ │ │ │
66
+ │ ▼ │
67
+ │ ┌──────────────────────┐ │
68
+ │ │ GGUF Quantization │ │
69
+ │ ├──────────────────────┤ │
70
+ │ │ f16 → 65 GB │ │
71
+ │ │ Q4_K_M → 20 GB │ │
72
+ │ │ Q6_K → 27 GB │ │
73
+ │ └──────────────────────┘ │
74
+ │ │
75
+ └──────────────────────────────────────────────────────────────────┘
76
+ ```
77
+
78
+ ### Model Soup (Weight Averaging)
79
+
80
+ Each corresponding weight tensor in the two models is blended linearly:
81
+
82
+ ```
83
+ W_R2A103 = 0.7 × W_R2 + 0.3 × W_Agents-A1
84
+ ```
85
+
86
+ This preserves the **agentic reasoning and tool-use capabilities** of R2 while incorporating the **broader multi-domain agent skills** (long-horizon search, engineering, scientific research) from Agents-A1.
87
+
88
+ ### Architecture Compatibility
89
+
90
+ Both models share the **same `qwen3_5_moe` architecture**:
91
+
92
+ | Property | Value |
93
+ |:---------|:------|
94
+ | Architecture | Qwen3.5 MoE |
95
+ | Hidden size | 2048 |
96
+ | Layers | 40 |
97
+ | Attention heads | 16 |
98
+ | KV heads | 2 |
99
+ | Experts | 256 (8 active per token) |
100
+ | Shared experts | 1 |
101
+ | Vocab size | 248,320 |
102
+ | Context length | 32,768 |
103
+
104
+ ---
105
+
106
+ ## 📦 Files
107
+
108
+ | File | Size | Format |
109
+ |:----|:----:|:-------|
110
+ | Safetensors (14 shards) | 70 GB | Transformers |
111
+ | `GGUF/Qwen35-Agent-R2A103.f16.gguf` | 65 GB | GGUF f16 |
112
+ | `GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf` | 20 GB | GGUF Q4_K_M |
113
+ | `GGUF/Qwen35-Agent-R2A103.Q6_K.gguf` | 27 GB | GGUF Q6_K |
114
+
115
+ ---
116
+
117
+ ## 🚀 Usage
118
+
119
+ ### With Transformers
120
+
121
+ ```python
122
+ from transformers import AutoModelForCausalLM, AutoTokenizer
123
+
124
+ model = AutoModelForCausalLM.from_pretrained(
125
+ "hotdogs/Qwen35-Agent-R2A103",
126
+ device_map="auto",
127
+ trust_remote_code=True,
128
+ torch_dtype="auto",
129
+ )
130
+ tokenizer = AutoTokenizer.from_pretrained("hotdogs/Qwen35-Agent-R2A103")
131
+
132
+ messages = [{"role": "user", "content": "What is the capital of Thailand?"}]
133
+ inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
134
+ outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
135
+ print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
136
+ ```
137
+
138
+ ### With llama.cpp (GGUF)
139
+
140
+ ```bash
141
+ # Q4_K_M (recommended - best size/speed/quality balance)
142
+ llama-cli \
143
+ -m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \
144
+ -n 256 -p "What is the capital of Thailand?" --temp 0.6 -ngl 99
145
+
146
+ # Or run as server:
147
+ llama-server \
148
+ -m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \
149
+ --port 8080 --host 0.0.0.0 -ngl 99 -c 4096
150
+ ```
151
+
152
+ ### With Ollama
153
+
154
+ ```bash
155
+ ollama create qwen35-r2a103 -f Modelfile
156
+ ollama run qwen35-r2a103
157
+ ```
158
+
159
+ **Modelfile:**
160
+ ```dockerfile
161
+ FROM ./GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf
162
+
163
+ PARAMETER temperature 0.6
164
+ PARAMETER top_k 40
165
+ PARAMETER top_p 0.9
166
+ PARAMETER min_p 0.05
167
+ PARAMETER repeat_penalty 1.03
168
+
169
+ TEMPLATE "{{ if .System }}<|im_start|>system
170
+ {{ .System }}<|im_end|>
171
+ {{ end }}<|im_start|>user
172
+ {{ .Prompt }}<|im_end|>
173
+ <|im_start|>assistant
174
+ "
175
+ ```
176
+
177
+ ---
178
+
179
+ ## 🧠 Capabilities
180
+
181
+ This model inherits skills from both parents:
182
+
183
+ | Skill | From R2 | From Agents-A1 |
184
+ |:------|:-------:|:--------------:|
185
+ | ✅ Tool calling | ✓ | ✓ |
186
+ | ✅ Multi-step reasoning | ✓ | ✓ |
187
+ | ✅ Instruction following | ✓ | ✓ |
188
+ | ✅ Code generation | ✓ | ✓ |
189
+ | ✅ Thai language | ✓ | ✓ |
190
+ | ✅ Long-horizon search | - | ✓ |
191
+ | ✅ Engineering tasks | - | ✓ |
192
+ | ✅ Scientific reasoning | - | ✓ |
193
+ | ✅ Vision (multimodal) | - | (via separate mmproj) |
194
+
195
+ ---
196
+
197
+ ## 📊 Performance
198
+
199
+ | Format | Size | BPW | Notes |
200
+ |:-------|:----:|:---:|:------|
201
+ | f16 | 65 GB | 16.0 | Full precision reference |
202
+ | Q6_K | 27 GB | 6.58 | High quality, fast |
203
+ | **Q4_K_M** | **20 GB** | **4.88** | **Recommended** |
204
+ | Q4_K_M inference | 20 GB | — | ~110 t/s on 7×RTX 3090 |
205
+
206
+ Benchmarked on 7× NVIDIA RTX 3090 with llama.cpp:
207
+
208
+ - **Prompt processing:** 41.7 t/s (11 tokens)
209
+ - **Token generation:** 92.1–110 t/s
210
+
211
+ ---
212
+
213
+ ## 🔗 References
214
+
215
+ - **R2 Base:** [hotdogs/Qwen35B-Agent-R2](https://huggingface.co/hotdogs/Qwen35B-Agent-R2)
216
+ - **Agents-A1:** [InternScience/Agents-A1](https://huggingface.co/InternScience/Agents-A1)
217
+ - **Qwen3.5 MoE:** [Qwen/Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B)
218
+ - **GGUF:** [llama.cpp](https://github.com/ggml-org/llama.cpp)
219
+
220
+ ---
221
+
222
+ ## 📄 License
223
+
224
+ Apache 2.0