metadata
license: apache-2.0
language:
- en
- th
tags:
- qwen
- moe
- mixture-of-experts
- agent
- agent-world
- tool-use
- tool-calling
- reasoning
- agents-a1
- model-soup
- weight-averaging
- transformers
- text-generation
base_model:
- hotdogs/Qwen35B-Agent-R2
library_name: transformers
pipeline_tag: text-generation
🚀 Qwen35-Agent-R2A103 — R2 + Agents-A1 Model Soup (0.7 : 0.3)
Building on hotdogs/Qwen35B-Agent-R2 as the base, blended with insights from InternScience/Agents-A1 via model soup (0.7 : 0.3).
🧬 How This Model Was Built
┌──────────────────────────────────────────────────────────────────┐
│ Qwen35-Agent-R2A103 Construction │
├──────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌──────────────────────────┐ │
│ │ Qwen35B-Agent-R2 │ │ InternScience/Agents-A1│ │
│ │ (7 LoRAs fused) │ │ (Multi-teacher distilled)│ │
│ │ - Opus | Fable │ │ - Tool Use | Reasoning │ │
│ │ - Tool | Routing │ │ - Search | Engineering │ │
│ │ - Math | Mythos │ │ - Scientific | Instruct │ │
│ │ - ToolFmt │ │ - Full-domain SFT │ │
│ └─────────┬───────────┘ └────────────┬──────────────┘ │
│ │ │ │
│ └────────── Model Soup ──────────┘ │
│ │ 0.7 : 0.3 │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ Qwen35-Agent-R2A103 │ │
│ │ 31,666 tensors │ │
│ │ 70.2 GB │ │
│ └──────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────┐ │
│ │ GGUF Quantization │ │
│ ├──────────────────────┤ │
│ │ f16 → 65 GB │ │
│ │ Q4_K_M → 20 GB │ │
│ │ Q6_K → 27 GB │ │
│ └──────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────┘
Model Soup (Weight Averaging)
Building on hotdogs/Qwen35B-Agent-R2 as the base, we blend corresponding weight tensors with InternScience/Agents-A1:
W_R2A103 = 0.7 × W_R2 + 0.3 × W_Agents-A1
This preserves the agentic reasoning and tool-use capabilities of R2 while incorporating the broader multi-domain agent skills (long-horizon search, engineering, scientific research) from Agents-A1.
Architecture Compatibility
Both models share the same qwen3_5_moe architecture:
| Property | Value |
|---|---|
| Architecture | Qwen3.5 MoE |
| Hidden size | 2048 |
| Layers | 40 |
| Attention heads | 16 |
| KV heads | 2 |
| Experts | 256 (8 active per token) |
| Shared experts | 1 |
| Vocab size | 248,320 |
| Context length | 32,768 |
📦 Files
| File | Size | Format |
|---|---|---|
| Safetensors (14 shards) | 70 GB | Transformers |
GGUF/Qwen35-Agent-R2A103.f16.gguf |
65 GB | GGUF f16 |
GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf |
20 GB | GGUF Q4_K_M |
GGUF/Qwen35-Agent-R2A103.Q6_K.gguf |
27 GB | GGUF Q6_K |
🚀 Usage
With Transformers
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"hotdogs/Qwen35-Agent-R2A103",
device_map="auto",
trust_remote_code=True,
torch_dtype="auto",
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/Qwen35-Agent-R2A103")
messages = [{"role": "user", "content": "What is the capital of Thailand?"}]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
With llama.cpp (GGUF)
# Q4_K_M (recommended - best size/speed/quality balance)
llama-cli \
-m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \
-n 256 -p "What is the capital of Thailand?" --temp 0.6 -ngl 99
# Or run as server:
llama-server \
-m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \
--port 8080 --host 0.0.0.0 -ngl 99 -c 4096
With Ollama
ollama create qwen35-r2a103 -f Modelfile
ollama run qwen35-r2a103
Modelfile:
FROM ./GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf
PARAMETER temperature 0.6
PARAMETER top_k 40
PARAMETER top_p 0.9
PARAMETER min_p 0.05
PARAMETER repeat_penalty 1.03
TEMPLATE "{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"
🧠 Capabilities
This model inherits skills from both parents:
| Skill | From R2 | From Agents-A1 |
|---|---|---|
| ✅ Tool calling | ✓ | ✓ |
| ✅ Multi-step reasoning | ✓ | ✓ |
| ✅ Instruction following | ✓ | ✓ |
| ✅ Code generation | ✓ | ✓ |
| ✅ Thai language | ✓ | ✓ |
| ✅ Long-horizon search | - | ✓ |
| ✅ Engineering tasks | - | ✓ |
| ✅ Scientific reasoning | - | ✓ |
| ✅ Vision (multimodal) | - | (via separate mmproj) |
📊 Performance
| Format | Size | BPW | Notes |
|---|---|---|---|
| f16 | 65 GB | 16.0 | Full precision reference |
| Q6_K | 27 GB | 6.58 | High quality, fast |
| Q4_K_M | 20 GB | 4.88 | Recommended |
| Q4_K_M inference | 20 GB | — | ~110 t/s on 7×RTX 3090 |
Benchmarked on 7× NVIDIA RTX 3090 with llama.cpp:
- Prompt processing: 41.7 t/s (11 tokens)
- Token generation: 92.1–110 t/s
🔗 References
- R2 Base: hotdogs/Qwen35B-Agent-R2
- Agents-A1: InternScience/Agents-A1
- Qwen3.5 MoE: Qwen/Qwen-AgentWorld-35B-A3B
- GGUF: llama.cpp
🙏 Credits
- hotdogs — Qwen35B-Agent-R2 as the base model
- InternScience / Agents-A1 — Multi-domain agent capabilities (tool-use, search, engineering, scientific reasoning, instruction following). Check out their paper
- Qwen Team (Alibaba) — Qwen3.5 MoE architecture
- llama.cpp — GGUF conversion and inference framework
📄 License
Apache 2.0