🚀 Qwen35-Agent-R2A103 — R2 + Agents-A1 Model Soup (0.7 : 0.3)
Building on hotdogs/Qwen35B-Agent-R2 as the base, blended with insights from InternScience/Agents-A1 via model soup (0.7 : 0.3).
--- ## 🧬 How This Model Was Built ``` ┌──────────────────────────────────────────────────────────────────┐ │ Qwen35-Agent-R2A103 Construction │ ├──────────────────────────────────────────────────────────────────┤ │ │ │ ┌─────────────────────┐ ┌──────────────────────────┐ │ │ │ Qwen35B-Agent-R2 │ │ InternScience/Agents-A1│ │ │ │ (7 LoRAs fused) │ │ (Multi-teacher distilled)│ │ │ │ - Opus | Fable │ │ - Tool Use | Reasoning │ │ │ │ - Tool | Routing │ │ - Search | Engineering │ │ │ │ - Math | Mythos │ │ - Scientific | Instruct │ │ │ │ - ToolFmt │ │ - Full-domain SFT │ │ │ └─────────┬───────────┘ └────────────┬──────────────┘ │ │ │ │ │ │ └────────── Model Soup ──────────┘ │ │ │ 0.7 : 0.3 │ │ ▼ │ │ ┌──────────────────────┐ │ │ │ Qwen35-Agent-R2A103 │ │ │ │ 31,666 tensors │ │ │ │ 70.2 GB │ │ │ └──────────────────────┘ │ │ │ │ │ ▼ │ │ ┌──────────────────────┐ │ │ │ GGUF Quantization │ │ │ ├──────────────────────┤ │ │ │ f16 → 65 GB │ │ │ │ Q4_K_M → 20 GB │ │ │ │ Q6_K → 27 GB │ │ │ └──────────────────────┘ │ │ │ └──────────────────────────────────────────────────────────────────┘ ``` ### Model Soup (Weight Averaging) Building on **hotdogs/Qwen35B-Agent-R2** as the base, we blend corresponding weight tensors with **InternScience/Agents-A1**: ``` W_R2A103 = 0.7 × W_R2 + 0.3 × W_Agents-A1 ``` This preserves the **agentic reasoning and tool-use capabilities** of R2 while incorporating the **broader multi-domain agent skills** (long-horizon search, engineering, scientific research) from Agents-A1. ### Architecture Compatibility Both models share the **same `qwen3_5_moe` architecture**: | Property | Value | |:---------|:------| | Architecture | Qwen3.5 MoE | | Hidden size | 2048 | | Layers | 40 | | Attention heads | 16 | | KV heads | 2 | | Experts | 256 (8 active per token) | | Shared experts | 1 | | Vocab size | 248,320 | | Context length | 32,768 | --- ## 📦 Files | File | Size | Format | |:----|:----:|:-------| | Safetensors (14 shards) | 70 GB | Transformers | | `GGUF/Qwen35-Agent-R2A103.f16.gguf` | 65 GB | GGUF f16 | | `GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf` | 20 GB | GGUF Q4_K_M | | `GGUF/Qwen35-Agent-R2A103.Q6_K.gguf` | 27 GB | GGUF Q6_K | --- ## 🚀 Usage ### With Transformers ```python from transformers import AutoModelForCausalLM, AutoTokenizer model = AutoModelForCausalLM.from_pretrained( "hotdogs/Qwen35-Agent-R2A103", device_map="auto", trust_remote_code=True, torch_dtype="auto", ) tokenizer = AutoTokenizer.from_pretrained("hotdogs/Qwen35-Agent-R2A103") messages = [{"role": "user", "content": "What is the capital of Thailand?"}] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device) outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6) print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)) ``` ### With llama.cpp (GGUF) ```bash # Q4_K_M (recommended - best size/speed/quality balance) llama-cli \ -m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \ -n 256 -p "What is the capital of Thailand?" --temp 0.6 -ngl 99 # Or run as server: llama-server \ -m GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf \ --port 8080 --host 0.0.0.0 -ngl 99 -c 4096 ``` ### With Ollama ```bash ollama create qwen35-r2a103 -f Modelfile ollama run qwen35-r2a103 ``` **Modelfile:** ```dockerfile FROM ./GGUF/Qwen35-Agent-R2A103.Q4_K_M.gguf PARAMETER temperature 0.6 PARAMETER top_k 40 PARAMETER top_p 0.9 PARAMETER min_p 0.05 PARAMETER repeat_penalty 1.03 TEMPLATE "{{ if .System }}<|im_start|>system {{ .System }}<|im_end|> {{ end }}<|im_start|>user {{ .Prompt }}<|im_end|> <|im_start|>assistant " ``` --- ## 🧠 Capabilities This model inherits skills from both parents: | Skill | From R2 | From Agents-A1 | |:------|:-------:|:--------------:| | ✅ Tool calling | ✓ | ✓ | | ✅ Multi-step reasoning | ✓ | ✓ | | ✅ Instruction following | ✓ | ✓ | | ✅ Code generation | ✓ | ✓ | | ✅ Thai language | ✓ | ✓ | | ✅ Long-horizon search | - | ✓ | | ✅ Engineering tasks | - | ✓ | | ✅ Scientific reasoning | - | ✓ | | ✅ Vision (multimodal) | - | (via separate mmproj) | --- ## 📊 Performance | Format | Size | BPW | Notes | |:-------|:----:|:---:|:------| | f16 | 65 GB | 16.0 | Full precision reference | | Q6_K | 27 GB | 6.58 | High quality, fast | | **Q4_K_M** | **20 GB** | **4.88** | **Recommended** | | Q4_K_M inference | 20 GB | — | ~110 t/s on 7×RTX 3090 | Benchmarked on 7× NVIDIA RTX 3090 with llama.cpp: - **Prompt processing:** 41.7 t/s (11 tokens) - **Token generation:** 92.1–110 t/s --- ## 🔗 References - **R2 Base:** [hotdogs/Qwen35B-Agent-R2](https://huggingface.co/hotdogs/Qwen35B-Agent-R2) - **Agents-A1:** [InternScience/Agents-A1](https://huggingface.co/InternScience/Agents-A1) - **Qwen3.5 MoE:** [Qwen/Qwen-AgentWorld-35B-A3B](https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B) - **GGUF:** [llama.cpp](https://github.com/ggml-org/llama.cpp) --- ## 🙏 Credits - **[hotdogs](https://huggingface.co/hotdogs)** — Qwen35B-Agent-R2 as the base model - **[InternScience / Agents-A1](https://huggingface.co/InternScience/Agents-A1)** — Multi-domain agent capabilities (tool-use, search, engineering, scientific reasoning, instruction following). Check out their [paper](https://arxiv.org/abs/2606.30616) - **[Qwen Team (Alibaba)](https://huggingface.co/Qwen)** — Qwen3.5 MoE architecture - **llama.cpp** — GGUF conversion and inference framework --- ## 📄 License Apache 2.0