README.md

metadata

license: agpl-3.0
language:
  - en
  - th
tags:
  - qwen
  - agent
  - tool-call
  - tool-use
  - function-calling
  - reasoning
  - opus
  - fable
  - conversational
  - mtp
  - multi-token-prediction
  - transformers
  - text-generation
  - thai
  - speculative-decoding
  - preview
base_model:
  - Qwen/Qwen3.6-27B
datasets:
  - hotdogs/uka-fable-reasoning
  - NousResearch/hermes-function-calling-v1
  - 11-47/claude_opus_4.8_max_thinking_5k_v2
library_name: transformers
pipeline_tag: text-generation

🐉 qwen27b-agent-R2-preview

27B Agent Model — MTP · Tool-Calling · Multi-LoRA Fusion

Preview release — Built on Qwen3.6-27B with multi-LoRA fusion. Features Multi-Token Prediction (MTP) for speculative decoding, tool-calling, and Opus + Fable reasoning. Standard (non-abliterated) version.

---

✨ Key Features

Capability	Description
⚡ MTP Speculative Decoding	Draft 2 tokens at a time — up to +85% decode TPS on single GPU
🔧 Tool Calling	Hermes/Qwen function-calling format via llama.cpp --tools all
🧠 Reasoning	Opus 4.8 + Fable-style reasoning with step-by-step CoT
🌏 Thai + English	Native bilingual support
💻 Code	Python, shell, system tasks

---

🚀 Usage

llama.cpp (Recommended)

# Quick test
./llama-cli -m qwen27b-agent-R2-preview.Q4_K_M.gguf \
  -p "Hello" -n 100 --temp 0.6

# Full agent server with tool calling + MTP speculative decoding
./llama-server \
  -m qwen27b-agent-R2-preview.Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8081 \
  -c 262144 \
  -ngl 99 \
  --cache-type-k bf16 \
  --cache-type-v bf16 \
  --flash-attn on \
  --tools all \
  --cont-batching \
  --temp 0.6 \
  --top-k 40 \
  --top-p 0.9 \
  --min-p 0.05 \
  --repeat-penalty 1.03 \
  --dry-multiplier 0 \
  --verbose \
  -n -1 \
  --parallel 1 \
  --jinja \
  --dry-sequence-breaker none \
  --spec-type draft-mtp \
  --spec-draft-n-max 2

Parameter	Purpose
--cache-type-k bf16 / --cache-type-v bf16	BF16 KV cache for quality
--flash-attn on	Flash attention for speed
--tools all	Enable tool/function calling
--spec-type draft-mtp	MTP speculative decoding (draft 2 tokens)
--spec-draft-n-max 2	Max 2 draft tokens per step
--cont-batching	Continuous batching for multi-turn
--jinja	Use Jinja2 chat template from GGUF

Python (Transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "hotdogs/qwen27b-agent-R2-preview",
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/qwen27b-agent-R2-preview")

messages = [{"role": "user", "content": "Hello"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))

---

📦 Downloads

File	Size	Quant	Description
qwen27b-agent-R2-preview.Q4_K_M.gguf	16 GB	Q4_K_M	Recommended — balanced quality/speed
qwen27b-agent-R2-preview.Q6_K.gguf	21 GB	Q6_K	Higher quality, slightly slower
qwen27b-agent-R2-preview.f16.gguf	51 GB	f16	Full precision

🎯 Q4_K_M is recommended for most users — good quality with 16 GB VRAM usage.

📷 Multimodal Projector (mmproj)

For vision support, pair this model with the mmproj from Qwen/Qwen3.6-27B:

# Extract mmproj from Qwen3.6-27B vision model
python3 ./llama.cpp/convert_hf_to_gguf.py \
  --mmproj Qwen/Qwen3.6-27B \
  --outfile mmproj-qwen3.6-27b.gguf

# Use with llama-server for vision + tool calling
./llama-server \
  -m qwen27b-agent-R2-preview.Q4_K_M.gguf \
  --mmproj mmproj-qwen3.6-27b.gguf

---

🧬 Architecture

Parameter	Value
Base	Qwen/Qwen3.6-27B
Parameters	~27B
Hidden Size	5,120
Attention	Linear + Standard hybrid
Context	8,192 tokens (extendable)
Precision	BF16 / GGUF quantized
Format	ChatML (Jinja2 template)
MTP Head	✅ 1 extra layer (draft 2 tokens)

---

🧬 How This Model Was Built

┌──────────────────────────────────────────────────────────────┐
│              qwen27b-agent-R2-preview Construction            │
├──────────────────────────────────────────────────────────────┤
│                                                              │
│  Qwen/Qwen3.6-27B (Base)                                    │
│       │                                                     │
│       ├── Multi-LoRA Fusion (4 LoRAs):                      │
│       │   ┌──────────────────────────────────┐             │
│       │   │ LoRa        │ Source      │             │
│       │   ├──────────────┼─────────────┤             │
│       │   │ Opus SFT     │ SFT on      │             │
│       │   │              │ Opus 4.8    │             │
│       │   ├──────────────┼─────────────┤             │
│       │   │ CxCMU Agent  │ AgentWorld   │             │
│       │   │              │ trajectories │             │
│       │   ├──────────────┼─────────────┤             │
│       │   │ General SFT  │ Reasoning    │             │
│       │   │              │ + Hermes FC  │             │
│       │   ├──────────────┼─────────────┤             │
│       │   │ Tachibana    │ Coding agent │             │
│       │   │ Agent        │ dataset      │             │
│       │   └──────────────┴─────────────┘             │
│       │                                                     │
│       └── + MTP Head (15 tensors)                           │
│           └── From huihui-ai/Huihui-Qwen3.6-27B-abliterated  │
│                                                              │
│  Result: 866 tensors, MTP=1, 16.8 GB (Q4_K_M)               │
│                                                              │
└──────────────────────────────────────────────────────────────┘

Training Details

Each LoRA was trained independently via SFT on specialized datasets. Then all 4 were fused into the base model at their respective scales. The low scales (0.15-0.4) ensure no single LoRA overpowers the model — creating a balanced agent.

MTP Tensor Injection

The Multi-Token Prediction head (15 tensors) was injected from huihui-ai/Huihui-Qwen3.6-27B-abliterated to enable speculative decoding:

# MTP head adds blk.64.* tensors for draft-2-token prediction
--spec-type draft-mtp  # Enables in llama.cpp
--spec-draft-n-max 2   # Max draft tokens

---

⚡ MTP Speculative Decoding

Multi-Token Prediction enables speculative decoding:

Standard:  [token₁] → [token₂] → [token₃] → ...  (~36 TPS)
MTP:       [token₁ token₂] → [token₃ token₄] → ...  (~66 TPS)

• MTP head adds ~849 MB to model size
• Uses --spec-type draft-mtp in llama.cpp
• Best for single-user agent workloads
• ~1.2–1.8× decode speedup

---

💖 Support / โปรดสนับสนุน

If you find this model useful, please consider supporting my work!
หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏

₿ Bitcoin — BTC:

bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v

Thank you for your support! 🙏✨
ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗

---

🙏 Acknowledgements / ขอบคุณ

• Qwen Team (Alibaba) — For the Qwen3.6 architecture
• Nous Research — For Hermes Agent framework
• huihui-ai — For MTP tensor support
• All dataset contributors and the open-source AI community ❤️

---

Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert