Text Generation
Transformers
Safetensors
Chinese
English
qwen3
conversational
text-generation-inference
Instructions to use tencent/TCAndon-Router with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/TCAndon-Router with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="tencent/TCAndon-Router") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("tencent/TCAndon-Router") model = AutoModelForCausalLM.from_pretrained("tencent/TCAndon-Router") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps
- vLLM
How to use tencent/TCAndon-Router with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/TCAndon-Router" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/TCAndon-Router", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/tencent/TCAndon-Router
- SGLang
How to use tencent/TCAndon-Router with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/TCAndon-Router" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/TCAndon-Router", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/TCAndon-Router" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/TCAndon-Router", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use tencent/TCAndon-Router with Docker Model Runner:
docker model run hf.co/tencent/TCAndon-Router
Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TCAndon-Router
|
| 2 |
+
|
| 3 |
+
<p align="center">
|
| 4 |
+
<img src="https://raw.githubusercontent.com/Tencent/TCAndon-Router/refs/heads/main/assets/router.png" width="500"/>
|
| 5 |
+
</p>
|
| 6 |
+
|
| 7 |
+
<p align="center">
|
| 8 |
+
<a href="https://github.com/Tencent/TCAndon-Router">Github</a> | 📑 <a href="https://arxiv.org/abs/TCAndonRouter">Paper</a>
|
| 9 |
+
</p>
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
## 🌟 Introduction
|
| 13 |
+
|
| 14 |
+
In multi-agent systems, the ability to select the appropriate agent(s) to handle a user query is a key determinant of overall system performance.
|
| 15 |
+
|
| 16 |
+
TCAndonRouter is a reasoning-centric multi-intent routing module whose primary role is to perform agent routing in multi-agent systems.
|
| 17 |
+
Beyond agent routing, TCAndonRouter can be applied to any intent-routing scenario, including agent skill selection.
|
| 18 |
+
|
| 19 |
+
The main advantages of TCAndonRouter include:
|
| 20 |
+
|
| 21 |
+
+ Designed specifically for real-world enterprise applications
|
| 22 |
+
+ Supports dynamic onboarding of new agents (intents) New agents can be added simply by appending their descriptions, without retraining
|
| 23 |
+
+ Provides transparent and interpretable routing decisions, improving explainability, robustness, and cross-domain generalization, and making post-deployment bad-case analysis easier
|
| 24 |
+
+ Effectively resolves agent conflicts caused by overlapping responsibilities, leading to higher-quality final responses. When multiple agents are applicable, TCAndonRouter preserves all relevant agents. Each downstream agent generates its own response, and a Refining Agent subsequently merges these outputs into a single final answer
|
| 25 |
+
|
| 26 |
+
TCAndonRouter is trained using Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (DAPO), and achieves state-of-the-art performance on large-scale, real-world enterprise datasets, including HWU64, MINDS14, SGD, and the Tencent Cloud ITSM dataset(QCloud).
|
| 27 |
+
|
| 28 |
+
| **Models** | **CLINC150** | **HWU64** | **MINDS14** | **SGD** | **QCloud** |
|
| 29 |
+
|------------------------|--------------|-----------|-------------|-----------|-----------------|
|
| 30 |
+
| GPT-5.1 | 93.84 | 85.59 | 95.59 | 73.90 | 92.80/93.06 |
|
| 31 |
+
| Claude-Sonnet-4.5 | **94.21** | 87.40 | 96.20 | 76.02 | 88.82/94.25 |
|
| 32 |
+
| DeepSeek-v3.1-terminus | 88.29 | 88.10 | 95.72 | 79.70 | 94.09/91.89 |
|
| 33 |
+
| ArcRouter | 62.98 | 69.33 | 91.79 | 65.59 | - |
|
| 34 |
+
| Qwen3-Embedding-4B | 57.21 | 54.27 | 94.12 | 37.02 | - |
|
| 35 |
+
| Qwen3-4B-Instruct-2507 | 70.12 | 80.29 | 90.08 | 58.74 | 82.23/79.44 |
|
| 36 |
+
| **TCAndonRouter** | 91.25 | **91.63** | **96.70** | **91.58** | **95.21/92.78** |
|
| 37 |
+
|
| 38 |
+
## 🔧 How to use
|
| 39 |
+
|
| 40 |
+
Please refer to [GitHub](https://github.com/Tencent/TCAndon-Router) for code usage.
|
| 41 |
+
|
| 42 |
+
```python
|
| 43 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 44 |
+
from prompt import router_prompt
|
| 45 |
+
from utils import load_config
|
| 46 |
+
|
| 47 |
+
tokenizer = AutoTokenizer.from_pretrained("tencent/TCAndon-Router")
|
| 48 |
+
model = AutoModelForCausalLM.from_pretrained("tencent/TCAndon-Router")
|
| 49 |
+
|
| 50 |
+
agents = load_config('config/hwu64_config.xml')
|
| 51 |
+
query = "Can you recommend any pub in mg road"
|
| 52 |
+
|
| 53 |
+
prompt = router_prompt.format(agents=agents) + 'user:' + query
|
| 54 |
+
response = model.generate(prompt)
|
| 55 |
+
print(response)
|
| 56 |
+
```
|
| 57 |
+
### Generate Agent Descriptions
|
| 58 |
+
|
| 59 |
+
If you want to use TCAndonRouter on your own dataset, you need to provide agent descriptions. The required format is defined in `config/xxx_config.xml`.
|
| 60 |
+
You can generate agent descriptions using an LLM via generate_agent_desc.py, or write them manually.
|
| 61 |
+
```shell
|
| 62 |
+
python generate_agent_desc.py --dataset hwu64 --limit 50
|
| 63 |
+
```
|
| 64 |
+
|
| 65 |
+
## 🤝 Citation
|
| 66 |
+
|
| 67 |
+
If you use TCAndonRouter in your work, please cite our paper:
|
| 68 |
+
|
| 69 |
+
```
|
| 70 |
+
@article{zhao2025TCAndonRouter,
|
| 71 |
+
title={TCAndonRouter: Adaptive Reasoning Router for Multi-Agent Collaboration},
|
| 72 |
+
author={Jiuzhou Zhao et al.},
|
| 73 |
+
journal={arXiv},
|
| 74 |
+
year={2025}
|
| 75 |
+
}
|
| 76 |
+
```
|