Update README.md

README.md

@@ -1,36 +1,292 @@
-# SimpleTool: Parallel Decoding for Real-Time LLM Function Calling
+<p align="center">
+  <a href="README.md">English</a> | <a href="README_zh.md">中文</a>
+</p>
+<h1 align="center">SimpleTool</h1>
 
-[**Hugging Face**](https://huggingface.co/Cialtion/SimpleTool) | [**ModelScope**](https://www.modelscope.cn/models/cialtion/SimpleTool) | [**GitHub**](https://github.com/HaxxorCialtion/SimpleTool)
+<p align="center">
+  <b>Parallel Decoding for Real-Time LLM Function Calling</b>
+</p>
 
-This repository contains the weights for **RT-Qwen** (RealtimeTool), a series of models optimized for low-latency, parallel LLM function calling.
+<p align="center">
+  <a href="https://arxiv.org/abs/2603.00030"><img src="https://img.shields.io/badge/arXiv-2603.00030-red"></a>
+  <a href="https://huggingface.co/Cialtion/SimpleTool"><img src="https://img.shields.io/badge/🤗-Models-yellow"></a>
+  <a href="https://www.modelscope.cn/models/cialtion/SimpleTool"><img src="https://img.shields.io/badge/ModelScope-Models-blue"></a>
+  <a href="#demo-videos"><img src="https://img.shields.io/badge/Bilibili-Demo-00A1D6?logo=bilibili&logoColor=white"></a>
+  <a href="#demo-videos"><img src="https://img.shields.io/badge/YouTube-Demo-FF0000?logo=youtube&logoColor=white"></a>
+  <a href="#license"><img src="https://img.shields.io/badge/License-Apache%202.0-green"></a>
+</p>
 
-## 📁 Model Directory Structure
+<p align="center">
+  A 4B-parameter LLM achieving <b>16 Hz end-to-end real-time function calling</b> — fast enough to drive game AI, robotic arms, and digital humans.
+</p>
 
-The models are organized by scale, quantization format, and inference framework.
+---
+
+SimpleTool enables **real-time LLM function calling** through multi-head parallel decoding. By introducing special tokens that compress redundant structured output (4–6×) and enable independent generation of function name and arguments, we achieve **3–6× end-to-end speedup** while maintaining competitive accuracy across three application domains: **games**, **robotic control**, and **digital human animation**.
+
+<p align="center">
+  <img src="assets/fig_title_panel_a.png" alt="SimpleTool Overview" width="700">
+</p>
+
+## How It Works
+
+Traditional function calling generates tokens sequentially — `function → arg1 → arg2 → ...` — so latency scales linearly with output length. SimpleTool exploits two key observations:
+
+1. **Token Redundancy**: Structured outputs contain predictable tokens (brackets, parameter names, quotes) that can be compressed into single special tokens.
+2. **Weak Causal Dependencies**: Function arguments are largely independent of each other and can be generated in parallel.
+
+<p align="center">
+  <img src="assets/overview.png" alt="SimpleTool Architecture" width="600">
+</p>
+
+By decoding function name and arguments as parallel streams sharing the same prefix KV cache, latency drops from `sum(all_token_times)` to `max(per_head_time)`. The parallel heads utilize idle compute capacity within the memory-bandwidth-bound decode phase, making parallelization nearly free.
+
+For more details, see our [arXiv paper](https://arxiv.org/abs/2603.00030).
+
+---
+
+## Quick Start
+
+### 1. Setup Environment
+
+```bash
+git clone https://github.com/HaxxorCialtion/SimpleTool.git
+cd SimpleTool
+```
+
+**Option A — uv (recommended)**
+```bash
+uv venv env_rt -p python3.12
+source env_rt/bin/activate
+uv pip install -r requirements.txt
+```
+
+**Option B — conda**
+```bash
+conda create -n simpletool python=3.12 -y
+conda activate simpletool
+pip install -r requirements.txt
+```
+
+**Option C — pip**
+```bash
+python3.12 -m venv env_rt
+source env_rt/bin/activate
+pip install -r requirements.txt
+```
+
+### 2. Download Model
+
+The recommended default model is **RT-Qwen3-4B-AWQ-v2** (4B parameters, AWQ W4A16 quantized, v2 prompt format). All scripts default to `./models/RT-Qwen3-4B-AWQ-v2`.
+
+```bash
+# HuggingFace
+huggingface-cli download Cialtion/SimpleTool \
+  --include "RT-Qwen3-4B-AWQ-v2/*" --local-dir ./models
+
+# Or ModelScope
+modelscope download --model cialtion/SimpleTool \
+  --include "RT-Qwen3-4B-AWQ-v2/*" --local_dir ./models
+```
+
+<details>
+<summary><b>All Available Models</b></summary>
+
+| Model | Params | Latency | HuggingFace | ModelScope |
+|-------|--------|---------|-------------|------------|
+| RT-Qwen2.5-0.5B-AWQ | 0.5B | ~30ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen2.5-0.5B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen2.5-0.5B-AWQ) |
+| RT-Qwen2.5-1.5B-AWQ | 1.5B | ~40ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen2.5-1.5B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen2.5-1.5B-AWQ) |
+| RT-Qwen2.5-3B-AWQ | 3B | ~50ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen2.5-3B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen2.5-3B-AWQ) |
+| **RT-Qwen3-4B-AWQ-v2** | **4B** | **~60ms** | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen3-4B-AWQ-v2) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen3-4B-AWQ-v2) |
+| RT-Qwen3-4B-AWQ | 4B | ~60ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen3-4B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen3-4B-AWQ) |
+| RT-Qwen2.5-7B-AWQ | 7B | ~70ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen2.5-7B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen2.5-7B-AWQ) |
+| RT-Qwen2.5-14B-AWQ | 14B | ~130ms | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen2.5-14B-AWQ) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen2.5-14B-AWQ) |
+| RT-Qwen3-30B-A3B-AWQ | 30B(A3B) | ~ | [🤗](https://huggingface.co/Cialtion/SimpleTool/tree/main/RT-Qwen3-30B_awq_w4a16) | [Link](https://www.modelscope.cn/models/cialtion/SimpleTool/tree/master/RT-Qwen3-30B_awq_w4a16) |
+
+> Latency measured on RTX 4090 with vLLM prefix caching. v2 models use an improved prompt format with domain-specific system prompts; v1 models use a generic multi-head instruction header.
+
+</details>
+
+### 3. Run Benchmark (No Server Needed)
+
+`01_benchmark.py` runs multi-head parallel decoding directly via vLLM across three application domains — game AI, robotic arm control, and digital human animation — with cold start / hot prefill / decode bottleneck analysis.
+
+```bash
+# v2 model (default)
+python 01_benchmark.py --version v2
+
+# v1 model
+python 01_benchmark.py --version v1 --model ./models/RT-Qwen3-4B-AWQ
+
+# Auto-detect optimal head count per scenario
+python 01_benchmark.py --n-args auto
+```
+
+Example output:
+```
+  PARALLEL TEST (v2)
+
+─── Game — Tower Defense ───
+PASS  use_skill(Amiya)
+  function   use_skill                                     4    OK
+  arg1       Amiya                                         4    FILL
+  arg2       <|null|>                                      3    NULL
+  e2e=24.6ms  max_tok=4
+
+─── Robotic Arm — Assembly ───
+PASS  move_to(300,150,50,slow)
+  function   move_to                                       4    OK
+  arg1       300                                           5    FILL
+  arg2       150                                           5    FILL
+  arg3       500                                           5    FILL
+  arg4       slow                                          3    FILL
+  e2e=39.9ms  max_tok=5
+
+─── Digital Human — Streamer ───
+PASS  speak(welcome,cheerful)
+  function   speak                                         4    OK
+  arg1       Welcome!                                      4    FILL
+  arg2       cheerful                                      5    FILL
+  e2e=29.1ms  max_tok=5
 
-### 1. SFT & AWQ Models (vLLM / Transformers)
-Directly use these folders for inference via `vLLM` or `Transformers`.
-*   **RT-Qwen2.5-0.5B** / **-0.5B-AWQ**
-*   **RT-Qwen2.5-1.5B** / **-1.5B-AWQ**
-*   **RT-Qwen2.5-3B** / **-3B-AWQ**
-*   **RT-Qwen2.5-7B** / **-7B-AWQ**
-*   **RT-Qwen2.5-14B** / **-14B-AWQ**
-*   **RT-Qwen3-4B** / **-4B-AWQ**
-*   **RT-Qwen3-30B** / **-30B-AWQ**
+  SUMMARY (v2)
+  Accuracy       : 3/3
+  Cold start avg : 56.1ms
+  Hot prefill avg: 29.3ms
+  E2E avg (hot)  : 31.2ms
+  E2E / max_tok  : 6.7ms/tok (decode bottleneck)
+```
 
-### 2. GGUF Models (llama.cpp)
-*   **`gguf_models/`**: Full-precision (F16) GGUF files for all versions.
-*   **`gguf_quantized/`**: Quantized GGUF versions including **Q4_K_M**, **Q5_K_M**, and **Q8_0**.
+The script also prints the full prompt structure and reconstructed multi-head output for inspection.
+
+### 4. Start Server
+
+`02_server.py` wraps the engine in a FastAPI server with CORS support. HTML game clients connect to it.
+
+```bash
+python 02_server.py
+```
+
+Server starts at `http://localhost:8899` with two endpoints:
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check, model version info |
+| `/v1/function_call` | POST | Multi-head parallel function call |
+
+Edit `MODEL_PATH` and `MODEL_VERSION` at the top of `02_server.py` to switch between v1/v2 models.
+
+### 5. Test Server
+
+With the server running, test it from another terminal:
+
+```bash
+python 03_test_server.py
+```
+
+This sends the same three domain scenarios (game, robotic arm, digital human) to the server API and reports accuracy, cold/hot latency, and per-head output.
+
+```bash
+# Custom server URL
+python 03_test_server.py --url http://192.168.1.100:8899
+
+# More hot rounds
+python 03_test_server.py --rounds 10
+```
+
+### 6. Play Demos
+
+Open demo HTML files in your browser. They connect to the running SimpleTool server.
+
+| Demo | Description | File |
+|------|-------------|------|
+| **Pong** | AI vs Human paddle game | `demos/pong_game.html` |
+| **Neon Arena** | Multi-AI battle shooter | `demos/neon_arena.html` |
+
+For games with extra assets:
+```bash
+cd demos/neon_arena
+python3 -m http.server 8080 --bind 127.0.0.1
+```
+Then open http://127.0.0.1:8080/neon_arena.html and enter your SimpleTool server URL (default: `http://localhost:8899`).
+
+<p align="center">
+  <video src="https://github.com/user-attachments/assets/436e3b97-e8ab-4d36-9fa0-8f1962da4a38" autoplay loop muted width="400"></video>
+  <video src="https://github.com/user-attachments/assets/f9b127da-b65e-4a06-b48f-836e759a6029" autoplay loop muted width="400"></video>
+</p>
 
 ---
 
-## 📝 TODO
+## Project Structure
 
-- [ ] Release Arxiv Paper
-- [ ] Complete GitHub Documentation
-- [ ] Add Performance Benchmarks
-- [ ] Provide Citation Info
+```
+SimpleTool/
+├── 01_benchmark.py          # Step 1: Direct parallel decode benchmark
+├── 02_server.py             # Step 2: FastAPI vLLM server
+├── 03_test_server.py        # Step 3: Server API test client
+├── prompts/                 # External prompt & scenario files
+│   ├── v1_system.txt        #   v1 multi-head system prompt
+│   ├── scenarios.json       #   3 domain test scenarios
+│   ├── tools_game.jsonl     #   Tower defense tool definitions
+│   ├── tools_arm.jsonl      #   Robotic arm tool definitions
+│   └── tools_avatar.jsonl   #   Digital human tool definitions
+├── models/                  # Downloaded models go here
+│   └── RT-Qwen3-4B-AWQ-v2/ #   Default model
+├── demos/                   # HTML game clients
+│   ├── pong_game.html
+│   └── neon_arena/
+├── assets/                  # Figures for README
+├── requirements.txt
+├── simpletool-game.skill.md # Guide for building new games with AI
+├── README.md
+└── README_zh.md
+```
+
+## Build Your Own Game
+
+Feed **`simpletool-game.skill.md`** along with this **`README.md`** into your AI coding agent (Claude Code, Codex, Antigravity, etc.) — the skill file covers server API spec, tool definition format, query design best practices, frontend templates, and dynamic head optimization tips, while the README helps the agent understand the overall project structure. Together they provide everything needed to vibe-code a SimpleTool-powered game.
 
 ---
-**License**: Apache-2.0  
-**Status**: Models Uploading / Placeholder README
+
+## Roadmap
+
+- [ ] **World Simulation** — Large-scale (1,000+ NPCs) real-time AI world simulation with < 200ms action latency per agent
+- [ ] **Native Windows Support** — Windows game engine plugins and native runtime(no need for docker or wsl)
+- [ ] **Apple Ecosystem** — Mac and iPhone on-device deployment (CoreML / Metal)
+- [ ] **v3 Architecture** — Fast thinking (real-time SimpleTool) + slow thinking (async meta-cognition) fusion
+- [ ] **Embodied Intelligence** — Virtual 3D digital humans, large-scale game engine integration demos
+- [ ] **Open Source Training** — Full training code and dataset release
+
+---
+
+## Demo Videos
+
+<p align="center">
+  <a href="#"><img src="https://img.shields.io/badge/Bilibili-Demo-00A1D6?logo=bilibili&logoColor=white"></a>
+  <a href="#"><img src="https://img.shields.io/badge/YouTube-Demo-FF0000?logo=youtube&logoColor=white"></a>
+</p>
+
+> Video demos coming soon — showcasing real-time game AI, robotic arm control, and digital human animation.
+
+---
+
+## Citation
+
+```bibtex
+@article{shi2026simpletool,
+  title={SimpleTool: Parallel Decoding for Real-Time LLM Function Calling},
+  author={Shi, Xiaoxin and Wan, Jiaxin and Dong, Linkang and Jiang, Wei and Liu, Yue and Huang, Zengfeng},
+  journal={arXiv preprint arXiv:2603.00030},
+  year={2026}
+}
+```
+
+## Contact
+
+- **Email**: cialtion737410@sjtu.edu.cn / cialtion@outlook.com
+- **QQ Group**: 861244702
+- **Bilibili**: [Cialtion](https://space.bilibili.com/Cialtion)
+
+## License
+
+Apache 2.0