File size: 2,488 Bytes
25fbe9c 25ca46c 25fbe9c 088b259 25fbe9c dbfe01b 85607ca 25fbe9c 9defe2c 7da18c2 25fbe9c 9defe2c 7da18c2 25fbe9c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | ---
license: apache-2.0
datasets:
- Zichen1024/SAP-9k
base_model:
- Qwen/Qwen3-4B-Instruct-2507
---
# Model Card: SAP-4B
**paper:** [SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Use](https://arxiv.org/abs/2609.06124)
## Model Overview
SAP-4B is a 4B-parameter agentic language model specialized in **long-horizon multi-turn tool use and function calling**. It is trained via supervised fine-tuning (SFT, no reinforcement learning) on high-quality, executor-validated trajectories with explicit argument provenance annotations. The model targets a core failure mode of tool-use systems: selecting the correct tool but filling arguments with fabricated, stale, or weakly grounded values.
Three stages:
- (1) FSM skeleton synthesis by A_FSM with provenance tags
- (2) per-call planning + executor execution by A_plan + ε (two-track output: executor args θ_exec + provenance metadata θ_prov, parallel grouping, per-call retry)
- (3) post-hoc dialogue synthesis by A_msg
---
## Model Details
| Item | Specification |
|------|---------------|
| Model Name | SAP-4B |
| Backbone | Qwen3-4B-Instruct-2507 |
| Parameter Size | 4B |
| Training Paradigm | Pure SFT (no RL) |
| Training Data | 9,644 multi-turn tool-use trajectories, all validated against live tool executors |
| Training Framework | verl |
| Hyperparameters | AdamW optimizer, learning rate 1e-6, batch size 128, 10 epochs |
---
## Evaluation Results
### 1. BFCL v4 Multi-Turn
| Benchmark Subset | Accuracy |
|------------------|----------|
| Overall Average | ~30.4% |
| multi_turn_base | 35.50% |
| multi_turn_long_context | 35.50% |
| multi_turn_miss_func | 24.50% |
| multi_turn_miss_param | 26.00% |
### 2. τ²-bench
| Benchmark Split | Score |
|-----------------|-------|
| Overall Average | 35.1% |
| Retail | 42.1% |
| Airline | 28.0% |
---
## Key Capabilities
1. **Cross-turn argument grounding**: Consistently sources tool argument values from verifiable upstream context (prior tool returns, initial state, user messages) rather than hallucinating parameters.
2. **Long-horizon task execution**: Handles multi-step tool workflows across long context windows, maintaining dependency chains across turns.
3. **Deployment robustness**: Performs reliably in missing-parameter and missing-function challenge scenarios that mimic real-world imperfect inputs.
|