metadata
license: apache-2.0
datasets:
- Zichen1024/SAP-9k
base_model:
- Qwen/Qwen3-4B-Instruct-2507
Model Card: SAP-4B
paper: SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Use
Model Overview
SAP-4B is a 4B-parameter agentic language model specialized in long-horizon multi-turn tool use and function calling. It is trained via supervised fine-tuning (SFT, no reinforcement learning) on high-quality, executor-validated trajectories with explicit argument provenance annotations. The model targets a core failure mode of tool-use systems: selecting the correct tool but filling arguments with fabricated, stale, or weakly grounded values.
Three stages:
- (1) FSM skeleton synthesis by A_FSM with provenance tags
- (2) per-call planning + executor execution by A_plan + ε (two-track output: executor args θ_exec + provenance metadata θ_prov, parallel grouping, per-call retry)
- (3) post-hoc dialogue synthesis by A_msg
Model Details
| Item | Specification |
|---|---|
| Model Name | SAP-4B |
| Backbone | Qwen3-4B-Instruct-2507 |
| Parameter Size | 4B |
| Training Paradigm | Pure SFT (no RL) |
| Training Data | 9,644 multi-turn tool-use trajectories, all validated against live tool executors |
| Training Framework | verl |
| Hyperparameters | AdamW optimizer, learning rate 1e-6, batch size 128, 10 epochs |
Evaluation Results
1. BFCL v4 Multi-Turn
| Benchmark Subset | Accuracy |
|---|---|
| Overall Average | ~30.4% |
| multi_turn_base | 35.50% |
| multi_turn_long_context | 35.50% |
| multi_turn_miss_func | 24.50% |
| multi_turn_miss_param | 26.00% |
2. τ²-bench
| Benchmark Split | Score |
|---|---|
| Overall Average | 35.1% |
| Retail | 42.1% |
| Airline | 28.0% |
Key Capabilities
- Cross-turn argument grounding: Consistently sources tool argument values from verifiable upstream context (prior tool returns, initial state, user messages) rather than hallucinating parameters.
- Long-horizon task execution: Handles multi-step tool workflows across long context windows, maintaining dependency chains across turns.
- Deployment robustness: Performs reliably in missing-parameter and missing-function challenge scenarios that mimic real-world imperfect inputs.