Update README.md
Browse files
README.md
CHANGED
|
@@ -30,7 +30,7 @@ SAP-4B is a 4B-parameter agentic language model specialized in **long-horizon mu
|
|
| 30 |
|
| 31 |
## Evaluation Results
|
| 32 |
|
| 33 |
-
### 1. BFCL v4 Multi-Turn
|
| 34 |
|
| 35 |
| Benchmark Subset | Accuracy |
|
| 36 |
|------------------|----------|
|
|
@@ -40,7 +40,7 @@ SAP-4B is a 4B-parameter agentic language model specialized in **long-horizon mu
|
|
| 40 |
| multi_turn_miss_func | 24.50% |
|
| 41 |
| multi_turn_miss_param | 26.00% |
|
| 42 |
|
| 43 |
-
### 2. τ²-bench
|
| 44 |
|
| 45 |
| Benchmark Split | Score |
|
| 46 |
|-----------------|-------|
|
|
@@ -50,7 +50,7 @@ SAP-4B is a 4B-parameter agentic language model specialized in **long-horizon mu
|
|
| 50 |
|
| 51 |
---
|
| 52 |
|
| 53 |
-
### Performance Note
|
| 54 |
- The paper has been submitted to EMNLP 2026, and its reported results use the benchmark evaluation setting available at the time of submission.
|
| 55 |
- For BFCL v4 Multi-Turn: subsequent updates to the evaluation setting led to minor subset-level score fluctuations in the latest re-evaluation, while the overall average performance remains consistent with the paper's conclusions. Fluctuations are purely caused by evaluation pipeline adjustments (stricter matching rules for base/long-context subsets, more favorable calibration for robustness subsets), with no changes to the model or training data.
|
| 56 |
- For τ²-bench: only the submission-version result is currently available; re-evaluation under an updated setting may produce slightly different subset-level scores.
|
|
|
|
| 30 |
|
| 31 |
## Evaluation Results
|
| 32 |
|
| 33 |
+
### 1. BFCL v4 Multi-Turn
|
| 34 |
|
| 35 |
| Benchmark Subset | Accuracy |
|
| 36 |
|------------------|----------|
|
|
|
|
| 40 |
| multi_turn_miss_func | 24.50% |
|
| 41 |
| multi_turn_miss_param | 26.00% |
|
| 42 |
|
| 43 |
+
### 2. τ²-bench
|
| 44 |
|
| 45 |
| Benchmark Split | Score |
|
| 46 |
|-----------------|-------|
|
|
|
|
| 50 |
|
| 51 |
---
|
| 52 |
|
| 53 |
+
### Performance Note
|
| 54 |
- The paper has been submitted to EMNLP 2026, and its reported results use the benchmark evaluation setting available at the time of submission.
|
| 55 |
- For BFCL v4 Multi-Turn: subsequent updates to the evaluation setting led to minor subset-level score fluctuations in the latest re-evaluation, while the overall average performance remains consistent with the paper's conclusions. Fluctuations are purely caused by evaluation pipeline adjustments (stricter matching rules for base/long-context subsets, more favorable calibration for robustness subsets), with no changes to the model or training data.
|
| 56 |
- For τ²-bench: only the submission-version result is currently available; re-evaluation under an updated setting may produce slightly different subset-level scores.
|