honest v1 status: experimental, not yet reliable tool-caller
Browse files
README.md
CHANGED
|
@@ -40,5 +40,17 @@ that gap on a tiny (≤2B, Tiny-Titan-class) model.
|
|
| 40 |
Standard Qwen2.5 chat template with `tools=`. The model responds with
|
| 41 |
`<tool_call>{"name": ..., "arguments": ...}</tool_call>` when a tool is warranted.
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
## License
|
| 44 |
Apache-2.0 (inherits from the base model).
|
|
|
|
| 40 |
Standard Qwen2.5 chat template with `tools=`. The model responds with
|
| 41 |
`<tool_call>{"name": ..., "arguments": ...}</tool_call>` when a tool is warranted.
|
| 42 |
|
| 43 |
+
## Status — experimental v1 ⚠️
|
| 44 |
+
This first pass (1 epoch, `max_length=1024`, ~3.7k examples) **does not yet
|
| 45 |
+
reliably emit `<tool_call>`** in free generation: teacher-forced token accuracy
|
| 46 |
+
was 0.92, but greedy decoding is degenerate and sensitive to the prompt template
|
| 47 |
+
(it was trained on the Hermes ChatML rendering, not Qwen's `apply_chat_template`
|
| 48 |
+
output — a train/inference mismatch). Treat as a proof-of-pipeline, not a
|
| 49 |
+
production tool-caller.
|
| 50 |
+
|
| 51 |
+
Known fixes for v2: align train and inference templates (use
|
| 52 |
+
`apply_chat_template(tools=...)` for both), more epochs, full sequence length,
|
| 53 |
+
and a held-out eval on tool-call emission.
|
| 54 |
+
|
| 55 |
## License
|
| 56 |
Apache-2.0 (inherits from the base model).
|