seanpoyner commited on
Commit
1ddf864
·
verified ·
1 Parent(s): cea4e21

honest v1 status: experimental, not yet reliable tool-caller

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -40,5 +40,17 @@ that gap on a tiny (≤2B, Tiny-Titan-class) model.
40
  Standard Qwen2.5 chat template with `tools=`. The model responds with
41
  `<tool_call>{"name": ..., "arguments": ...}</tool_call>` when a tool is warranted.
42
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  ## License
44
  Apache-2.0 (inherits from the base model).
 
40
  Standard Qwen2.5 chat template with `tools=`. The model responds with
41
  `<tool_call>{"name": ..., "arguments": ...}</tool_call>` when a tool is warranted.
42
 
43
+ ## Status — experimental v1 ⚠️
44
+ This first pass (1 epoch, `max_length=1024`, ~3.7k examples) **does not yet
45
+ reliably emit `<tool_call>`** in free generation: teacher-forced token accuracy
46
+ was 0.92, but greedy decoding is degenerate and sensitive to the prompt template
47
+ (it was trained on the Hermes ChatML rendering, not Qwen's `apply_chat_template`
48
+ output — a train/inference mismatch). Treat as a proof-of-pipeline, not a
49
+ production tool-caller.
50
+
51
+ Known fixes for v2: align train and inference templates (use
52
+ `apply_chat_template(tools=...)` for both), more epochs, full sequence length,
53
+ and a held-out eval on tool-call emission.
54
+
55
  ## License
56
  Apache-2.0 (inherits from the base model).