lucas-mella commited on
Commit
431f3ee
·
verified ·
1 Parent(s): bc95272

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -29,6 +29,27 @@ The LoRA tunes **persona and behavior**, not raw coding (that stays at the base'
29
  self-contained ```html``` block that renders live — no preamble, no copy-paste.
30
  - Concise **Rioplatense Spanish** for chat, plus a voice-friendly register for TTS.
31
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
  ## Run standalone
33
  ```bash
34
  huggingface-cli download lucas-mella/Daimon-R --local-dir ./daimon-models
 
29
  self-contained ```html``` block that renders live — no preamble, no copy-paste.
30
  - Concise **Rioplatense Spanish** for chat, plus a voice-friendly register for TTS.
31
 
32
+ ## Benchmarks (real, measured locally)
33
+ Daimon uses this model to pick JSON tool-call actions for its browser co-pilot, computer-use,
34
+ and code-editing agents (e.g. `{"action":"click","ref":3}`). Measured on 198 held-out
35
+ synthetic scenarios — disjoint vocabulary/phrasing from training, zero string-level overlap
36
+ with the training set, graded objectively (not by text similarity):
37
+
38
+ | Metric | Base Qwen3-4B + identity-only LoRA | **Daimon-R** |
39
+ |---|---|---|
40
+ | Valid JSON | 97.5% | **100%** |
41
+ | Correct action chosen | 65.8% | **94.4%** |
42
+ | Correct action + correct fields (ref/path/command/...) | 39.2% | **94.4%** |
43
+
44
+ Persona/behavior suite (identity, canvas-HTML convention, Rioplatense style, no-refusal):
45
+ **12/12**, no regression from the identity-only checkpoint.
46
+
47
+ We also tried fine-tuning on Lucas's own commit history to reproduce his exact diffs
48
+ (3 attempts, different data/hyperparameter fixes each time) — it never beat the base model's
49
+ overlap with held-out commits, so **that LoRA was never shipped**. We only publish results
50
+ that actually win a real, pre-registered comparison.
51
+
52
+
53
  ## Run standalone
54
  ```bash
55
  huggingface-cli download lucas-mella/Daimon-R --local-dir ./daimon-models