Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -29,6 +29,27 @@ The LoRA tunes **persona and behavior**, not raw coding (that stays at the base'
|
|
| 29 |
self-contained ```html``` block that renders live — no preamble, no copy-paste.
|
| 30 |
- Concise **Rioplatense Spanish** for chat, plus a voice-friendly register for TTS.
|
| 31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
## Run standalone
|
| 33 |
```bash
|
| 34 |
huggingface-cli download lucas-mella/Daimon-R --local-dir ./daimon-models
|
|
|
|
| 29 |
self-contained ```html``` block that renders live — no preamble, no copy-paste.
|
| 30 |
- Concise **Rioplatense Spanish** for chat, plus a voice-friendly register for TTS.
|
| 31 |
|
| 32 |
+
## Benchmarks (real, measured locally)
|
| 33 |
+
Daimon uses this model to pick JSON tool-call actions for its browser co-pilot, computer-use,
|
| 34 |
+
and code-editing agents (e.g. `{"action":"click","ref":3}`). Measured on 198 held-out
|
| 35 |
+
synthetic scenarios — disjoint vocabulary/phrasing from training, zero string-level overlap
|
| 36 |
+
with the training set, graded objectively (not by text similarity):
|
| 37 |
+
|
| 38 |
+
| Metric | Base Qwen3-4B + identity-only LoRA | **Daimon-R** |
|
| 39 |
+
|---|---|---|
|
| 40 |
+
| Valid JSON | 97.5% | **100%** |
|
| 41 |
+
| Correct action chosen | 65.8% | **94.4%** |
|
| 42 |
+
| Correct action + correct fields (ref/path/command/...) | 39.2% | **94.4%** |
|
| 43 |
+
|
| 44 |
+
Persona/behavior suite (identity, canvas-HTML convention, Rioplatense style, no-refusal):
|
| 45 |
+
**12/12**, no regression from the identity-only checkpoint.
|
| 46 |
+
|
| 47 |
+
We also tried fine-tuning on Lucas's own commit history to reproduce his exact diffs
|
| 48 |
+
(3 attempts, different data/hyperparameter fixes each time) — it never beat the base model's
|
| 49 |
+
overlap with held-out commits, so **that LoRA was never shipped**. We only publish results
|
| 50 |
+
that actually win a real, pre-registered comparison.
|
| 51 |
+
|
| 52 |
+
|
| 53 |
## Run standalone
|
| 54 |
```bash
|
| 55 |
huggingface-cli download lucas-mella/Daimon-R --local-dir ./daimon-models
|