card: 265->268 MB (hub-displayed size)
Browse files
README.md
CHANGED
|
@@ -64,7 +64,7 @@ never on this benchmark.
|
|
| 64 |
deliberately adversarial: every positive uses phrasing templates the model never saw,
|
| 65 |
gold is multi-label, and 24% of cases are near-miss negatives (third-party probes,
|
| 66 |
mechanism/definition traps) built to punish keyword matching. For scale on the *same*
|
| 67 |
-
benchmark: **mistral:7b, a 4.4 GB LLM, scores 0.495 macro-F1** — this
|
| 68 |
it while running every turn on-device.
|
| 69 |
|
| 70 |
| metric | value | mistral:7b, same benchmark |
|
|
@@ -90,7 +90,7 @@ LLM's actual answer on those exact cases.
|
|
| 90 |
| **hybrid: this model + llama3.2** | **0.500** | **0.508** | **0.976** | 0.146 | 16.4% |
|
| 91 |
|
| 92 |
The result that matters for low-RAM machines: **a 2 GB-LLM machine running the hybrid
|
| 93 |
-
(0.500) outperforms a 4.4 GB-LLM machine running LLM-only (0.495)** — the head is
|
| 94 |
and identical on every hardware tier, so only ~1/6th of traffic degrades with weaker
|
| 95 |
hardware.
|
| 96 |
|
|
|
|
| 64 |
deliberately adversarial: every positive uses phrasing templates the model never saw,
|
| 65 |
gold is multi-label, and 24% of cases are near-miss negatives (third-party probes,
|
| 66 |
mechanism/definition traps) built to punish keyword matching. For scale on the *same*
|
| 67 |
+
benchmark: **mistral:7b, a 4.4 GB LLM, scores 0.495 macro-F1** — this 268 MB model beats
|
| 68 |
it while running every turn on-device.
|
| 69 |
|
| 70 |
| metric | value | mistral:7b, same benchmark |
|
|
|
|
| 90 |
| **hybrid: this model + llama3.2** | **0.500** | **0.508** | **0.976** | 0.146 | 16.4% |
|
| 91 |
|
| 92 |
The result that matters for low-RAM machines: **a 2 GB-LLM machine running the hybrid
|
| 93 |
+
(0.500) outperforms a 4.4 GB-LLM machine running LLM-only (0.495)** — the head is 268 MB
|
| 94 |
and identical on every hardware tier, so only ~1/6th of traffic degrades with weaker
|
| 95 |
hardware.
|
| 96 |
|