jonny commited on
Commit
77c7eff
·
verified ·
1 Parent(s): 2968c68

card: 265->268 MB (hub-displayed size)

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -64,7 +64,7 @@ never on this benchmark.
64
  deliberately adversarial: every positive uses phrasing templates the model never saw,
65
  gold is multi-label, and 24% of cases are near-miss negatives (third-party probes,
66
  mechanism/definition traps) built to punish keyword matching. For scale on the *same*
67
- benchmark: **mistral:7b, a 4.4 GB LLM, scores 0.495 macro-F1** — this 265 MB model beats
68
  it while running every turn on-device.
69
 
70
  | metric | value | mistral:7b, same benchmark |
@@ -90,7 +90,7 @@ LLM's actual answer on those exact cases.
90
  | **hybrid: this model + llama3.2** | **0.500** | **0.508** | **0.976** | 0.146 | 16.4% |
91
 
92
  The result that matters for low-RAM machines: **a 2 GB-LLM machine running the hybrid
93
- (0.500) outperforms a 4.4 GB-LLM machine running LLM-only (0.495)** — the head is 265 MB
94
  and identical on every hardware tier, so only ~1/6th of traffic degrades with weaker
95
  hardware.
96
 
 
64
  deliberately adversarial: every positive uses phrasing templates the model never saw,
65
  gold is multi-label, and 24% of cases are near-miss negatives (third-party probes,
66
  mechanism/definition traps) built to punish keyword matching. For scale on the *same*
67
+ benchmark: **mistral:7b, a 4.4 GB LLM, scores 0.495 macro-F1** — this 268 MB model beats
68
  it while running every turn on-device.
69
 
70
  | metric | value | mistral:7b, same benchmark |
 
90
  | **hybrid: this model + llama3.2** | **0.500** | **0.508** | **0.976** | 0.146 | 16.4% |
91
 
92
  The result that matters for low-RAM machines: **a 2 GB-LLM machine running the hybrid
93
+ (0.500) outperforms a 4.4 GB-LLM machine running LLM-only (0.495)** — the head is 268 MB
94
  and identical on every hardware tier, so only ~1/6th of traffic degrades with weaker
95
  hardware.
96