xezpeleta commited on
Commit
f57b4e7
·
verified ·
1 Parent(s): 171ad0f

docs: add base Gemma 4 ablation column to comparison table

Browse files
Files changed (1) hide show
  1. README.md +13 -7
README.md CHANGED
@@ -50,13 +50,13 @@ Evaluated on the horkonpon-corpus held-out eval split (1,037 errorful +
50
 
51
  ### Correction
52
 
53
- | Metric | GECToR v2 (mt) | GECToR v1 | Gemma 4 E4B |
54
- |--------|:--------------:|:---------:|:-----------:|
55
- | **F0.5** | 77.6 | 47.5¹ | **80.8** |
56
- | Exact match | 51.3% | 18.0%¹ | 65.5% |
57
- | Precision | 87.6% | 83.8%¹ | 86.3% |
58
- | Recall | 53.3% | 17.4%¹ | 64.4% |
59
- | Clean FP | 1.8% | 1.7%¹ | 8.6% |
60
 
61
  ¹ v1 on horkonpon eval (domain shift — v1 was trained on Elhuyar morphology
62
  only, so it misses spelling/calque/zalantza errors). v1 scores F0.5=90.2 on its
@@ -67,6 +67,12 @@ own Elhuyar Dem eval set.
67
  classification. GECToR wins decisively on **clean FP rate** (1.8% vs Gemma 4's
68
  8.6%) and is lightweight enough for **in-browser ONNX deployment** (~80MB).
69
 
 
 
 
 
 
 
70
  ### Error-type classification (new capability)
71
 
72
  | Metric | Value |
 
50
 
51
  ### Correction
52
 
53
+ | Metric | GECToR v2 (mt) | GECToR v1 | Gemma 4 (FT) | Gemma 4 (base) |
54
+ |--------|:--------------:|:---------:|:-----------:|:--------------:|
55
+ | **F0.5** | 77.6 | 47.5¹ | **80.8** | 2.2 |
56
+ | Exact match | 51.3% | 18.0%¹ | 65.5% | 7.0% |
57
+ | Precision | 87.6% | 83.8%¹ | 86.3% | 1.8% |
58
+ | Recall | 53.3% | 17.4%¹ | 64.4% | 16.5% |
59
+ | Clean FP | 1.8% | 1.7%¹ | 8.6% | 97.2% |
60
 
61
  ¹ v1 on horkonpon eval (domain shift — v1 was trained on Elhuyar morphology
62
  only, so it misses spelling/calque/zalantza errors). v1 scores F0.5=90.2 on its
 
67
  classification. GECToR wins decisively on **clean FP rate** (1.8% vs Gemma 4's
68
  8.6%) and is lightweight enough for **in-browser ONNX deployment** (~80MB).
69
 
70
+ The **base Gemma 4 ablation** (F0.5=2.2, 97.2% clean FP) confirms that
71
+ fine-tuning is essential — the base LLM has linguistic knowledge but no
72
+ minimal-edit discipline. See
73
+ [`itzune/gemma-4-e4b-horkonpon`](https://huggingface.co/itzune/gemma-4-e4b-horkonpon)
74
+ for the full ablation report.
75
+
76
  ### Error-type classification (new capability)
77
 
78
  | Metric | Value |