Benchmark Summary
ONNX q8
| Suite | F1 | Examples/s |
|---|---|---|
| Irish core | 0.9664 | 247.0809 |
| Edge | 1.0000 | 247.8374 |
| Finance | 1.0000 | 260.5229 |
| Finance boundary | 1.0000 | 111.3480 |
| User PPSN | 1.0000 | 240.0219 |
| GA weak PPSN | 1.0000 | 121.8613 |
| Multilingual PPSN | 0.9212 | 256.1316 |
| Hardening exact | 0.9744 | 231.8666 |
| UAT replay exact | 0.8276 | 183.6675 |
Full checkpoint
| Suite | F1 | Examples/s |
|---|---|---|
| Irish core | 0.9664 | 47.2794 |
| Edge | 1.0000 | 38.1395 |
| Multilingual PPSN | 0.9212 | 65.8959 |
| Hardening exact | 0.9744 | 31.0518 |
UAT Replay Exact Comparison
| Model | F1 | Precision | Recall | Examples/s |
|---|---|---|---|---|
IrishCore-DiffMask-135M-v1-rc1 q8 |
0.4545 | 1.0000 | 0.2941 | 238.6524 |
OpenMed-mLiteClinical-IrishCorePII-135M-v2-rc8 q8 |
0.3636 | 0.3750 | 0.3529 | 110.7595 |
IrishCore-DiffMask-135M-v1-rc2 q8 |
0.8276 | 1.0000 | 0.7059 | 183.6675 |
Notes
rc2was selected from an interpolation blend after cleaning label contamination in the v5 training mix.- The remaining known misses on the UAT replay suite are
071 967 2616,R93 EC57inside a longer centre block,EPStamp4@enterprise.gov.ie, and oneD02 XY45form.