Irish Core Release Benchmark
Models
base:OpenMed/OpenMed-PII-mLiteClinical-Base-135M-v1(transformers)current:temsa/OpenMed-mLiteClinical-IrishPPSN-135M-v1(transformers)candidate:temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v1(transformers)onnx_q8:temsa/OpenMed-mLiteClinical-IrishCorePII-135M-v1#onnx_q8(onnx dynamic int8, per-channel)
Core Suite F1
| Label | base | current | candidate | onnx_q8 |
|---|---|---|---|---|
PPSN |
0.0000 | 0.0800 | 0.8000 | 0.7273 |
account_number |
0.3333 | 0.3333 | 1.0000 | 1.0000 |
bank_routing_number |
0.0000 | 0.0000 | 1.0000 | 1.0000 |
credit_debit_card |
0.1538 | 0.1818 | 1.0000 | 0.3333 |
PASSPORT_NUMBER |
0.0000 | 0.0000 | 1.0000 | 1.0000 |
postcode |
0.0000 | 0.0000 | 1.0000 | 1.0000 |
phone_number |
0.0000 | 0.0000 | 0.8571 | 0.8571 |
email |
0.7059 | 1.0000 | 1.0000 | 1.0000 |
first_name |
0.8947 | 0.8947 | 1.0000 | 1.0000 |
last_name |
0.8889 | 0.8889 | 1.0000 | 1.0000 |
swift_bic |
0.0000 | 0.0000 | 1.0000 | 1.0000 |
Edge And Multilingual F1
| Suite | base | current | candidate | onnx_q8 |
|---|---|---|---|---|
edge_ppsn |
0.0000 | 0.4211 | 0.5000 | 0.4000 |
edge_phone_number |
0.1429 | 0.1429 | 0.6316 | 0.5000 |
multilingual_ppsn |
0.0000 | 0.9704 | 0.9940 | 0.9882 |