| | Metric | Value | |
| |---|---| |
| | Test set | Banking77 official test split, 3080 queries, all answered (coverage 100.0%) | |
| | Accuracy | 94.55% (Wilson 95% interval 93.69-95.29) | |
| | Mean latency | 1.42 ms (per-pass means 1.40-1.44, 5 passes) | |
| | p50 / p95 / p99 | 0.25 / 6.26 / 10.41 ms | |
| | Tier 1 (tf-idf) | 77.3% of queries, mean 0.25 ms | |
| | Tier 2 (int8 MiniLM-L6 + kNN) | 22.7% of queries, mean 5.41 ms | |
| | Hardware | Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used | |
| | Software | Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3 | |
| | Package | 51 MB, loads in 0.2 s | |
|
|
| Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN, label lookup), one query at a time, 1 thread, warm process, |
| GC paused in the timed loop; 5 passes over the test set in different random orders. Excludes process start-up. Shared cloud CPUs vary by session; compare runs from the same session. |
|
|
