File size: 921 Bytes
ba1c728 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | | Metric | Value | |---|---| | Test set | Banking77 official test split, 3080 queries, all answered (coverage 100.0%) | | Accuracy | 94.55% (Wilson 95% interval 93.69-95.29) | | Mean latency | 1.42 ms (per-pass means 1.40-1.44, 5 passes) | | p50 / p95 / p99 | 0.25 / 6.26 / 10.41 ms | | Tier 1 (tf-idf) | 77.3% of queries, mean 0.25 ms | | Tier 2 (int8 MiniLM-L6 + kNN) | 22.7% of queries, mean 5.41 ms | | Hardware | Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used | | Software | Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3 | | Package | 51 MB, loads in 0.2 s | Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN, label lookup), one query at a time, 1 thread, warm process, GC paused in the timed loop; 5 passes over the test set in different random orders. Excludes process start-up. Shared cloud CPUs vary by session; compare runs from the same session. |