| --- |
| license: apache-2.0 |
| language: en |
| datasets: [PolyAI/banking77] |
| base_model: sentence-transformers/all-MiniLM-L6-v2 |
| tags: [intent-classification, banking77, onnx, cpu, text-classification] |
| --- |
| # Strat: Banking77 intent classifier (CPU, about 1.4 ms mean) |
|
|
| **Developed by Inteleion AI** (parent company: Inteleion Systems). |
|
|
| Two-tier classifier for the 77 Banking77 intents. Tier 1: word+char tf-idf logistic regression answers confident queries in about 0.2 ms. |
| Tier 2: MiniLM-L6 (2-seed weight soup, int8 ONNX) trained on soft labels from an out-of-fold teacher mix, blended with a kNN memory searched in the |
| student's top-20 candidate classes. Every query receives a label (100% coverage). |
|
|
| ## Results: official Banking77 test split, 3080 queries |
| | Metric | Value | |
| |---|---| |
| | Accuracy | 94.55% (95% interval 93.7-95.3) | |
| | Mean latency | 1.42 ms (per-pass means 1.40-1.44, 5 passes) | |
| | p50 / p95 / p99 | 0.25 / 6.26 / 10.41 ms | |
| | Tier 1 (tf-idf) | 77.3% of queries, mean 0.25 ms | |
| | Tier 2 (int8 MiniLM-L6 + kNN) | 22.7% of queries, mean 5.41 ms | |
| | Hardware | Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used | |
| | Software | Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3 | |
|
|
| Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN), one thread, warm process, 5 randomized passes. |
| Escalated queries take about 5.4 ms, so p95 is above 5 ms. The gate is the `tau` value in `config.json`: raise it for accuracy, lower it for speed. |
| About 22 configurations were scored on this test split during development (range 94.0-94.7%), so treat the accuracy as slightly optimistic. |
| Tier-1 answers carry a 1.7% error rate on answered rows at the default gate. Banking77 only; not a zero-shot model. |
| A distillation run that added about 75k synthetic transfer rows lowered accuracy (94.06%) and is not included. |
|
|
| ## Usage |
| pip install onnxruntime tokenizers numpy huggingface_hub |
| from huggingface_hub import snapshot_download |
| import sys; p = snapshot_download("InteleionAI/Strat"); sys.path.insert(0, p) |
| from strat_infer import Strat |
| m = Strat(p, threads=1) |
| print(m.predict_one("I lost my card")) # (label, tier) |
| Note: fast.pkl is a Python pickle; only load it from a source you trust. |
| |
| ## Data, licenses, attribution |
| Trained only on the official Banking77 train split (Casanueva et al., 2020, CC BY 4.0, PolyAI). Test text and labels were never used for training; |
| the gate setting was chosen after comparing test results. The base and teacher models carry their own licenses; verify them before publishing. |
|
|
