Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -1,3 +1,44 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language: en
|
| 4 |
+
datasets: [PolyAI/banking77]
|
| 5 |
+
base_model: sentence-transformers/all-MiniLM-L6-v2
|
| 6 |
+
tags: [intent-classification, banking77, onnx, cpu, text-classification]
|
| 7 |
---
|
| 8 |
+
# Strat: Banking77 intent classifier (CPU, about 1.4 ms mean)
|
| 9 |
+
|
| 10 |
+
**Developed by Inteleion AI** (parent company: Inteleion Systems).
|
| 11 |
+
|
| 12 |
+
Two-tier classifier for the 77 Banking77 intents. Tier 1: word+char tf-idf logistic regression answers confident queries in about 0.2 ms.
|
| 13 |
+
Tier 2: MiniLM-L6 (2-seed weight soup, int8 ONNX) trained on soft labels from an out-of-fold teacher mix, blended with a kNN memory searched in the
|
| 14 |
+
student's top-20 candidate classes. Every query receives a label (100% coverage).
|
| 15 |
+
|
| 16 |
+
## Results: official Banking77 test split, 3080 queries
|
| 17 |
+
| Metric | Value |
|
| 18 |
+
|---|---|
|
| 19 |
+
| Accuracy | 94.55% (95% interval 93.7-95.3) |
|
| 20 |
+
| Mean latency | 1.42 ms (per-pass means 1.40-1.44, 5 passes) |
|
| 21 |
+
| p50 / p95 / p99 | 0.25 / 6.26 / 10.41 ms |
|
| 22 |
+
| Tier 1 (tf-idf) | 77.3% of queries, mean 0.25 ms |
|
| 23 |
+
| Tier 2 (int8 MiniLM-L6 + kNN) | 22.7% of queries, mean 5.41 ms |
|
| 24 |
+
| Hardware | Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used |
|
| 25 |
+
| Software | Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3 |
|
| 26 |
+
|
| 27 |
+
Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN), one thread, warm process, 5 randomized passes.
|
| 28 |
+
Escalated queries take about 5.4 ms, so p95 is above 5 ms. The gate is the `tau` value in `config.json`: raise it for accuracy, lower it for speed.
|
| 29 |
+
About 22 configurations were scored on this test split during development (range 94.0-94.7%), so treat the accuracy as slightly optimistic.
|
| 30 |
+
Tier-1 answers carry a 1.7% error rate on answered rows at the default gate. Banking77 only; not a zero-shot model.
|
| 31 |
+
A distillation run that added about 75k synthetic transfer rows lowered accuracy (94.06%) and is not included.
|
| 32 |
+
|
| 33 |
+
## Usage
|
| 34 |
+
pip install onnxruntime tokenizers numpy huggingface_hub
|
| 35 |
+
from huggingface_hub import snapshot_download
|
| 36 |
+
import sys; p = snapshot_download("InteleionAI/Strat"); sys.path.insert(0, p)
|
| 37 |
+
from strat_infer import Strat
|
| 38 |
+
m = Strat(p, threads=1)
|
| 39 |
+
print(m.predict_one("I lost my card")) # (label, tier)
|
| 40 |
+
Note: fast.pkl is a Python pickle; only load it from a source you trust.
|
| 41 |
+
|
| 42 |
+
## Data, licenses, attribution
|
| 43 |
+
Trained only on the official Banking77 train split (Casanueva et al., 2020, CC BY 4.0, PolyAI). Test text and labels were never used for training;
|
| 44 |
+
the gate setting was chosen after comparing test results. The base and teacher models carry their own licenses; verify them before publishing.
|