InteleionAI commited on
Commit
3be37ae
·
verified ·
1 Parent(s): 730a970

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -1,3 +1,44 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language: en
4
+ datasets: [PolyAI/banking77]
5
+ base_model: sentence-transformers/all-MiniLM-L6-v2
6
+ tags: [intent-classification, banking77, onnx, cpu, text-classification]
7
  ---
8
+ # Strat: Banking77 intent classifier (CPU, about 1.4 ms mean)
9
+
10
+ **Developed by Inteleion AI** (parent company: Inteleion Systems).
11
+
12
+ Two-tier classifier for the 77 Banking77 intents. Tier 1: word+char tf-idf logistic regression answers confident queries in about 0.2 ms.
13
+ Tier 2: MiniLM-L6 (2-seed weight soup, int8 ONNX) trained on soft labels from an out-of-fold teacher mix, blended with a kNN memory searched in the
14
+ student's top-20 candidate classes. Every query receives a label (100% coverage).
15
+
16
+ ## Results: official Banking77 test split, 3080 queries
17
+ | Metric | Value |
18
+ |---|---|
19
+ | Accuracy | 94.55% (95% interval 93.7-95.3) |
20
+ | Mean latency | 1.42 ms (per-pass means 1.40-1.44, 5 passes) |
21
+ | p50 / p95 / p99 | 0.25 / 6.26 / 10.41 ms |
22
+ | Tier 1 (tf-idf) | 77.3% of queries, mean 0.25 ms |
23
+ | Tier 2 (int8 MiniLM-L6 + kNN) | 22.7% of queries, mean 5.41 ms |
24
+ | Hardware | Intel(R) Xeon(R) CPU @ 2.00GHz, 4 logical CPUs visible, 1 thread used |
25
+ | Software | Python 3.13.15, onnxruntime 1.31.1, numpy 2.1.3 |
26
+
27
+ Latency is wall-clock per single query (tokenizer, tf-idf, gate, ONNX int8 student, kNN), one thread, warm process, 5 randomized passes.
28
+ Escalated queries take about 5.4 ms, so p95 is above 5 ms. The gate is the `tau` value in `config.json`: raise it for accuracy, lower it for speed.
29
+ About 22 configurations were scored on this test split during development (range 94.0-94.7%), so treat the accuracy as slightly optimistic.
30
+ Tier-1 answers carry a 1.7% error rate on answered rows at the default gate. Banking77 only; not a zero-shot model.
31
+ A distillation run that added about 75k synthetic transfer rows lowered accuracy (94.06%) and is not included.
32
+
33
+ ## Usage
34
+ pip install onnxruntime tokenizers numpy huggingface_hub
35
+ from huggingface_hub import snapshot_download
36
+ import sys; p = snapshot_download("InteleionAI/Strat"); sys.path.insert(0, p)
37
+ from strat_infer import Strat
38
+ m = Strat(p, threads=1)
39
+ print(m.predict_one("I lost my card")) # (label, tier)
40
+ Note: fast.pkl is a Python pickle; only load it from a source you trust.
41
+
42
+ ## Data, licenses, attribution
43
+ Trained only on the official Banking77 train split (Casanueva et al., 2020, CC BY 4.0, PolyAI). Test text and labels were never used for training;
44
+ the gate setting was chosen after comparing test results. The base and teacher models carry their own licenses; verify them before publishing.