EverestAn commited on
Commit
6cf033d
·
verified ·
1 Parent(s): e475371

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +71 -0
README.md ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: pytorch
3
+ license: mit
4
+ tags:
5
+ - liquid-neural-network
6
+ - intent-classification
7
+ - edge
8
+ - calibrated
9
+ ---
10
+
11
+ # O1-Flash
12
+
13
+ **A liquid-core intent router — 4.53M parameters answering 77-way
14
+ customer-service routing as a calibrated multiple-choice decision.**
15
+
16
+ Part of the AwareLiquid O-Series. The recurrent core carries per-channel
17
+ learnable time constants (τ, geometrically spaced), so one small model holds
18
+ both phoneme-scale and utterance-scale state with a fixed-size hidden state.
19
+
20
+ ## Honest status
21
+
22
+ Measured on **banking77** (10,003 train / 3,080 test rows, 77 intents,
23
+ byte-level tokenizer, no external vocab), default config d256/4L/4.53M params,
24
+ batch 64:
25
+
26
+ | steps | test acc | ECE | ECE (temperature-scaled) |
27
+ |---|---|---|---|
28
+ | 1,500 | 23.28% | — | — |
29
+ | 20,000 (lr 1e-3) | 30.23% | 0.485 | 0.161 |
30
+ | 20,000 (scheduled 3e-3) | 33.28% | 0.469 | 0.161 |
31
+ | **200,000 (scheduled)** | **40.75%** | 0.456 | **0.127** |
32
+
33
+ - Chance = 1.3%. The default-config model is far from a production router;
34
+ 40.75% at 4.53M params is the honest ceiling measured so far.
35
+ - **Calibration works**: temperature scaling (LBFGS on a held-out half) cuts
36
+ ECE by ~3.6× on the final model (T=4.41).
37
+ - **Training-stability finding (2026-10-04)**: at lr 3e-3, training diverges
38
+ under GPU contention (learns, then collapses to chance — reproduced on two
39
+ machines). Fix: `--lr 1e-3` default plus linear-warmup + cosine schedule.
40
+ The 200k run above ran fully stable under the schedule.
41
+ - The temperature parameter (T=4.41) is saved with the evaluation artifact,
42
+ not baked into the checkpoint: apply it to the softmax only when calibrated
43
+ probabilities are needed.
44
+
45
+ ## Files
46
+
47
+ - `o1flash_banking77_200k_sched.pt` — PyTorch checkpoint (18.2 MB,
48
+ 4.53M params, trained 200k steps with the lr schedule). Contains
49
+ `model` state dict + `config`.
50
+
51
+ ## Usage
52
+
53
+ ```python
54
+ import torch
55
+ from benchmarks.real_data_bench import O1Flash, FlashConfig # github.com/AwareLiquid/O1-Flash
56
+
57
+ ck = torch.load("o1flash_banking77_200k_sched.pt", map_location="cpu",
58
+ weights_only=True)
59
+ model = O1Flash(FlashConfig(**ck["config"]))
60
+ model.load_state_dict(ck["model"])
61
+ ```
62
+
63
+ ## Limits
64
+
65
+ - English customer-service text only; byte-level tokenizer.
66
+ - 40.75% top-1 on 77 intents — not production. The point of this checkpoint
67
+ is reproducibility of the measured numbers and the stability/calibration
68
+ findings, not deployment.
69
+ - The same architecture/discussion as the rest of the O-Series: see
70
+ [EverestAn/MT-LNN](https://huggingface.co/EverestAn/MT-LNN) for the family
71
+ card and the awareliquid.ai benchmark page.