Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: pytorch
|
| 3 |
+
license: mit
|
| 4 |
+
tags:
|
| 5 |
+
- liquid-neural-network
|
| 6 |
+
- intent-classification
|
| 7 |
+
- edge
|
| 8 |
+
- calibrated
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# O1-Flash
|
| 12 |
+
|
| 13 |
+
**A liquid-core intent router — 4.53M parameters answering 77-way
|
| 14 |
+
customer-service routing as a calibrated multiple-choice decision.**
|
| 15 |
+
|
| 16 |
+
Part of the AwareLiquid O-Series. The recurrent core carries per-channel
|
| 17 |
+
learnable time constants (τ, geometrically spaced), so one small model holds
|
| 18 |
+
both phoneme-scale and utterance-scale state with a fixed-size hidden state.
|
| 19 |
+
|
| 20 |
+
## Honest status
|
| 21 |
+
|
| 22 |
+
Measured on **banking77** (10,003 train / 3,080 test rows, 77 intents,
|
| 23 |
+
byte-level tokenizer, no external vocab), default config d256/4L/4.53M params,
|
| 24 |
+
batch 64:
|
| 25 |
+
|
| 26 |
+
| steps | test acc | ECE | ECE (temperature-scaled) |
|
| 27 |
+
|---|---|---|---|
|
| 28 |
+
| 1,500 | 23.28% | — | — |
|
| 29 |
+
| 20,000 (lr 1e-3) | 30.23% | 0.485 | 0.161 |
|
| 30 |
+
| 20,000 (scheduled 3e-3) | 33.28% | 0.469 | 0.161 |
|
| 31 |
+
| **200,000 (scheduled)** | **40.75%** | 0.456 | **0.127** |
|
| 32 |
+
|
| 33 |
+
- Chance = 1.3%. The default-config model is far from a production router;
|
| 34 |
+
40.75% at 4.53M params is the honest ceiling measured so far.
|
| 35 |
+
- **Calibration works**: temperature scaling (LBFGS on a held-out half) cuts
|
| 36 |
+
ECE by ~3.6× on the final model (T=4.41).
|
| 37 |
+
- **Training-stability finding (2026-10-04)**: at lr 3e-3, training diverges
|
| 38 |
+
under GPU contention (learns, then collapses to chance — reproduced on two
|
| 39 |
+
machines). Fix: `--lr 1e-3` default plus linear-warmup + cosine schedule.
|
| 40 |
+
The 200k run above ran fully stable under the schedule.
|
| 41 |
+
- The temperature parameter (T=4.41) is saved with the evaluation artifact,
|
| 42 |
+
not baked into the checkpoint: apply it to the softmax only when calibrated
|
| 43 |
+
probabilities are needed.
|
| 44 |
+
|
| 45 |
+
## Files
|
| 46 |
+
|
| 47 |
+
- `o1flash_banking77_200k_sched.pt` — PyTorch checkpoint (18.2 MB,
|
| 48 |
+
4.53M params, trained 200k steps with the lr schedule). Contains
|
| 49 |
+
`model` state dict + `config`.
|
| 50 |
+
|
| 51 |
+
## Usage
|
| 52 |
+
|
| 53 |
+
```python
|
| 54 |
+
import torch
|
| 55 |
+
from benchmarks.real_data_bench import O1Flash, FlashConfig # github.com/AwareLiquid/O1-Flash
|
| 56 |
+
|
| 57 |
+
ck = torch.load("o1flash_banking77_200k_sched.pt", map_location="cpu",
|
| 58 |
+
weights_only=True)
|
| 59 |
+
model = O1Flash(FlashConfig(**ck["config"]))
|
| 60 |
+
model.load_state_dict(ck["model"])
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
## Limits
|
| 64 |
+
|
| 65 |
+
- English customer-service text only; byte-level tokenizer.
|
| 66 |
+
- 40.75% top-1 on 77 intents — not production. The point of this checkpoint
|
| 67 |
+
is reproducibility of the measured numbers and the stability/calibration
|
| 68 |
+
findings, not deployment.
|
| 69 |
+
- The same architecture/discussion as the rest of the O-Series: see
|
| 70 |
+
[EverestAn/MT-LNN](https://huggingface.co/EverestAn/MT-LNN) for the family
|
| 71 |
+
card and the awareliquid.ai benchmark page.
|