---
license: apache-2.0
library_name: tflite
pipeline_tag: text-classification
tags:
- edge-tpu
- coral
- tflite
- int8
- typed-decisions
- systemone
- decision-index
- reef
base_model: minishlab/potion-base-8M
---

# Reef

**A general typed-decision model that runs entirely on a Google Coral Edge TPU.** You send a
state (text or JSON) and typed questions: yes/no (`noul`), one-of-N (`choice`) or an ordinal rubric
(`score`). You get a calibrated probability for every option. About 11M parameters in total,
int8, one Edge TPU subgraph that fits in the Coral's on-chip memory.

Part of **[Reef](https://github.com/SeraphimSerapis/reef)**: give your retired Coral a new job.

## Decision Index 0.2.1

Self-measured with the public harness ([apolinario/decision-index](https://github.com/apolinario/decision-index),
MIT) on the full local suite (150,246 requests). **This is not an official leaderboard entry.**
The comparison rows are the published board scores.

| Model | Params (total) | Index |
|---|---|---|
| **Reef (this model, int8)** | **11.2M** | **4.73** |
| GLiNER 2.5 small ([fastino/gliner2.5-small-v1](https://huggingface.co/fastino/gliner2.5-small-v1)) | 73.9M | 3.82 |
| Reef v1 (previous version, a homelab specialist) | 10.3M | 2.38 |

How the Reef score was measured:

- **Serving mode.** Measured with the server's default (lenient) mode: over-long inputs are truncated to the trained limits instead of refused. With `--strict`, the score is about 0.15–0.2 lower (measured on a 19,640-row subset with an earlier checkpoint).
- **HLE.** HLE has no rows in the local suite and is scored 0, so the number is a lower bound for the board.
- **Run-to-run variance.** Retraining moved single benchmarks by up to ±0.1 skill (PhishNChips most), so treat differences of about ±0.3 Index points between checkpoints as noise.
- **Params.** 7.38M are in the int8 encoder on the Edge TPU and 3.78M in the host's token-embedding table (frozen [potion-base-8M](https://huggingface.co/minishlab/potion-base-8M) vectors projected to 128 dimensions). All neural-network compute runs on the Coral. The host tokenises, looks up token vectors and applies a softmax.

| Area | Skill (×100) |
|---|---|
| Retrieval | 10.47 |
| Tools | 5.56 |
| Language | 4.79 |
| Arts & human taste | 1.62 |
| Knowledge | 0.84 |

**Where it does well:** intent detection (BANKING77 0.26 skill, CLINC150+OOS 0.14 with 77/151
options), When2Call (0.25), FinEntity (0.37), PhishNChips (0.18), ContractNLI (0.10).

**Where it doesn't:** it sits at chance, with no detectable signal, on tool retrieval and
selection (BFCL, ToolRet), BRIGHT, HoVer, ANLI, VAST, Amazon ESCI and the knowledge and reasoning
benchmarks (MMLU-Pro, BBH, GSM8K, CLadder, CRUXEval). An 11M model trained from scratch has little
world knowledge.

**Shortcuts deliberately not used.** We did not rank documents by the harness's BM25 order
(ToolRet/BRIGHT), and we did not decode BFCL as "exactly one tool". Both would have added Index
points without any model skill.

## On the Edge TPU

Same int8 model on a Coral USB Accelerator:

| Test set (held out) | Accuracy | KL ↓ |
|---|---|---|
| Homelab packs (5 packs, Gemma 4 soft labels) | 0.807 | 0.212 |
| Answer-audit (judge for agent transcripts) | 0.705 | 0.634 |
| typed-decisions test | 0.535 | 0.294 |
| Open-Jev OOD | 0.695 | 0.610 |

For the narrow homelab packs alone, the previous homelab-specialist version is still slightly
better (0.84). This model is the generalist.

## Quickstart

```bash
pip install "reef-decide @ git+https://github.com/SeraphimSerapis/reef"
reef serve --host 0.0.0.0 --port 8080      # fetches this model; Edge TPU if attached, else CPU
```

```bash
curl -s localhost:8080/v1/systemone -d '{
  "state": {"source": "uptime-kuma", "monitor": "nextcloud", "status": "down", "down_for": "14m"},
  "questions": {"page": {"type": "noul", "instructions": "Should someone be woken up for this?"}}
}'
```

## Files

| File | What |
|---|---|
| `reef_int8_edgetpu.tflite` | encoder compiled for the Edge TPU (1 subgraph, 7.36 MiB on-chip, 0 B off-chip) |
| `reef_int8.tflite` | the same int8 encoder for CPU (LiteRT, XNNPACK) |
| `embeddings.npz` | token / role / level embedding tables (looked up on the host) |
| `tokenizer.json`, `config.json` | tokenizer, shapes and per-type calibration temperatures |
| `reef_float.weights.h5` | float training weights (Keras), for `reef train --init seraphimserapis/reef` |

## Model

- **Convolutional encoder**, no attention (the Edge TPU compiler cannot map it): 11 blocks, width 256, MLP expansion 3, a 3×5 depthwise conv and mean+max state summaries, plus two question bands with their own summaries and a multiplicative gate.
- **Input**: a 384-token state region plus a 384-token question region, folded into a 24×32 grid. Long states are read in overlapping windows, and questions with many options as a knockout.
- **Training stages**:
  1. pre-training on about 300k public decision cases;
  2. fine-tuning on homelab packs, answer-audit, typed-decisions, Open-Jev and the public train sets below;
  3. quantisation-aware fine-tuning, so the int8 model stays within about 1–2 accuracy points of the float model.

## Training data

All public data comes from **train splits only**, deduplicated against the Decision Index test
inputs (normalised exact-match on distinguishing inputs, lines and options), with permissive
licences. It uses human labels with label smoothing. One exception was kept on purpose: tool
descriptions shared by public tool catalogs (Glaive, ToolACE, When2Call) also appear in the
ToolRet/BRIGHT candidate corpus; the model scores at chance on ToolRet.

- **Intents:** MASSIVE en/de, SNIPS, BANKING77, CLINC150, MINDS-14
- **NLI and contracts:** WANLI, MultiNLI (OANC genres only), ContractNLI
- **Tools:** Glaive function calling v2, ToolACE, When2Call, API-Bank
- **Retrieval:** Amazon ESCI
- **Sentiment, finance and toxicity:** GoEmotions, Civil Comments, SMS Spam, SEntFiN, Twitter financial news topic/sentiment, iSarcasmEval
- **Commonsense:** HellaSwag, WinoGrande, GSM8K
- **Benchmark data:** [ZefanCai/Open-Jev](https://huggingface.co/datasets/ZefanCai/Open-Jev) (CC0) and [LocalLLaMA/typed-decisions](https://huggingface.co/datasets/LocalLLaMA/typed-decisions) (Apache-2.0)
- **Generated home-appliance households:** the harness's generator, MIT, with seeds disjoint from the test households

Licences: CC BY 4.0, CC BY 3.0, CC0, Apache-2.0 and MIT.

The homelab packs and the answer-audit pack are synthetic (no real home data). Their soft labels
are answer distributions from **Gemma 4**:

- homelab packs: E4B and 26B-A4B, an ensemble of option logprobs;
- answer-audit: E4B.

Thank you to the Gemma team. Question wording for the public sets mirrors the Decision Index
adapters (MIT).

## Limitations

- **Decisions, not reasoning.** It doesn't know facts or do arithmetic. Use it to filter, route and rank cheap, frequent decisions, and send low-confidence ones to a bigger model.
- **Inherited judgement.** Distilled labels inherit the teacher's judgement and blind spots. Keep a human in the loop for anything safety-critical.
- **Speed.** On a modern laptop CPU it runs at a similar speed. The Coral makes it cheap to run next to Home Assistant on a Pi, an N100 or a NAS.

## License

Apache-2.0. See [NOTICE](https://github.com/SeraphimSerapis/reef/blob/main/NOTICE) for third-party credits.