README.md

---
version: "2.0"
release_date: "2026-06-10"
license: other
license_name: sml-1.0
license_link: https://huggingface.co/sprapp/meeny/blob/main/LICENSE-MODELS.md
extra_gated_prompt: >-
  **meeny is free** for personal, research, education & evaluation — and for **commercial use under
  US$10,000/yr product revenue** ("free until you make money"). **Commercial use above that needs a
  paid license: US$49/year or US$99 lifetime, per product or domain.** 60% off (code `PPP60`) for
  low/lower-middle-income countries. Scale (> US$1M revenue or > 100K MAU) = custom license.
  Buy → https://www.sprapp.com/buy · questions → sprappcom@gmail.com.
  By requesting access you agree to the Sprapp Model License v1.0 (SML-1.0).
extra_gated_fields:
  Intended use:
    type: select
    options:
      - Personal / research / education
      - Commercial — under US$10K/yr revenue (free)
      - Commercial — over US$10K/yr revenue (license required)
      - Other
  Company (if commercial):
    type: text
  I agree to the Sprapp Model License v1.0 (SML-1.0) terms:
    type: checkbox
extra_gated_button_content: Agree & request access
datasets:
  - roneneldan/TinyStories
language:
  - en
tags:
  - tiny
  - tinystories
  - ternary
  - quantization-aware-training
  - wasm
  - on-device
  - sprapp
pipeline_tag: text-generation
---

# meeny — a 6.2M ternary story model (BPB 0.519)

**meeny** is a TinyStories generator with **ternary {-1, 0, +1} weights** (≈1.58-bit, BitNet-style
quantization-aware training) that runs **in a browser tab, offline, no GPU**. At 6.2M total parameters
and a ~7 MB file it reaches **bits-per-byte 0.519** on held-out TinyStories — substantially better than
[TinyStories-1M](https://huggingface.co/roneneldan/TinyStories-1M) (0.707) and better than our own
999K [eeny](https://huggingface.co/sprapp/eeny) (0.625), with noticeably cleaner words and grammar.

## Numbers (all measured)

| metric | meeny | eeny | TinyStories-1M |
|---|---|---|---|
| total params | 6,228,481 | 999,328 | ~3M |
| non-embedding params | 4,262,401 | — | ~1M |
| **bits-per-byte** (held-out val, lower=better) | **0.5186** | 0.625 | 0.707 |
| projection weights | **ternary {-1,0,+1}** | int8 | fp |
| speed (node WASM, single thread) | **343 tok/s** (2.9 ms/tok) | 1949 | — |
| file size (KNM1 v3) | **6.99 MB** | 1.76 MB | — |

Bits-per-byte normalizes cross-entropy by UTF-8 bytes — tokenizer-independent, so the comparison is fair.

## What it is

- Decoder-only transformer: dim 320, 4 layers, GQA (5 heads / 1 kv), SwiGLU, RoPE, tied embeddings,
  **6144-token BPE** tokenizer (the larger vocab removes the word-fragmentation seen in 4k-vocab tinies).
- **Ternary QAT**: projection weights {-1,0,+1} with per-tensor absmean scale (2-bit packed, 4 weights/byte),
  trained with a straight-through estimator — not post-hoc rounding. Embeddings f16, lm_head int8, norms fp.
- **Knowledge distillation** (KL T=2 + 0.5·CE) from a dim-576 / 10-layer full-precision teacher,
  3B tokens on 8×RTX 5090.
- Runs on the Sprapp Rust→WASM engine (`gemv_ternary` SIMD kernel) — the whole model generates in a
  browser tab, offline after first load.

## Sample (greedy, in-engine)

```
Once upon a time, there was a little  →  girl named Lily. She loved to play outside in the sunshine
  and run around with her friends. One day, she saw an orange ball on the ground ...
The dog ran to the                    →  park. He saw a big tree with many leaves and branches. The
  squirrel was very happy, but he did not know that there were some bees in the trees ...
```

## Files

| file | what |
|---|---|
| `model_meeny.knm` | ternary weights, KNM1 v3 (6.99 MB) — for the Sprapp WASM engine |
| `meeny_v2.pt` | PyTorch checkpoint (`{cfg, model}`, fp latents pre-ternarize) |
| `tokenizer_meeny6k.json` | 6144-vocab BPE tokenizer |

## Limitations

- Domain: TinyStories only (simple children's stories). Not a general LM, no world knowledge.
- Ternary + tiny: occasional repetition or odd turns; greedy decode is most stable, temp ≤ 0.8 suggested.

Part of the **Sprapp** project — offline on-device tiny LMs in the browser (eeny / meeny / miny family).
Trained on [TinyStories](https://huggingface.co/datasets/roneneldan/TinyStories) (Eldan & Li, 2023).

## License

**Sprapp Model License v1.0 (SML-1.0)** — free for personal, research, education, and evaluation.
**Commercial use is free under US$10K/yr product revenue** ("free until you make money");
above that: **US$49/yr (or $99 lifetime)** per product, 60% off in low/lower-middle-income countries (code `PPP60`).
Scale (> US$1M revenue or > 100K MAU) requires a custom license.
**Buy: https://www.sprapp.com/buy** · contact sprappcom@gmail.com. See `LICENSE-MODELS.md`.
Trained from scratch; TinyStories dataset attribution above.