README.md

---
title: min-spark-preview
emoji: ⚡
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: "5.50.0"
app_file: app.py
python_version: "3.12"
pinned: false
license: apache-2.0
tags:
  - text-generation
  - language-model
  - research
short_description: A 5M hybrid-looped LLM with effort levels
---

# min-spark · Meiosis preview

A research demo for **Meiosis** — PICO release 01, a sub-10M-parameter
decoder-only **looped-hybrid** language model trained from scratch on ~10B
tokens of filtered fineweb-edu + finemath.

The one mechanic: a single weight-shared body block runs **K times per token**.
This Space lets you set that effort (K = 2, 3, or 4) and prompt the model on a
free CPU — ~20 tokens/sec, sized for showing the mechanic, not for throughput.

## K-split

The loop-count trade-off is real (measured on the lm-eval harness):

- **K = 2** — favors commonsense tasks (PIQA, HellaSwag)
- **K = 3** — favors grammar (BLiMP), the default
- **K = 4** — deeper grammar passes; diminishing returns on CPU

## What's in here

- `assets/meiosis.safetensors` — the decay-p09 release candidate (23 MB, fp32)
- `assets/tokenizer.json` — byte-level BPE, vocab 4096 (ADR-0010)
- `assets/meiosis.py` — the model definition (torch-only, vendored)
- `loader.py` — model + tokenizer load, generation (mirrors PICO's `infer.py`)
- `app.py` — the Gradio interface

## Provenance

The preview squeeze gate (2026-07-27) tested whether weight-space averaging of
late trunk pins (SWA / EMA / Model Stock over p05–p09 + decay-p09) beat the
final decay-p09 checkpoint on reserved-val NLL. It did **not** — averaging across
mixed WSD phases regressed val perplexity — so the Space ships decay-p09 as-is.
See `PICO/results/post_train/preview_winner.json`.

PICO is a monthly series of cheap, fully-trained-and-evaluated small models.
Source: [eclipse-senpai/PICO](https://github.com/eclipse-senpai/PICO).