README.md

metadata

title: min-spark-preview
emoji: ⚡
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.50.0
app_file: app.py
python_version: '3.12'
pinned: false
license: apache-2.0
tags:
  - text-generation
  - language-model
  - research
short_description: A 5M hybrid-looped LLM with effort levels

min-spark · Meiosis preview

A research demo for Meiosis — PICO release 01, a sub-10M-parameter decoder-only looped-hybrid language model trained from scratch on ~10B tokens of filtered fineweb-edu + finemath.

The one mechanic: a single weight-shared body block runs K times per token. This Space lets you set that effort (K = 2, 3, or 4) and prompt the model on a free CPU — ~20 tokens/sec, sized for showing the mechanic, not for throughput.

K-split

The loop-count trade-off is real (measured on the lm-eval harness):

• K = 2 — favors commonsense tasks (PIQA, HellaSwag)
• K = 3 — favors grammar (BLiMP), the default
• K = 4 — deeper grammar passes; diminishing returns on CPU

What's in here

• assets/meiosis.safetensors — the decay-p09 release candidate (23 MB, fp32)
• assets/tokenizer.json — byte-level BPE, vocab 4096 (ADR-0010)
• assets/meiosis.py — the model definition (torch-only, vendored)
• loader.py — model + tokenizer load, generation (mirrors PICO's infer.py)
• app.py — the Gradio interface

Provenance

The preview squeeze gate (2026-07-27) tested whether weight-space averaging of late trunk pins (SWA / EMA / Model Stock over p05–p09 + decay-p09) beat the final decay-p09 checkpoint on reserved-val NLL. It did not — averaging across mixed WSD phases regressed val perplexity — so the Space ships decay-p09 as-is. See PICO/results/post_train/preview_winner.json.

PICO is a monthly series of cheap, fully-trained-and-evaluated small models. Source: eclipse-senpai/PICO.