README.md

---
title: MicroMixer-2
emoji: 🫪
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false
license: apache-2.0
short_description: Play with the four attention-free MicroMixer-2 checkpoints.
---

# MicroMixer-2 Discord-dialogue Playground

Interactive Gradio demo for the four attention-free, MLP-only language
models in [`llaa33219/micromixer-2`](https://huggingface.co/collections/llaa33219/micromixer-2).

| Variant | Max context | Params |
| --- | --- | --- |
| 100K | 64  | ~125K |
| 300K | 128 | ~431K |
| 500K | 128 | ~779K |
| 1M   | 4096| ~1.02M |

All checkpoints are byte-level (vocabulary = 256) and were trained on
[mookiezi/Discord-Dialogues](https://huggingface.co/datasets/mookiezi/Discord-Dialogues),
so prompts in `User:` / `Assistant:` format tend to work best.

## How it works

- The first time you pick a model, `app.py` downloads its `model.pt` from
  the Hub via `huggingface_hub.hf_hub_download` and caches it.
- The architecture (`src/model.py`) and the byte-level tokenizer
  (`src/tokenizer.py`) are vendored from
  [llaa33219/MicroMixer-2](https://github.com/llaa33219/MicroMixer-2).
- `MicroMixerConfig` is rebuilt from the exact hyperparameters listed on
  each model card, so `load_state_dict` matches the published checkpoints.

## Why Docker SDK

The HF Spaces "gradio" base image pins `gradio==4.44.0`, which has two
Python-3.13-specific problems out of the box:

- it imports `audioop` transitively (via `pydub`), but the module was
  removed from the stdlib in 3.13;
- its `oauth.py` still does `from huggingface_hub import HfFolder`,
  which was removed in `huggingface_hub` 1.0.

Pinning old versions to dodge those would mean a fragile `requirements.txt`.
Instead, this Space uses the `docker` SDK with a minimal Dockerfile that
installs the latest torch / gradio / huggingface_hub / audioop-lts from
scratch. No version pins anywhere - whatever the ecosystem ships today is
what runs.

## Files

```
.
├── app.py              # Gradio entry point
├── Dockerfile          # python:3.13-slim + deps + `python app.py`
├── requirements.txt    # no version pins, latest of everything
├── src/
│   ├── __init__.py
│   ├── model.py        # MicroMixer-2 V4 architecture
│   └── tokenizer.py    # ByteTokenizer (vocab=256)
└── README.md
```

## License

Apache 2.0, in line with the upstream project.