PRISM
PRISM is a small language model built and trained from scratch: its own tokenizer, its own architecture code, and no weights borrowed from any other model.
Try it in your browser: PRISM chat
Chat with it on your own computer (one command)
You only need Python 3 and NumPy, no GPU:
pip install numpy
python chat_prism.py
Download chat_prism.py first.
On its first run it downloads the brain (prism_brain.npz, ~560 MB) and then chats in your terminal.
The model
| Type | decoder-only transformer with a sparse mixture of experts |
| Size | 280M weights in total, ~110M used per token |
| Layers | 12 layers, width 768, 12 attention heads |
| Experts | 8 SwiGLU experts per layer, the top 2 chosen per token |
| Other | RoPE positions, QK-norm, RMSNorm |
| Context | 1,024 tokens |
| Tokenizer | PRISM's own byte-level BPE, 16,384 tokens |
| Training | ~5.6B tokens in ~6.5 hours on one H100, with JAX, then a short chat phase |
| Final held-out loss | 2.44 (pretraining) |
The brain file is plain NumPy (.npz, half precision) with the tokenizer inside, so it runs
the same on any computer, whatever GPU trained it.
Training data
Pretraining mix: FineWeb-Edu (50%), The Stack v3 (32%, permissively licensed code) and OpenWebMath (18%). The chat phase used human-written conversations and solutions only: OpenAssistant, Databricks Dolly 15k, GSM8K and the MATH dataset. No outputs from other AI models were used. Each dataset keeps its own license; see the dataset pages.
Limitations
This is an early, small model. It writes fluent text and code-shaped output, but it often makes up facts, gets math wrong and repeats itself. It isn't safety-tuned. Don't rely on what it says.
Files
prism_brain.npz: the chat-tuned brain (use this)prism_brain_pretrain_final.npz: the brain after pretraining only (text completion)prism_brain_train_state.pkl: full-precision weights and optimizer state, for training it furtherchat_prism.py: one-file chat program