BulmaX-ASM Baseline
A 2,067,130,880-parameter decoder-only transformer whose entire forward pass, backward pass and optimizer step run on hand-written x86-64 NASM kernels (AVX2+FMA), driven by a pure C runtime with no Python in the training loop.
Deliberately plain: no MoE, no coherence graph, no liquid network, no RSM, no
complex/quantum attention, no low-rank factorization. It exists as a clean CPU
control alongside the main BulmaX multimodal run, on the same tokenizer
(bulma_sp.model) for data comparability.
| dim | 2560 |
| layers | 24 |
| heads | 20 (head_dim 128) |
| SwiGLU hidden | 6912 |
| vocab | 32000 |
| max seq len | 1024 |
| embeddings | untied |
| params | 2,067,130,880 |
Checkpoint format
Raw binary BULMAXCP v1 -- not safetensors, not pickle. Fixed 64-byte header,
64-byte directory entries, 64-byte-aligned float32 blobs, so a C or assembly
loader can mmap it and walk the directory with no JSON parser and no Python.
Files hold weights plus both AdamW moments and the absolute step counter.