Daedalus-150M: A Convolution-Attention Hybrid Designed for CPU Inference
Abstract
A small hybrid language model using sparse attention and short convolutions achieves faster CPU inference and better benchmark scores than larger conventional models despite training on far less data.
Small language models are usually built like large ones and then squeezed onto a CPU afterwards. We did the opposite: we fixed the target first, one user, one token at a time, 4-bit weights, ordinary CPU, and chose the architecture to suit it. The result keeps full attention in only 6 of its 18 blocks. The other 12 use short convolutions whose memory is two timesteps wide no matter how long the conversation gets, so two thirds of the network never re-reads a growing cache. Trained from scratch on 59.9B tokens, the model scores 47.31 on a five-task benchmark against a bar of 42.20 that was fixed before training began. It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-neo-125M, all trained on three to six times more data, and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens. Validation bits-per-byte is 0.8685. To check the architecture rather than the training recipe, we trained a conventional all-attention model of the same size on the same data, and wrote down the winning condition before scoring either. The hybrid won the chosen quality metric by 0.81%, matched it on downstream tasks, produced a 6.3% smaller 4-bit file, and decoded 1.76x faster at 2048 tokens of context, 2.08x against an external model of similar size. In every measurement the speed advantage is near zero at an empty context and grows with length, which is what the mechanism predicts and what a merely leaner model would not show. A simple bandwidth calculation predicts only 1.17x, so memory volume alone does not explain the gap. We also report what did not work: an unmitigated 4-bit quality cost, roughly half the convolution channels ending up inert and impossible to remove, and a vocabulary larger than this model size warrants.
Community
Daedalus-150M is a 160M-parameter language model designed backwards from its deployment target: one user, one token at a time, 4-bit weights, an ordinary CPU. Only 6 of its 18 blocks use full attention; the other 12 use short depthwise convolutions whose recurrent state is two timesteps wide no matter how long the context gets, so two thirds of the network never re-reads a growing KV cache.
Trained from scratch on 59.9B tokens, it scores 47.31 on a five-task benchmark (HellaSwag, ARC-Easy, PIQA, OpenBookQA, WinoGrande) against a bar of 42.20 that was fixed before training began. It beats GPT-2 124M, Pythia-160M, OPT-125M and GPT-Neo-125M — all trained on 3–6× more data — and exceeds MobileLLM-125M's published score despite that model seeing a trillion tokens. Validation bits-per-byte is 0.8685.
To test the architecture rather than the training recipe, a parameter-matched all-attention twin was trained on identical data, with the winning condition written down before either arm was scored. The hybrid won the pre-registered quality metric by 0.81%, matched the twin on downstream tasks, produced a 6.3% smaller 4-bit file, and decoded 1.76× faster at 2048 tokens of context — 2.08× against an external 135M peer. In every measurement the speed advantage is near zero at empty context and grows with depth, which is what the mechanism predicts and what a merely leaner model would not show. A bandwidth-only cost model predicts just 1.17×, so memory volume alone does not explain the gap.
The paper also reports what did not work: an unmitigated 4-bit quality cost after quantisation-aware training failed on its first step, roughly half the convolution channels ending up inert and impossible to prune under the stock runtime's shape checks, and a vocabulary larger than this model size warrants.
Model: https://huggingface.co/Unseen1980/daedalus-150m-instruct
Checkpoints: https://huggingface.co/Unseen1980/daedalus-checkpoints
Code: https://github.com/unseen1980/daedalus
Fixing the target before picking the architecture is the right call — most of these hybrids read like the authors picked a fancy mix and then hunted for a benchmark that flatters it. The cache-bounded convolutions are genuinely clever, but I want numbers I can feel: token/s on a specific CPU, with a real batch size, not a score delta. 47.31 vs 42.20 tells me the model is better at the benchmark, not that it's faster where I'd actually run it. What's the latency at batch 1 on, say, an i5 or an EPYC core? That's the number that decides whether I swap my current setup. Until I see that, the cleverness stays theoretical.
Get this paper in your agent:
hf papers read 2608.20210 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper