File size: 3,122 Bytes
8502d68 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
language:
- en
tags:
- daedalus
- cpu-inference
- lfm2
- hybrid
widget:
- text: "What is the capital of France?"
- text: "Explain photosynthesis in one sentence."
- text: "What is the difference between a CPU and a GPU?"
inference:
parameters:
max_new_tokens: 96
temperature: 0.8
top_p: 0.9
repetition_penalty: 1.15
---
# Daedalus-150M — Instruct
A 150M-parameter language model built for **CPU inference**. Full attention is
kept in only 6 of its 18 layers; the other 12 use short convolutions whose
memory is two timesteps wide however long the conversation gets. Decoding
therefore does not slow down as context grows.
Trained from scratch on 59.9B tokens, then instruction-tuned (SFT on
smol-smoltalk + one DPO round on UltraFeedback).
- GGUF builds and full checkpoints: [Unseen1980/daedalus-checkpoints](https://huggingface.co/Unseen1980/daedalus-checkpoints)
- Code and paper: [unseen1980/daedalus](https://github.com/unseen1980/daedalus)
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Unseen1980/daedalus-150m-instruct"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
messages = [{"role": "user", "content": "What is the capital of France?"}]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")
out = model.generate(inputs, max_new_tokens=96, temperature=0.8,
top_p=0.9, repetition_penalty=1.15, do_sample=True)
print(tok.decode(out[0], skip_special_tokens=True))
```
**Set `repetition_penalty`.** Without it this model can lock onto a word and
repeat it until it runs out of tokens.
For CPU deployment, use the 4-bit GGUF (102 MB) from the checkpoints repository
rather than these weights.
## Results
Five-task mean over HellaSwag, ARC-Easy, PIQA, OpenBookQA and WinoGrande, with
every peer re-scored on the same harness rather than quoted from its paper:
| Model | Training tokens | 5-task |
|---|---|---|
| **Daedalus-150M** | **59.9B** | **47.31** |
| MobileLLM-125M | 1T | 46.3 (published) |
| GPT-2 124M | — | 42.2 |
| OPT-125M | 180B | 42.1 |
| GPT-neo-125M | 300B | 41.9 |
| Pythia-160M | 300B | 41.0 |
Validation bits-per-byte 0.8685. CPU decode ~440 tokens/second, and **1.76×
faster than a same-size all-attention model at 2048 tokens of context** — an
advantage that grows with context rather than staying constant.
## Limitations
It is a 150M model. It writes fluent, plausible text and gets plenty of facts
wrong; the fair comparison is GPT-2 124M, which it beats. Short factual answers
and explanations work best. Open-ended creative writing drifts after a few
lines. English only, 2048-token context, single seed.
The 4-bit build costs about 6% perplexity — quantisation-aware training was
built but did not run. Roughly 48% of the convolution channels are inert and
cannot be pruned, and the 49,152-entry vocabulary is larger than this model size
warrants. All three are documented in the paper.
|