metadata
license: mit
language:
- en
tags:
- shawarma
- character
- tiny-llm
- text-generation
- from-scratch
pipeline_tag: text-generation
HakimLM — 9M Parameter Shawarma Vendor Chat Model
A ~9M parameter LLM trained from scratch that talks like Hakim, a philosophical shawarma vendor. Built on the GuppyLM architecture — same vanilla transformer, different personality, trained on a fresh dataset.
This project exists to show that training your own language model is not magic. One small dataset, one Colab notebook, and you have a working LLM built from scratch — data generation, tokenizer, architecture, training loop, and inference.
Example
You> are you happy
Hakim> i have fire, meat, and people who are hungry. this is enough for today.
You> tell me something wise
Hakim> i wrap the same thing every day and it is never exactly the same. that is all i know about life.
You> don't you get bored
Hakim> do you get tired of breathing. no. you just breathe. it is like this for me.
You> it is raining
Hakim> rain comes. the fire stays. hunger does not check the weather.
Character
Hakim has been at his stand for as long as anyone can remember. He talks about the meat, the spices, the sauce. But somehow, by the end of the conversation, you've learned something about yourself.
- Short to medium sentences. Never rambling.
- Physical anchor first (bread, fire, knife), depth arrives naturally.
- Never preaches. States. Observes.
- Dry humour, very sparse.
Architecture
| Parameters | 8.7M |
| Type | Vanilla transformer (from scratch) |
| Layers | 6 |
| Hidden dim | 384 |
| Heads | 6 |
| FFN | 768 (ReLU) |
| Vocab | 4,096 (BPE) |
| Max sequence | 128 tokens |
| Norm | LayerNorm |
| Position | Learned embeddings |
| LM head | Weight-tied with embeddings |
No GQA, no RoPE, no SwiGLU, no early exit. As simple as it gets.
Training
- Data: 57K single-turn synthetic conversations across 38 topics (greeting, farewell, bread, meat, sauce, spices, fire, knife, hunger, patience, wisdom, customers, and more)
- Steps: 10,000
- Optimizer: AdamW (cosine LR schedule)
- No system prompt — personality is baked into the weights
Usage
from guppylm.inference import GuppyInference
engine = GuppyInference('checkpoints/best_model.pt', 'data/tokenizer.json')
r = engine.chat_completion([{'role': 'user', 'content': 'are you happy'}])
print(r['choices'][0]['message']['content'])
# i have fire, meat, and people who are hungry. this is enough for today.
Links
- Base architecture: github.com/arman-bd/guppylm
- Dataset: huggingface.co/datasets/hatoum/hakimlm-57k-shawarma
License
MIT