output = llm(
"Once upon a time,",
max_tokens=512,
echo=True
)
print(output)rb-nano
A 48M-parameter, GPT-2-style decoder-only transformer trained from scratch as part of the Leopard AI Model Suite. Small enough to run on CPU or any GPU; built as a learning/research model, not a production assistant.
Run it
With Ollama:
ollama run rafi-dev/rb-nano
With llama.cpp:
./llama-cli -m rb-nano-f16.gguf -p "user: hello\nai:"
What it is
rb-nano is a tiny chat model pretrained on web text and instruction-tuned for short, single- and multi-turn conversations. At 48M parameters it sits well below the knowledge capacity of mainstream models, so treat it as a fast, lightweight demonstrator rather than a factual reference.
Architecture
| Type | Decoder-only transformer (GPT-2 family) |
| Parameters | ~48M |
Embedding dim (d_model) |
512 |
| Layers | 10 |
| Attention heads | 8 |
| Context length | 1024 tokens |
| Position embeddings | Learned |
| Norm / activation | LayerNorm, GELU-tanh |
| Attention | Combined QKV, SDPA (flash) |
| Head | Weight-tied to token embeddings |
| Tokenizer | ByteLevel BPE, 32k vocab |
| Format | GGUF, f16 (gpt2 architecture) |
Training
- Pretrain — FineWeb-Edu (
sample-10BT), ~50M tokens. Final val loss ≈ 3.44. - Finetune — Alpaca, Alpaca-cleaned, CodeAlpaca-20k, Dolly-15k, and ShareGPT (full multi-turn threads, loss masked to assistant turns only). Final val loss ≈ 2.67.
Prompt format
The model is trained on a simple user: / ai: turn format (Ollama's chat template handles this automatically):
user: hello
ai: Hi there! How can I help you today?
user: what is python?
ai:
Recommended parameters
temperature 0.7
top_k 40
top_p 0.9
repeat_penalty 1.3
Limitations
- Knowledge. At 48M params the model has very limited factual knowledge and will confidently hallucinate (made-up libraries, wrong dates, etc.). It cannot be a reliable source of facts.
- Coherence. Good for short exchanges; longer or more technical answers drift.
- Scope. English-centric, 1024-token context. Best for demos, experimentation, and edge/CPU inference — not production use.
License / attribution
Released under CC BY-NC 4.0 (non-commercial, attribution required). The finetune mixes datasets with non-commercial terms (Alpaca, CodeAlpaca, ShareGPT — OpenAI-derived), so commercial use is not granted. Trained on publicly available datasets (FineWeb-Edu, Alpaca, Dolly, CodeAlpaca, ShareGPT); review each dataset's license before redistributing derived outputs.
Made with care
rb-nano was built by Rafi (13 years old) and Buddi (10 years old) — pretrained and finetuned from scratch on a single RTX 4070 (8 GB VRAM). It's a passion project: proof that a coherent little chat model can be trained end-to-end on consumer hardware.
If you enjoy it and want to support more experiments like this, you can buy us a coffee ☕. Thank you for trying rb-nano — we hope you like it.
rb-super might come
we plan to make rb-super 120 milion parameters
- Downloads last month
- 185
16-bit
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="leopard-Ai/rb-nano-GGUF", filename="rb-nano-f16.gguf", )