---
license: mit
language:
- en
pipeline_tag: text-generation
tags:
- gpt2
- tiny
- transformer
- experimental
- humor
- text-generation
- whirlwindai
new_version: WhirlwindAI/NanoZephyr
---

<div align="center">

<img src="https://readme-typing-svg.demolab.com?font=Space+Grotesk&weight=700&size=26&duration=1800&pause=900&color=A855F7&center=true&vCenter=true&width=820&lines=TinyZephyr;1%2C272+Parameters.;Technically+an+LLM.;Powered+by+Questionable+Engineering.;Runs+Before+You+Finish+Blinking." />

<br>

<img src="https://img.shields.io/badge/Parameters-1,272-8B5CF6?style=for-the-badge">
<img src="https://img.shields.io/badge/Architecture-GPT--2-A855F7?style=for-the-badge">
<img src="https://img.shields.io/badge/Status-Experimental-7C3AED?style=for-the-badge">
<img src="https://img.shields.io/badge/Humor-Included-6366F1?style=for-the-badge">

<br><br>

<img src="https://capsule-render.vercel.app/api?type=rounded&height=190&text=TinyZephyr&fontColor=ffffff&fontSize=48&animation=twinkling&color=0:8B5CF6,50:A855F7,100:2563EB"/>

</div>

---

# The Idea

<div align="center">

<table width="92%">
<tr>
<td align="center">

## Bigger isn't always better.

TinyZephyr explores the opposite extreme.

Instead of billions of parameters, this model asks one very important question:

**How ridiculously small can a transformer become before it completely loses its mind?**

The answer is... surprisingly entertaining.

</td>
</tr>
</table>

</div>

---

# Why?

Nobody needed this.

Nobody requested it.

Nobody funded it.

Yet somehow...

**TinyZephyr exists.**

It was built purely as an experiment to explore the lower limits of transformer architectures while proving that even microscopic language models deserve beautiful documentation.

---

# Specifications

| Property | Value |
|-----------|-------|
| Parameters | **1,272** |
| Architecture | GPT-2 |
| Layers | 1 |
| Attention Heads | 1 |
| Embedding Size | 8 |
| Context Length | 32 |
| Vocabulary | 50 Tokens |
| Model Size | ~25 KB |
| Training Time | ~4 Minutes (CPU) |

---

# Benchmark

| Task | Result |
|------|--------|
| Write Python | ❌ |
| Solve Math | ❌ |
| Explain Physics | ❌ |
| Generate Gibberish | ✅ |
| Exist | ✅ |

---

# Quick Start

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("WhirlwindAI/TinyZephyr")
model = AutoModelForCausalLM.from_pretrained("WhirlwindAI/TinyZephyr")

prompt = "The meaning of life is"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    do_sample=True,
    temperature=1.6,
    max_length=32
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))
````

Possible output:

```
The meaning of life is...
xqw fjczqnv lpoqa yv
```

Beautiful.

---

# Example Conversation

**You**

> Write a poem about space.

**TinyZephyr**

> moon potato quantum fish sandwich

Mission accomplished.

---

# System Requirements

CPU

> Yes.

GPU

> Optional.

RAM

> If your browser opens, you're probably fine.

Storage

> Less than most PNG files.

---

# Frequently Asked Questions

### Is this useful?

Not particularly.

### Is this serious research?

Surprisingly... yes.

### Can it replace ChatGPT?

Only if your expectations are extremely flexible.

### Why did you build this?

Curiosity.

And because somebody had to.

---

# Awards

🏆 Fastest Model To Finish Inference

🥇 Most Parameters Removed Without Deleting Everything

🥈 Best Random Sentence Generator

🥉 Self-Proclaimed Champion of Tiny AI

---

# Limitations

TinyZephyr was never trained to be helpful.

It doesn't know facts.

It doesn't reason.

It doesn't write code.

It mostly produces beautifully random nonsense.

And that's exactly what it was designed to do.

---

# License

MIT

Use it.

Benchmark it.

Laugh at it.

Make it even smaller.

---

<div align="center">

### Built by WhirlwindAI

*Sometimes the best experiments begin with terrible ideas.*

<br>

<img src="https://capsule-render.vercel.app/api?type=waving&height=120&section=footer&color=0:8B5CF6,100:2563EB"/>

</div>