---
library_name: transformers
license: apache-2.0
---

# Tiny LLaMA

A 6.27M parameter LLaMA-style causal language model trained on TinyStories.

## Model Specifications

| Property | Value |
|----------|-------|
| Parameters | 6,270,624 |
| Layers | 6 |
| Attention Heads | 6 |
| Key/Value Heads | 6 |
| Head Dimension | 48 |
| Hidden Size | 288 |
| Intermediate Size | 768 |
| Vocabulary Size | 512 |
| Training Sequence Length | 256 |
| Data Type | float32 |

## Intended Use

- TinyStories-style text generation
- Educational examples
- Small-model research
- ASHA backend inference testing

## Out-of-Scope Uses

- Production deployments
- Knowledge-intensive tasks
- Long-form generation
- Multilingual generation

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("manojredhat/tiny-llama")
model = AutoModelForCausalLM.from_pretrained("manojredhat/tiny-llama")

inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```