File size: 1,141 Bytes
82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 756e974 82332a9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
library_name: transformers
license: apache-2.0
---
# Tiny LLaMA
A 6.27M parameter LLaMA-style causal language model trained on TinyStories.
## Model Specifications
| Property | Value |
|----------|-------|
| Parameters | 6,270,624 |
| Layers | 6 |
| Attention Heads | 6 |
| Key/Value Heads | 6 |
| Head Dimension | 48 |
| Hidden Size | 288 |
| Intermediate Size | 768 |
| Vocabulary Size | 512 |
| Training Sequence Length | 256 |
| Data Type | float32 |
## Intended Use
- TinyStories-style text generation
- Educational examples
- Small-model research
- ASHA backend inference testing
## Out-of-Scope Uses
- Production deployments
- Knowledge-intensive tasks
- Long-form generation
- Multilingual generation
## Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("manojredhat/tiny-llama")
model = AutoModelForCausalLM.from_pretrained("manojredhat/tiny-llama")
inputs = tokenizer("Once upon a time", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=40, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```
|