---
license: mit
base_model: meta-llama/Llama-3.3-70B-Instruct
tags:
- tiny-model
- random-weights
- testing
- llama
---

# Llama-3.3-Tiny-Instruct

This is a tiny random version of the meta-llama/Llama-3.3-70B-Instruct model, created for testing and experimentation purposes.

## Model Details

- **Base model**: meta-llama/Llama-3.3-70B-Instruct
- **Seed**: 42
- **Hidden size**: 256
- **Number of layers**: 12
- **Number of attention heads**: 4
- **Vocabulary size**: 128256
- **Max position embeddings**: 131072

## Parameters

- **Total parameters**: ~42,277,376
- **Trainable parameters**: ~42,277,376

## Usage

```python
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")
tokenizer = AutoTokenizer.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")

# Generate text (note: this model has random weights!)
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))
```

## Important Notes

⚠️ **This model has random weights and is not trained!** It's designed for:
- Testing model loading and inference pipelines
- Benchmarking model architecture
- Educational purposes
- Rapid prototyping where actual model performance isn't needed

The model will generate random/nonsensical text since it hasn't been trained on any data.

## Creation

This model was created using the `upload_tiny_llama.py` script from the minimal-grpo-trainer repository.