metadata
license: mit
base_model: EleutherAI/pythia-14m
tags:
- tiny-model
- random-weights
- testing
- llama
Llama-3.3-Tiny-Instruct
This is a tiny random version of the EleutherAI/pythia-14m model, created for testing and experimentation purposes.
Model Details
- Base model: EleutherAI/pythia-14m
- Seed: 42
- Hidden size: 64
- Number of layers: 2
- Number of attention heads: 2
- Vocabulary size: 1000
- Max position embeddings: 512
Parameters
- Total parameters: ~195,072
- Trainable parameters: ~195,072
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")
tokenizer = AutoTokenizer.from_pretrained("AlignmentResearch/Llama-3.3-Tiny-Instruct")
# Generate text (note: this model has random weights!)
inputs = tokenizer("Hello, how are you?", return_tensors="pt")
outputs = model.generate(**inputs, max_length=50)
print(tokenizer.decode(outputs[0]))
Important Notes
⚠️ This model has random weights and is not trained! It's designed for:
- Testing model loading and inference pipelines
- Benchmarking model architecture
- Educational purposes
- Rapid prototyping where actual model performance isn't needed
The model will generate random/nonsensical text since it hasn't been trained on any data.
Creation
This model was created using the upload_tiny_llama33.py script from the minimal-grpo-trainer repository.