---
license: llama3.2
base_model: meta-llama/Llama-3.2-1B
datasets:
- bigcode/the-stack
library_name: transformers
tags:
- memorization
- fine-tuned
---

# llama-3.2-1b-code-run1

Llama-3.2-1B fine-tuned for one epoch on a 1M-sequence (128-token) sample of The Stack.

This is a **run1** model. Extractable-memorization labels (k=50, l=50 greedy) from run1 are the labels used to train the memorization-prediction classifier. The matching **run2** model is fine-tuned on a disjoint 1M sample and provides the fresh held-out evaluation.

- Base model, meta-llama/Llama-3.2-1B
- Dataset, The Stack (bigcode/the-stack)
- Training, 1 epoch, seq len 128, learning rate 5e-5, cosine schedule, seed 42