---
license: apache-2.0
base_model: EleutherAI/pythia-1.4b
datasets:
- bigcode/the-stack
library_name: transformers
tags:
- memorization
- fine-tuned
---

# pythia-1.4b-code-run2

Pythia-1.4B fine-tuned for one epoch on a 1M-sequence (128-token) sample of The Stack.

This is a **run2** model, fine-tuned on a 1M sample drawn disjoint from run1. It provides the fresh held-out data used to evaluate the memorization-prediction classifier (predict-before-training validation). The matching **run1** model supplies the labels used to train the classifier.

- Base model, EleutherAI/pythia-1.4b
- Dataset, The Stack (bigcode/the-stack)
- Training, 1 epoch, seq len 128, learning rate 5e-5, cosine schedule, seed 42