---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B
library_name: transformers
tags:
- memorization
- research-artifact
---

# qwen2.5-1.5b fine-tuned on The Stack (code) (Run 2)

Research artifact from *Predicting Memorization Before Fine-Tuning Language Models*.

This is `Qwen/Qwen2.5-1.5B` fine-tuned for one epoch on a 1M-sequence sample of **The Stack (code)** — the held-out
"Run 2" fine-tuning run used for evaluation in the paper. The trained classifiers in that work
**predict which sequences this fine-tuned model memorizes** from pre-fine-tuning (base-model)
features; i.e., the verbatim memorization of training data in this model is what those classifiers forecast.

> **Note.** This model was fine-tuned to study memorization and can reproduce portions of its
> fine-tuning data verbatim. It is released as a research artifact for reproducibility.