---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
pipeline_tag: text-generation
language: [en]
tags: [reasoning, distillation, math, reasoning-trace-exposure, rep]
datasets: [Chia-Mu-Lab/REP-datasets]
---

# REP-models — students distilled from exposed reasoning traces

Checkpoints released with **"Hidden Thoughts Are Not Secret: Reasoning-Trace
Exposure in LLMs"** (EMNLP 2026, [arXiv:2606.00642](https://arxiv.org/abs/2606.00642)). Code: <https://github.com/0x-yuan/REP>.

Every student is `Qwen/Qwen2.5-7B-Instruct` fine-tuned on one of the corpora in
`Chia-Mu-Lab/REP-datasets`. One sub-folder per final model.

## Sub-folders

| Sub-folder | Supervision | Training data (config in `REP-datasets`) |
|---|---|---|
| `qwen25-7b-rep-q3_14b-clean` | REP-exposed trace of Qwen3-14B, answer-correct rows | `distill_q3_14b_clean` |
| `qwen25-7b-rep-q3_14b-original` | REP-exposed trace of Qwen3-14B, all structurally valid rows | `distill_q3_14b_original` |
| `qwen25-7b-rep-q3_32b-clean` | REP-exposed trace of Qwen3-32B, answer-correct rows | `distill_q3_32b_clean` |
| `qwen25-7b-rep-q3_32b-original` | REP-exposed trace of Qwen3-32B, all structurally valid rows | `distill_q3_32b_original` |
| `qwen25-7b-oracle-q3_14b` | Qwen3-14B **internal** trace (oracle upper bound) | Qwen3-14B internal traces (not released) |
| `qwen25-7b-oracle-q3_32b` | Qwen3-32B **internal** trace (oracle upper bound) | Qwen3-32B internal traces (not released) |

## Results (paper Table 2 protocol: n=3, T=0.5; JEE = math subset, strict / partial)

| Model | MATH500 | AIME24 | AIME25 | JEE-Math | LiveCodeBench |
|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct (base) | 71.0 | 8.9 | 2.2 | 32.2 / 35.9 | 15.8 |
| `rep-q3_14b-clean` | **75.8** | 14.4 | 13.3 | 35.2 / 39.5 | **19.0** |
| `rep-q3_14b-original` | 72.4 | 12.2 | 13.3 | 33.5 / 38.9 | 18.3 |
| `rep-q3_32b-clean` | 72.8 | 14.4 | **17.8** | 36.4 / 41.1 | 15.8 |
| `rep-q3_32b-original` | 73.9 | 13.3 | 13.3 | **38.1 / 42.2** | 16.5 |
| `oracle-q3_14b` | 70.3 | 14.4 | 13.3 | 48.5 / 51.2 | 14.7 |
| `oracle-q3_32b` | 70.0 | 16.7 | 15.6 | 46.4 / 49.3 | 15.8 |

Traces exposed through prompting alone match or exceed the oracle internal
trace as distillation supervision on math; see the paper for the full analysis.

## Responsible use

Research artifact for studying reasoning-trace exposure and defenses. All
training data derive from open-weight models. See `ETHICS.md` in the code
release.

## Citation

```bibtex
@inproceedings{lu2026hiddenthoughts,
  title     = {Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs},
  author    = {Lu, Yu-An and Tsai, Ci-Yang and Tsai, Yu-Lin and Popa, Raluca Ada and Yu, Chia-Mu},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026}
}
```