File size: 2,829 Bytes
1937932 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
pipeline_tag: text-generation
language: [en]
tags: [reasoning, distillation, math, reasoning-trace-exposure, rep]
datasets: [Chia-Mu-Lab/REP-datasets]
---
# REP-models — students distilled from exposed reasoning traces
Checkpoints released with **"Hidden Thoughts Are Not Secret: Reasoning-Trace
Exposure in LLMs"** (EMNLP 2026, [arXiv:2606.00642](https://arxiv.org/abs/2606.00642)). Code: <https://github.com/0x-yuan/REP>.
Every student is `Qwen/Qwen2.5-7B-Instruct` fine-tuned on one of the corpora in
`Chia-Mu-Lab/REP-datasets`. One sub-folder per final model.
## Sub-folders
| Sub-folder | Supervision | Training data (config in `REP-datasets`) |
|---|---|---|
| `qwen25-7b-rep-q3_14b-clean` | REP-exposed trace of Qwen3-14B, answer-correct rows | `distill_q3_14b_clean` |
| `qwen25-7b-rep-q3_14b-original` | REP-exposed trace of Qwen3-14B, all structurally valid rows | `distill_q3_14b_original` |
| `qwen25-7b-rep-q3_32b-clean` | REP-exposed trace of Qwen3-32B, answer-correct rows | `distill_q3_32b_clean` |
| `qwen25-7b-rep-q3_32b-original` | REP-exposed trace of Qwen3-32B, all structurally valid rows | `distill_q3_32b_original` |
| `qwen25-7b-oracle-q3_14b` | Qwen3-14B **internal** trace (oracle upper bound) | Qwen3-14B internal traces (not released) |
| `qwen25-7b-oracle-q3_32b` | Qwen3-32B **internal** trace (oracle upper bound) | Qwen3-32B internal traces (not released) |
## Results (paper Table 2 protocol: n=3, T=0.5; JEE = math subset, strict / partial)
| Model | MATH500 | AIME24 | AIME25 | JEE-Math | LiveCodeBench |
|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct (base) | 71.0 | 8.9 | 2.2 | 32.2 / 35.9 | 15.8 |
| `rep-q3_14b-clean` | **75.8** | 14.4 | 13.3 | 35.2 / 39.5 | **19.0** |
| `rep-q3_14b-original` | 72.4 | 12.2 | 13.3 | 33.5 / 38.9 | 18.3 |
| `rep-q3_32b-clean` | 72.8 | 14.4 | **17.8** | 36.4 / 41.1 | 15.8 |
| `rep-q3_32b-original` | 73.9 | 13.3 | 13.3 | **38.1 / 42.2** | 16.5 |
| `oracle-q3_14b` | 70.3 | 14.4 | 13.3 | 48.5 / 51.2 | 14.7 |
| `oracle-q3_32b` | 70.0 | 16.7 | 15.6 | 46.4 / 49.3 | 15.8 |
Traces exposed through prompting alone match or exceed the oracle internal
trace as distillation supervision on math; see the paper for the full analysis.
## Responsible use
Research artifact for studying reasoning-trace exposure and defenses. All
training data derive from open-weight models. See `ETHICS.md` in the code
release.
## Citation
```bibtex
@inproceedings{lu2026hiddenthoughts,
title = {Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs},
author = {Lu, Yu-An and Tsai, Ci-Yang and Tsai, Yu-Lin and Popa, Raluca Ada and Yu, Chia-Mu},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
year = {2026}
}
```
|