metadata
license: apache-2.0
base_model: Qwen/Qwen2.5-7B-Instruct
pipeline_tag: text-generation
language:
- en
tags:
- reasoning
- distillation
- math
- reasoning-trace-exposure
- rep
datasets:
- Chia-Mu-Lab/REP-datasets
REP-models — students distilled from exposed reasoning traces
Checkpoints released with "Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs" (EMNLP 2026, arXiv:2606.00642). Code: https://github.com/0x-yuan/REP.
Every student is Qwen/Qwen2.5-7B-Instruct fine-tuned on one of the corpora in
Chia-Mu-Lab/REP-datasets. One sub-folder per final model.
Sub-folders
| Sub-folder | Supervision | Training data (config in REP-datasets) |
|---|---|---|
qwen25-7b-rep-q3_14b-clean |
REP-exposed trace of Qwen3-14B, answer-correct rows | distill_q3_14b_clean |
qwen25-7b-rep-q3_14b-original |
REP-exposed trace of Qwen3-14B, all structurally valid rows | distill_q3_14b_original |
qwen25-7b-rep-q3_32b-clean |
REP-exposed trace of Qwen3-32B, answer-correct rows | distill_q3_32b_clean |
qwen25-7b-rep-q3_32b-original |
REP-exposed trace of Qwen3-32B, all structurally valid rows | distill_q3_32b_original |
qwen25-7b-oracle-q3_14b |
Qwen3-14B internal trace (oracle upper bound) | Qwen3-14B internal traces (not released) |
qwen25-7b-oracle-q3_32b |
Qwen3-32B internal trace (oracle upper bound) | Qwen3-32B internal traces (not released) |
Results (paper Table 2 protocol: n=3, T=0.5; JEE = math subset, strict / partial)
| Model | MATH500 | AIME24 | AIME25 | JEE-Math | LiveCodeBench |
|---|---|---|---|---|---|
| Qwen2.5-7B-Instruct (base) | 71.0 | 8.9 | 2.2 | 32.2 / 35.9 | 15.8 |
rep-q3_14b-clean |
75.8 | 14.4 | 13.3 | 35.2 / 39.5 | 19.0 |
rep-q3_14b-original |
72.4 | 12.2 | 13.3 | 33.5 / 38.9 | 18.3 |
rep-q3_32b-clean |
72.8 | 14.4 | 17.8 | 36.4 / 41.1 | 15.8 |
rep-q3_32b-original |
73.9 | 13.3 | 13.3 | 38.1 / 42.2 | 16.5 |
oracle-q3_14b |
70.3 | 14.4 | 13.3 | 48.5 / 51.2 | 14.7 |
oracle-q3_32b |
70.0 | 16.7 | 15.6 | 46.4 / 49.3 | 15.8 |
Traces exposed through prompting alone match or exceed the oracle internal trace as distillation supervision on math; see the paper for the full analysis.
Responsible use
Research artifact for studying reasoning-trace exposure and defenses. All
training data derive from open-weight models. See ETHICS.md in the code
release.
Citation
@inproceedings{lu2026hiddenthoughts,
title = {Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs},
author = {Lu, Yu-An and Tsai, Ci-Yang and Tsai, Yu-Lin and Popa, Raluca Ada and Yu, Chia-Mu},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
year = {2026}
}