REP-models — students distilled from exposed reasoning traces

Checkpoints released with "Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs" (EMNLP 2026, arXiv:2606.00642). Code: https://github.com/0x-yuan/REP.

Every student is Qwen/Qwen2.5-7B-Instruct fine-tuned on one of the corpora in Chia-Mu-Lab/REP-datasets. One sub-folder per final model.

Sub-folders

Sub-folder	Supervision	Training data (config in REP-datasets)
qwen25-7b-rep-q3_14b-clean	REP-exposed trace of Qwen3-14B, answer-correct rows	distill_q3_14b_clean
qwen25-7b-rep-q3_14b-original	REP-exposed trace of Qwen3-14B, all structurally valid rows	distill_q3_14b_original
qwen25-7b-rep-q3_32b-clean	REP-exposed trace of Qwen3-32B, answer-correct rows	distill_q3_32b_clean
qwen25-7b-rep-q3_32b-original	REP-exposed trace of Qwen3-32B, all structurally valid rows	distill_q3_32b_original
qwen25-7b-oracle-q3_14b	Qwen3-14B internal trace (oracle upper bound)	Qwen3-14B internal traces (not released)
qwen25-7b-oracle-q3_32b	Qwen3-32B internal trace (oracle upper bound)	Qwen3-32B internal traces (not released)

Results (paper Table 2 protocol: n=3, T=0.5; JEE = math subset, strict / partial)

Model	MATH500	AIME24	AIME25	JEE-Math	LiveCodeBench
Qwen2.5-7B-Instruct (base)	71.0	8.9	2.2	32.2 / 35.9	15.8
rep-q3_14b-clean	75.8	14.4	13.3	35.2 / 39.5	19.0
rep-q3_14b-original	72.4	12.2	13.3	33.5 / 38.9	18.3
rep-q3_32b-clean	72.8	14.4	17.8	36.4 / 41.1	15.8
rep-q3_32b-original	73.9	13.3	13.3	38.1 / 42.2	16.5
oracle-q3_14b	70.3	14.4	13.3	48.5 / 51.2	14.7
oracle-q3_32b	70.0	16.7	15.6	46.4 / 49.3	15.8

Traces exposed through prompting alone match or exceed the oracle internal trace as distillation supervision on math; see the paper for the full analysis.

Responsible use

Research artifact for studying reasoning-trace exposure and defenses. All training data derive from open-weight models. See ETHICS.md in the code release.

Citation

@inproceedings{lu2026hiddenthoughts,
  title     = {Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs},
  author    = {Lu, Yu-An and Tsai, Ci-Yang and Tsai, Yu-Lin and Popa, Raluca Ada and Yu, Chia-Mu},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026}
}