| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen2.5-7B-Instruct |
| pipeline_tag: text-generation |
| language: [en] |
| tags: [reasoning, distillation, math, reasoning-trace-exposure, rep] |
| datasets: [Chia-Mu-Lab/REP-datasets] |
| --- |
| |
| # REP-models — students distilled from exposed reasoning traces |
|
|
| Checkpoints released with **"Hidden Thoughts Are Not Secret: Reasoning-Trace |
| Exposure in LLMs"** (EMNLP 2026, [arXiv:2606.00642](https://arxiv.org/abs/2606.00642)). Code: <https://github.com/0x-yuan/REP>. |
|
|
| Every student is `Qwen/Qwen2.5-7B-Instruct` fine-tuned on one of the corpora in |
| `Chia-Mu-Lab/REP-datasets`. One sub-folder per final model. |
|
|
| ## Sub-folders |
|
|
| | Sub-folder | Supervision | Training data (config in `REP-datasets`) | |
| |---|---|---| |
| | `qwen25-7b-rep-q3_14b-clean` | REP-exposed trace of Qwen3-14B, answer-correct rows | `distill_q3_14b_clean` | |
| | `qwen25-7b-rep-q3_14b-original` | REP-exposed trace of Qwen3-14B, all structurally valid rows | `distill_q3_14b_original` | |
| | `qwen25-7b-rep-q3_32b-clean` | REP-exposed trace of Qwen3-32B, answer-correct rows | `distill_q3_32b_clean` | |
| | `qwen25-7b-rep-q3_32b-original` | REP-exposed trace of Qwen3-32B, all structurally valid rows | `distill_q3_32b_original` | |
| | `qwen25-7b-oracle-q3_14b` | Qwen3-14B **internal** trace (oracle upper bound) | Qwen3-14B internal traces (not released) | |
| | `qwen25-7b-oracle-q3_32b` | Qwen3-32B **internal** trace (oracle upper bound) | Qwen3-32B internal traces (not released) | |
|
|
| ## Results (paper Table 2 protocol: n=3, T=0.5; JEE = math subset, strict / partial) |
|
|
| | Model | MATH500 | AIME24 | AIME25 | JEE-Math | LiveCodeBench | |
| |---|---|---|---|---|---| |
| | Qwen2.5-7B-Instruct (base) | 71.0 | 8.9 | 2.2 | 32.2 / 35.9 | 15.8 | |
| | `rep-q3_14b-clean` | **75.8** | 14.4 | 13.3 | 35.2 / 39.5 | **19.0** | |
| | `rep-q3_14b-original` | 72.4 | 12.2 | 13.3 | 33.5 / 38.9 | 18.3 | |
| | `rep-q3_32b-clean` | 72.8 | 14.4 | **17.8** | 36.4 / 41.1 | 15.8 | |
| | `rep-q3_32b-original` | 73.9 | 13.3 | 13.3 | **38.1 / 42.2** | 16.5 | |
| | `oracle-q3_14b` | 70.3 | 14.4 | 13.3 | 48.5 / 51.2 | 14.7 | |
| | `oracle-q3_32b` | 70.0 | 16.7 | 15.6 | 46.4 / 49.3 | 15.8 | |
|
|
| Traces exposed through prompting alone match or exceed the oracle internal |
| trace as distillation supervision on math; see the paper for the full analysis. |
|
|
| ## Responsible use |
|
|
| Research artifact for studying reasoning-trace exposure and defenses. All |
| training data derive from open-weight models. See `ETHICS.md` in the code |
| release. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{lu2026hiddenthoughts, |
| title = {Hidden Thoughts Are Not Secret: Reasoning-Trace Exposure in LLMs}, |
| author = {Lu, Yu-An and Tsai, Ci-Yang and Tsai, Yu-Lin and Popa, Raluca Ada and Yu, Chia-Mu}, |
| booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)}, |
| year = {2026} |
| } |
| ``` |
|
|
