Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen2.5-32B-Instruct
|
| 4 |
+
tags: [research-ideation, research-proposals, lora, sft, chain-of-thought, qwen2.5]
|
| 5 |
+
language: [en]
|
| 6 |
+
pipeline_tag: text-generation
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# proposal-cot-sft Qwen2.5-32B LoRA (V2)
|
| 10 |
+
|
| 11 |
+
LoRA adapter for Qwen2.5-32B-Instruct fine-tuned on `arxiv-proposal-cot-sft-32b-v2`
|
| 12 |
+
(928 CoT examples, `with_research_question` conditioning, 16k sequence length).
|
| 13 |
+
|
| 14 |
+
- **Recipe:** LoRA r=64, α=128, all-linear; lr 1e-5 cosine (warmup 0.03), wd 0.01; effective batch
|
| 15 |
+
128 (8×H800); 1 epoch; bf16; verl FSDP SFT trainer. Train time ≈49 min.
|
| 16 |
+
- **Behavior:** generates `<thinking>` + a 10-tag XML research proposal from a reading-list prompt.
|
| 17 |
+
Schema-complete and reference-grounded on held-out papers.
|
| 18 |
+
- **Honest result:** on a 10-task implementation benchmark the SFT model matched but did not beat
|
| 19 |
+
base pass-rate (base 1/10 vs SFT 0/10; several raw metrics improved). Released as a documented
|
| 20 |
+
baseline for creativity-distillation research (the project's V3 direction).
|
| 21 |
+
- Apply with PEFT on top of `Qwen/Qwen2.5-32B-Instruct` (Apache-2.0), or use the optional merged
|
| 22 |
+
weights repo. Code + eval harness: https://github.com/XinghanLi66/autoresearch_idea_harness
|