coder66 commited on
Commit
e451306
·
verified ·
1 Parent(s): 50d5244

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +22 -0
README.md ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-32B-Instruct
4
+ tags: [research-ideation, research-proposals, lora, sft, chain-of-thought, qwen2.5]
5
+ language: [en]
6
+ pipeline_tag: text-generation
7
+ ---
8
+
9
+ # proposal-cot-sft Qwen2.5-32B LoRA (V2)
10
+
11
+ LoRA adapter for Qwen2.5-32B-Instruct fine-tuned on `arxiv-proposal-cot-sft-32b-v2`
12
+ (928 CoT examples, `with_research_question` conditioning, 16k sequence length).
13
+
14
+ - **Recipe:** LoRA r=64, α=128, all-linear; lr 1e-5 cosine (warmup 0.03), wd 0.01; effective batch
15
+ 128 (8×H800); 1 epoch; bf16; verl FSDP SFT trainer. Train time ≈49 min.
16
+ - **Behavior:** generates `<thinking>` + a 10-tag XML research proposal from a reading-list prompt.
17
+ Schema-complete and reference-grounded on held-out papers.
18
+ - **Honest result:** on a 10-task implementation benchmark the SFT model matched but did not beat
19
+ base pass-rate (base 1/10 vs SFT 0/10; several raw metrics improved). Released as a documented
20
+ baseline for creativity-distillation research (the project's V3 direction).
21
+ - Apply with PEFT on top of `Qwen/Qwen2.5-32B-Instruct` (Apache-2.0), or use the optional merged
22
+ weights repo. Code + eval harness: https://github.com/XinghanLi66/autoresearch_idea_harness