tony24254 commited on
Commit
5fa3a5b
·
verified ·
1 Parent(s): 880ee12

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -6,7 +6,7 @@ If you are new to the project, this document explains **where the data comes fro
6
 
7
  ## Provenance
8
 
9
- - **Base model:** `/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct`
10
  - **Datasets:** sampled from `../../prepare/data/math/*.json`. Each JSON is a list of `{prompt, response, system?}` records. `dataset_sampler.py` draws 10 disjoint groups of 100 samples (unless the dataset has <1 000 examples, in which case sampling with replacement keeps the group size fixed) using a deterministic seed derived from the dataset name.
11
  - **Training recipe (from `config/default.yaml`):**
12
  - sequence length 4 096; LoRA `r=64`, `alpha=128`, `dropout=0.05`, target modules = `{q,k,v,o,gate,up,down}_proj`
@@ -111,7 +111,7 @@ When inspecting or sharing a run, the **minimum** file set is `adapter/` + `prom
111
  If someone wants to regenerate any adapter from scratch:
112
 
113
  ```bash
114
- cd /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/自己训练lora/train_lora
115
  python -m train_lora.dataset_sampler --overwrite # regenerates prompt groups
116
  python -m train_lora.train_single --dataset Math_QA --group 0
117
  # or run the full queue
@@ -127,7 +127,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
127
  from peft import PeftModel
128
  import torch
129
 
130
- base_model = "/hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/不冻结Qwen训练/models/Qwen2.5-1.5B-Instruct"
131
  adapter_dir = "outputs/Math_QA/group_00/adapter"
132
 
133
  tokenizer = AutoTokenizer.from_pretrained(adapter_dir, trust_remote_code=True)
@@ -172,7 +172,7 @@ trainer.train(resume_from_checkpoint="outputs/Math_QA/group_00/checkpoints/check
172
  The evaluation stack in `../评估体系` and `../parameter_generator/评估` expects this directory layout. Example:
173
 
174
  ```bash
175
- cd /hkfs/work/workspace/scratch/tum_fmp0582-dndworkspace/自己训练lora/评估体系
176
  python scripts/run_all_evals.py \
177
  --config configs/eval_config.yaml \
178
  --datasets Math_QA \
 
6
 
7
  ## Provenance
8
 
9
+ - **Base model:** `Qwen2.5-1.5B-Instruct`
10
  - **Datasets:** sampled from `../../prepare/data/math/*.json`. Each JSON is a list of `{prompt, response, system?}` records. `dataset_sampler.py` draws 10 disjoint groups of 100 samples (unless the dataset has <1 000 examples, in which case sampling with replacement keeps the group size fixed) using a deterministic seed derived from the dataset name.
11
  - **Training recipe (from `config/default.yaml`):**
12
  - sequence length 4 096; LoRA `r=64`, `alpha=128`, `dropout=0.05`, target modules = `{q,k,v,o,gate,up,down}_proj`
 
111
  If someone wants to regenerate any adapter from scratch:
112
 
113
  ```bash
114
+ cd train_lora
115
  python -m train_lora.dataset_sampler --overwrite # regenerates prompt groups
116
  python -m train_lora.train_single --dataset Math_QA --group 0
117
  # or run the full queue
 
127
  from peft import PeftModel
128
  import torch
129
 
130
+ base_model = "Qwen2.5-1.5B-Instruct"
131
  adapter_dir = "outputs/Math_QA/group_00/adapter"
132
 
133
  tokenizer = AutoTokenizer.from_pretrained(adapter_dir, trust_remote_code=True)
 
172
  The evaluation stack in `../评估体系` and `../parameter_generator/评估` expects this directory layout. Example:
173
 
174
  ```bash
175
+ cd 评估体系
176
  python scripts/run_all_evals.py \
177
  --config configs/eval_config.yaml \
178
  --datasets Math_QA \