Update README.md
Browse files
README.md
CHANGED
|
@@ -6,7 +6,7 @@ If you are new to the project, this document explains **where the data comes fro
|
|
| 6 |
|
| 7 |
## Provenance
|
| 8 |
|
| 9 |
-
- **Base model:**
|
| 10 |
- **Datasets:** sampled from `../../prepare/data/math/*.json`. Each JSON is a list of `{prompt, response, system?}` records. `dataset_sampler.py` draws 10 disjoint groups of 100 samples (unless the dataset has <1 000 examples, in which case sampling with replacement keeps the group size fixed) using a deterministic seed derived from the dataset name.
|
| 11 |
- **Training recipe (from `config/default.yaml`):**
|
| 12 |
- sequence length 4 096; LoRA `r=64`, `alpha=128`, `dropout=0.05`, target modules = `{q,k,v,o,gate,up,down}_proj`
|
|
@@ -111,7 +111,7 @@ When inspecting or sharing a run, the **minimum** file set is `adapter/` + `prom
|
|
| 111 |
If someone wants to regenerate any adapter from scratch:
|
| 112 |
|
| 113 |
```bash
|
| 114 |
-
cd
|
| 115 |
python -m train_lora.dataset_sampler --overwrite # regenerates prompt groups
|
| 116 |
python -m train_lora.train_single --dataset Math_QA --group 0
|
| 117 |
# or run the full queue
|
|
@@ -127,7 +127,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
| 127 |
from peft import PeftModel
|
| 128 |
import torch
|
| 129 |
|
| 130 |
-
base_model = "
|
| 131 |
adapter_dir = "outputs/Math_QA/group_00/adapter"
|
| 132 |
|
| 133 |
tokenizer = AutoTokenizer.from_pretrained(adapter_dir, trust_remote_code=True)
|
|
@@ -172,7 +172,7 @@ trainer.train(resume_from_checkpoint="outputs/Math_QA/group_00/checkpoints/check
|
|
| 172 |
The evaluation stack in `../评估体系` and `../parameter_generator/评估` expects this directory layout. Example:
|
| 173 |
|
| 174 |
```bash
|
| 175 |
-
cd
|
| 176 |
python scripts/run_all_evals.py \
|
| 177 |
--config configs/eval_config.yaml \
|
| 178 |
--datasets Math_QA \
|
|
|
|
| 6 |
|
| 7 |
## Provenance
|
| 8 |
|
| 9 |
+
- **Base model:** `Qwen2.5-1.5B-Instruct`
|
| 10 |
- **Datasets:** sampled from `../../prepare/data/math/*.json`. Each JSON is a list of `{prompt, response, system?}` records. `dataset_sampler.py` draws 10 disjoint groups of 100 samples (unless the dataset has <1 000 examples, in which case sampling with replacement keeps the group size fixed) using a deterministic seed derived from the dataset name.
|
| 11 |
- **Training recipe (from `config/default.yaml`):**
|
| 12 |
- sequence length 4 096; LoRA `r=64`, `alpha=128`, `dropout=0.05`, target modules = `{q,k,v,o,gate,up,down}_proj`
|
|
|
|
| 111 |
If someone wants to regenerate any adapter from scratch:
|
| 112 |
|
| 113 |
```bash
|
| 114 |
+
cd train_lora
|
| 115 |
python -m train_lora.dataset_sampler --overwrite # regenerates prompt groups
|
| 116 |
python -m train_lora.train_single --dataset Math_QA --group 0
|
| 117 |
# or run the full queue
|
|
|
|
| 127 |
from peft import PeftModel
|
| 128 |
import torch
|
| 129 |
|
| 130 |
+
base_model = "Qwen2.5-1.5B-Instruct"
|
| 131 |
adapter_dir = "outputs/Math_QA/group_00/adapter"
|
| 132 |
|
| 133 |
tokenizer = AutoTokenizer.from_pretrained(adapter_dir, trust_remote_code=True)
|
|
|
|
| 172 |
The evaluation stack in `../评估体系` and `../parameter_generator/评估` expects this directory layout. Example:
|
| 173 |
|
| 174 |
```bash
|
| 175 |
+
cd 评估体系
|
| 176 |
python scripts/run_all_evals.py \
|
| 177 |
--config configs/eval_config.yaml \
|
| 178 |
--datasets Math_QA \
|