takayosh commited on
Commit
5727d2d
·
verified ·
1 Parent(s): f134b8d

Upload final merged Qwen2.5-7B-Instruct ALF+DBB model

Browse files
README.md CHANGED
@@ -25,7 +25,7 @@ This repository provides a **merged full model** based on
25
 
26
  1. Train LoRA adapter on ALFWorld
27
  2. Train LoRA adapter on DBBench
28
- 3. Merge adapters using `ties` (density=0.3)
29
  4. Apply additional stabilization fine-tuning (LoRA)
30
  5. Merge final adapter into base model
31
 
@@ -35,11 +35,11 @@ This repository contains **full merged weights (no adapter required)**.
35
 
36
  - Base model: Qwen/Qwen2.5-7B-Instruct
37
  - Merge method: ties
38
- - Merge density: 0.3
39
  - Final stage epochs: 1
40
- - Learning rate: 2e-05
41
- - Final LoRA: r=64, alpha=128
42
- - Max sequence length: 3072
43
 
44
 
45
  ## Datasets
@@ -67,7 +67,7 @@ model = AutoModelForCausalLM.from_pretrained(
67
 
68
  ## Sources & Terms (IMPORTANT)
69
 
70
- Training data:
71
  - u-10bei/sft_alfworld_trajectory_dataset_v5
72
  - u-10bei/dbbench_sft_dataset_react_v4
73
 
 
25
 
26
  1. Train LoRA adapter on ALFWorld
27
  2. Train LoRA adapter on DBBench
28
+ 3. Merge adapters using `ties` (density=0.1)
29
  4. Apply additional stabilization fine-tuning (LoRA)
30
  5. Merge final adapter into base model
31
 
 
35
 
36
  - Base model: Qwen/Qwen2.5-7B-Instruct
37
  - Merge method: ties
38
+ - Merge density: 0.1
39
  - Final stage epochs: 1
40
+ - Learning rate: 1e-05
41
+ - Final LoRA: r=16, alpha=16
42
+ - Max sequence length: 2024
43
 
44
 
45
  ## Datasets
 
67
 
68
  ## Sources & Terms (IMPORTANT)
69
 
70
+ Training data:
71
  - u-10bei/sft_alfworld_trajectory_dataset_v5
72
  - u-10bei/dbbench_sft_dataset_react_v4
73
 
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e1a3c9723a6d1f95b40c057f642ab5f6425b628202913170256b92b38604c070
3
  size 3945426872
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:117decf3859109ba1183684f69bb6669ec40ab974390dfb5498c2fda237a76ac
3
  size 3945426872
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d4ca249210e7b6756c49c895098255899d74b6b045879af41664d5bbc984accf
3
  size 3864726352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:349b48a2c1f37c32756e4b550df0494cc5e4e4eb15073be9b5bacef54e90d379
3
  size 3864726352
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7aaa15a392b451141dc7f0b4c4b0208393f0a71119ee5c9d855da19c7a73c84a
3
  size 3864726408
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4b483225fbd2b6d514861bb04185ba141bdaf5bd5808c3aba95dbf2b4a7c127
3
  size 3864726408
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:be55353751ad461c5ba0bb1979e1a96a95ce2f43cf86b98ab88583cd20bd3f81
3
  size 3556392240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:37498e609b53015d199760e5921bbaa344fd58251ea5063f0a3a365195183630
3
  size 3556392240