Hugging Face's logo

dalyaff
/
phi2-QA-Arabic-phi

PEFT
Safetensors
phi
Generated from Trainer
custom_code
Model card Files
xet
Community
dalyaff commited on
Commit
5114e71
·
verified ·
1 Parent(s): 3768036

End of training

Browse files
Files changed (4) hide show
  1. README.md +32 -47
  2. adapter_config.json +2 -2
  3. adapter_model.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -5,7 +5,7 @@ library_name: peft
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
- - khalidalt/tydiqa-goldp
9
  base_model: microsoft/phi-2
10
  model-index:
11
  - name: phi-2
@@ -17,9 +17,9 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  # phi-2
19
 
20
- This model is a fine-tuned version of [microsoftl](https://huggingface.co/microsoftl) on the khalidalt/tydiqa-goldp dataset.
21
  It achieves the following results on the evaluation set:
22
- - Loss: 1.1016
23
 
24
  ## Model description
25
 
@@ -39,66 +39,51 @@ More information needed
39
 
40
  The following hyperparameters were used during training:
41
  - learning_rate: 2.5e-05
42
- - train_batch_size: 1
43
  - eval_batch_size: 8
44
  - seed: 42
45
  - gradient_accumulation_steps: 4
46
- - total_train_batch_size: 4
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
  - lr_scheduler_warmup_steps: 5
50
- - training_steps: 2000
51
 
52
  ### Training results
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:-----:|:----:|:---------------:|
56
- | 1.7708 | 0.02 | 50 | 1.5184 |
57
- | 1.4903 | 0.04 | 100 | 1.2735 |
58
- | 1.3474 | 0.06 | 150 | 1.2167 |
59
- | 1.3178 | 0.08 | 200 | 1.1864 |
60
- | 1.2963 | 0.1 | 250 | 1.1750 |
61
- | 1.2671 | 0.12 | 300 | 1.1676 |
62
- | 1.2484 | 0.14 | 350 | 1.1597 |
63
- | 1.2797 | 0.16 | 400 | 1.1546 |
64
- | 1.3196 | 0.18 | 450 | 1.1495 |
65
- | 1.2881 | 0.2 | 500 | 1.1435 |
66
- | 1.2389 | 0.22 | 550 | 1.1416 |
67
- | 1.2489 | 0.24 | 600 | 1.1371 |
68
- | 1.2223 | 0.26 | 650 | 1.1339 |
69
- | 1.2012 | 0.28 | 700 | 1.1307 |
70
- | 1.2285 | 0.3 | 750 | 1.1285 |
71
- | 1.255 | 0.32 | 800 | 1.1251 |
72
- | 1.2739 | 0.34 | 850 | 1.1229 |
73
- | 1.2412 | 0.36 | 900 | 1.1217 |
74
- | 1.2094 | 0.38 | 950 | 1.1204 |
75
- | 1.246 | 0.4 | 1000 | 1.1202 |
76
- | 1.1737 | 0.42 | 1050 | 1.1161 |
77
- | 1.2427 | 0.44 | 1100 | 1.1144 |
78
- | 1.2235 | 0.46 | 1150 | 1.1131 |
79
- | 1.2301 | 0.48 | 1200 | 1.1119 |
80
- | 1.1854 | 0.5 | 1250 | 1.1111 |
81
- | 1.1949 | 0.52 | 1300 | 1.1094 |
82
- | 1.243 | 0.54 | 1350 | 1.1088 |
83
- | 1.2121 | 0.56 | 1400 | 1.1081 |
84
- | 1.2124 | 0.58 | 1450 | 1.1081 |
85
- | 1.2065 | 0.6 | 1500 | 1.1061 |
86
- | 1.2357 | 0.62 | 1550 | 1.1058 |
87
- | 1.2253 | 0.64 | 1600 | 1.1050 |
88
- | 1.1751 | 0.66 | 1650 | 1.1034 |
89
- | 1.2171 | 0.68 | 1700 | 1.1042 |
90
- | 1.2091 | 0.7 | 1750 | 1.1038 |
91
- | 1.2111 | 0.72 | 1800 | 1.1027 |
92
- | 1.1808 | 0.74 | 1850 | 1.1023 |
93
- | 1.1233 | 0.76 | 1900 | 1.1020 |
94
- | 1.2327 | 0.78 | 1950 | 1.1020 |
95
- | 1.1534 | 0.8 | 2000 | 1.1016 |
96
 
97
 
98
  ### Framework versions
99
 
100
  - PEFT 0.7.2.dev0
101
- - Transformers 4.37.0.dev0
102
  - Pytorch 2.1.0+cu121
103
  - Datasets 2.16.1
104
  - Tokenizers 0.15.0
 
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
+ - dalyaff/darebah
9
  base_model: microsoft/phi-2
10
  model-index:
11
  - name: phi-2
 
17
 
18
  # phi-2
19
 
20
+ This model is a fine-tuned version of [microsoftl](https://huggingface.co/microsoftl) on the dalyaff/darebah dataset.
21
  It achieves the following results on the evaluation set:
22
+ - Loss: 0.7778
23
 
24
  ## Model description
25
 
 
39
 
40
  The following hyperparameters were used during training:
41
  - learning_rate: 2.5e-05
42
+ - train_batch_size: 2
43
  - eval_batch_size: 8
44
  - seed: 42
45
  - gradient_accumulation_steps: 4
46
+ - total_train_batch_size: 8
47
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
48
  - lr_scheduler_type: linear
49
  - lr_scheduler_warmup_steps: 5
50
+ - training_steps: 2500
51
 
52
  ### Training results
53
 
54
  | Training Loss | Epoch | Step | Validation Loss |
55
  |:-------------:|:-----:|:----:|:---------------:|
56
+ | 1.1134 | 0.89 | 100 | 1.0092 |
57
+ | 0.8768 | 1.78 | 200 | 0.8800 |
58
+ | 0.7644 | 2.67 | 300 | 0.8329 |
59
+ | 0.7516 | 3.56 | 400 | 0.8081 |
60
+ | 0.6618 | 4.44 | 500 | 0.7909 |
61
+ | 0.6373 | 5.33 | 600 | 0.7845 |
62
+ | 0.6154 | 6.22 | 700 | 0.7688 |
63
+ | 0.6056 | 7.11 | 800 | 0.7716 |
64
+ | 0.5719 | 8.0 | 900 | 0.7662 |
65
+ | 0.5575 | 8.89 | 1000 | 0.7700 |
66
+ | 0.5302 | 9.78 | 1100 | 0.7689 |
67
+ | 0.5465 | 10.67 | 1200 | 0.7688 |
68
+ | 0.5321 | 11.56 | 1300 | 0.7719 |
69
+ | 0.5141 | 12.44 | 1400 | 0.7684 |
70
+ | 0.5033 | 13.33 | 1500 | 0.7716 |
71
+ | 0.4931 | 14.22 | 1600 | 0.7664 |
72
+ | 0.4882 | 15.11 | 1700 | 0.7739 |
73
+ | 0.4742 | 16.0 | 1800 | 0.7757 |
74
+ | 0.4701 | 16.89 | 1900 | 0.7717 |
75
+ | 0.4932 | 17.78 | 2000 | 0.7748 |
76
+ | 0.4665 | 18.67 | 2100 | 0.7734 |
77
+ | 0.4614 | 19.56 | 2200 | 0.7809 |
78
+ | 0.4669 | 20.44 | 2300 | 0.7793 |
79
+ | 0.4635 | 21.33 | 2400 | 0.7750 |
80
+ | 0.452 | 22.22 | 2500 | 0.7778 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
 
82
 
83
  ### Framework versions
84
 
85
  - PEFT 0.7.2.dev0
86
+ - Transformers 4.38.0.dev0
87
  - Pytorch 2.1.0+cu121
88
  - Datasets 2.16.1
89
  - Tokenizers 0.15.0
adapter_config.json CHANGED
@@ -19,9 +19,9 @@
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
- "fc2",
23
  "Wqkv",
24
- "fc1"
25
  ],
26
  "task_type": "CAUSAL_LM",
27
  "use_rslora": false
 
19
  "rank_pattern": {},
20
  "revision": null,
21
  "target_modules": [
22
+ "fc1",
23
  "Wqkv",
24
+ "fc2"
25
  ],
26
  "task_type": "CAUSAL_LM",
27
  "use_rslora": false
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2dd10889dadd087fa291436864ccdac17731c0dd6208b018f4dea2b697502500
3
  size 26230352
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a79aaf3d7aa0749d8eeac205eda983a3043a10e340e9c4c3309ea982c1701255
3
  size 26230352
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:170d2b4f2c9cdc6279d2c61d071af5acc95390892cc5d989ccbd02fd5053105a
3
  size 4664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:81805563aacbaa907dc88ef1c48e665dc7cae63d4c7712b9231d6b3b149d7357
3
  size 4664