Hugging Face's logo

chenghuzi
/
pyphi

PEFT
Safetensors
Generated from Trainer
Model card Files
xet
 Community
chenghuzi commited on
Commit
8b2db7a
·
verified ·
1 Parent(s): 402e2bf

End of training

Browse files
Files changed (4) hide show
  1. README.md +8 -7
  2. adapter_config.json +3 -3
  3. adapter_model.safetensors +2 -2
  4. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 7.7076
20
 
21
  ## Model description
22
 
@@ -43,15 +43,16 @@ The following hyperparameters were used during training:
43
  - total_train_batch_size: 12
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
- - num_epochs: 3
47
 
48
  ### Training results
49
 
50
- | Training Loss | Epoch | Step | Validation Loss |
51
- |:-------------:|:------:|:----:|:---------------:|
52
- | 8.9719 | 0.9971 | 115 | 8.4008 |
53
- | 8.1061 | 1.9942 | 230 | 7.8448 |
54
- | 7.6961 | 2.9913 | 345 | 7.7076 |
 
55
 
56
 
57
  ### Framework versions
 
16
 
17
  This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.5101
20
 
21
  ## Model description
22
 
 
43
  - total_train_batch_size: 12
44
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
45
  - lr_scheduler_type: linear
46
+ - num_epochs: 4
47
 
48
  ### Training results
49
 
50
+ | Training Loss | Epoch | Step | Validation Loss |
51
+ |:-------------:|:------:|:-----:|:---------------:|
52
+ | 1.5134 | 0.9998 | 4651 | 1.5228 |
53
+ | 1.402 | 1.9999 | 9303 | 1.5084 |
54
+ | 1.2921 | 2.9999 | 13955 | 1.5076 |
55
+ | 1.2776 | 3.9993 | 18604 | 1.5101 |
56
 
57
 
58
  ### Framework versions
adapter_config.json CHANGED
@@ -23,9 +23,9 @@
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
- "layers.0.self_attn.q_proj",
27
- "layers.0.self_attn.v_proj",
28
- "layers.0.self_attn.k_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
31
  "use_dora": false,
 
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
+ "dense",
27
+ "q_proj",
28
+ "v_proj"
29
  ],
30
  "task_type": "CAUSAL_LM",
31
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:335431f0fb932a59e74de44a3ad8dd3e6a865c3e1ea6861cfed73b0ccf1a1a70
3
- size 1050748064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3b1c8387dfa0d70a92f050462502afa32e89b614865ec5d44d7bec45d0f6a64f
3
+ size 1111722280
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d03002142727fdec843e77baeab8b5ab068b477af88093783e8c0e0718e8d815
3
  size 4984
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:934e7d7d3924eb1fa1951cd5ca644c6a8f9c05113af133a84b2f90057153ec97
3
  size 4984