Hugging Face's logo

jbenbudd
/
ADPrLlama

Text Generation
PEFT
Safetensors
llama
llama-factory
lora
Generated from Trainer
conversational
Model card Files
xet
 Community
jbenbudd commited on
Commit
020945a
·
1 Parent(s): bb35aa9

train_residue_list_lr_3e-4_5_epochs_neg_5

Browse files
Files changed (11) hide show
  1. README.md +36 -16
  2. adapter_config.json +4 -4
  3. adapter_model.safetensors +1 -1
  4. all_results.json +11 -11
  5. eval_results.json +6 -6
  6. model_eval_results.csv +0 -0
  7. trainer_log.jsonl +0 -0
  8. trainer_state.json +0 -0
  9. training_args.bin +2 -2
  10. training_eval_loss.png +2 -2
  11. training_loss.png +2 -2
README.md CHANGED
@@ -7,19 +7,19 @@ tags:
7
  - lora
8
  - generated_from_trainer
9
  model-index:
10
- - name: train_residue_list_lr_3e-4_5_epochs
11
  results: []
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
  should probably proofread and complete it, then remove this comment. -->
16
 
17
- # train_residue_list_lr_3e-4_5_epochs
18
 
19
  This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
20
  It achieves the following results on the evaluation set:
21
- - Loss: 0.2519
22
- - Num Input Tokens Seen: 13420336
23
 
24
  ## Model description
25
 
@@ -53,22 +53,42 @@ The following hyperparameters were used during training:
53
 
54
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
55
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
56
- | 0.4552 | 0.4561 | 100 | 0.4630 | 1229824 |
57
- | 0.3728 | 0.9122 | 200 | 0.3762 | 2457344 |
58
- | 0.3172 | 1.3649 | 300 | 0.3195 | 3679728 |
59
- | 0.2906 | 1.8210 | 400 | 0.2807 | 4908144 |
60
- | 0.221 | 2.2737 | 500 | 0.2369 | 6131072 |
61
- | 0.2005 | 2.7298 | 600 | 0.2124 | 7358336 |
62
- | 0.1387 | 3.1824 | 700 | 0.2251 | 8576496 |
63
- | 0.1317 | 3.6385 | 800 | 0.2068 | 9805424 |
64
- | 0.0605 | 4.0912 | 900 | 0.2535 | 11023792 |
65
- | 0.0647 | 4.5473 | 1000 | 0.2509 | 12251056 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
66
 
67
 
68
  ### Framework versions
69
 
70
  - PEFT 0.14.0
71
- - Transformers 4.50.3
72
  - Pytorch 2.3.1+cu121
73
  - Datasets 3.5.0
74
- - Tokenizers 0.21.0
 
7
  - lora
8
  - generated_from_trainer
9
  model-index:
10
+ - name: train_residue_list_lr_3e-4_5_epochs_neg_5
11
  results: []
12
  ---
13
 
14
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
  should probably proofread and complete it, then remove this comment. -->
16
 
17
+ # train_residue_list_lr_3e-4_5_epochs_neg_5
18
 
19
  This model is a fine-tuned version of [GreatCaptainNemo/ProLLaMA_Stage_1](https://huggingface.co/GreatCaptainNemo/ProLLaMA_Stage_1) on the adpr_train dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.1675
22
+ - Num Input Tokens Seen: 36312064
23
 
24
  ## Model description
25
 
 
53
 
54
  | Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen |
55
  |:-------------:|:------:|:----:|:---------------:|:-----------------:|
56
+ | 0.314 | 0.1617 | 100 | 0.3099 | 1177472 |
57
+ | 1.0912 | 0.3235 | 200 | 0.2833 | 2353792 |
58
+ | 0.2867 | 0.4852 | 300 | 0.2665 | 3529856 |
59
+ | 0.2527 | 0.6470 | 400 | 0.2551 | 4704512 |
60
+ | 0.251 | 0.8087 | 500 | 0.2453 | 5881088 |
61
+ | 0.2157 | 0.9705 | 600 | 0.2286 | 7056896 |
62
+ | 0.2341 | 1.1310 | 700 | 0.2221 | 8223104 |
63
+ | 0.2017 | 1.2928 | 800 | 0.2096 | 9403904 |
64
+ | 0.1898 | 1.4545 | 900 | 0.2038 | 10580480 |
65
+ | 0.2013 | 1.6163 | 1000 | 0.2014 | 11757056 |
66
+ | 0.2059 | 1.7780 | 1100 | 0.1935 | 12932352 |
67
+ | 0.1938 | 1.9397 | 1200 | 0.1979 | 14108928 |
68
+ | 0.1803 | 2.1003 | 1300 | 0.1866 | 15276800 |
69
+ | 0.1628 | 2.2620 | 1400 | 0.1774 | 16452224 |
70
+ | 0.178 | 2.4238 | 1500 | 0.1787 | 17630080 |
71
+ | 0.1751 | 2.5855 | 1600 | 0.1708 | 18805632 |
72
+ | 0.1678 | 2.7473 | 1700 | 0.1674 | 19982720 |
73
+ | 0.1581 | 2.9090 | 1800 | 0.1618 | 21154816 |
74
+ | 0.1348 | 3.0696 | 1900 | 0.1682 | 22320640 |
75
+ | 0.1192 | 3.2313 | 2000 | 0.1588 | 23493120 |
76
+ | 0.1323 | 3.3930 | 2100 | 0.1559 | 24670976 |
77
+ | 0.1263 | 3.5548 | 2200 | 0.1524 | 25849728 |
78
+ | 0.1155 | 3.7165 | 2300 | 0.1491 | 27029120 |
79
+ | 0.1162 | 3.8783 | 2400 | 0.1464 | 28204928 |
80
+ | 0.0687 | 4.0388 | 2500 | 0.1594 | 29370240 |
81
+ | 0.0695 | 4.2006 | 2600 | 0.1708 | 30547712 |
82
+ | 0.0631 | 4.3623 | 2700 | 0.1689 | 31720576 |
83
+ | 0.0614 | 4.5241 | 2800 | 0.1685 | 32897024 |
84
+ | 0.0692 | 4.6858 | 2900 | 0.1681 | 34075520 |
85
+ | 0.0651 | 4.8476 | 3000 | 0.1680 | 35252352 |
86
 
87
 
88
  ### Framework versions
89
 
90
  - PEFT 0.14.0
91
+ - Transformers 4.51.1
92
  - Pytorch 2.3.1+cu121
93
  - Datasets 3.5.0
94
+ - Tokenizers 0.21.1
adapter_config.json CHANGED
@@ -24,12 +24,12 @@
24
  "revision": null,
25
  "target_modules": [
26
  "o_proj",
27
- "v_proj",
28
- "down_proj",
29
- "gate_proj",
30
  "q_proj",
 
 
31
  "k_proj",
32
- "up_proj"
 
33
  ],
34
  "task_type": "CAUSAL_LM",
35
  "use_dora": false,
 
24
  "revision": null,
25
  "target_modules": [
26
  "o_proj",
 
 
 
27
  "q_proj",
28
+ "gate_proj",
29
+ "up_proj",
30
  "k_proj",
31
+ "v_proj",
32
+ "down_proj"
33
  ],
34
  "task_type": "CAUSAL_LM",
35
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3b06d8e6f9e5a57088cc0ffb91f0185daaf8dd39f94cdc28b42e99bc4d955d04
3
  size 639691872
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee1fd50f64de9fd8d7d8430bffb36a78e5e7892844e990686b97457a5e61c5a3
3
  size 639691872
all_results.json CHANGED
@@ -1,13 +1,13 @@
1
  {
2
- "epoch": 4.980615735461802,
3
- "eval_loss": 0.25186342000961304,
4
- "eval_runtime": 33.1464,
5
- "eval_samples_per_second": 94.068,
6
- "eval_steps_per_second": 5.883,
7
- "num_input_tokens_seen": 13420336,
8
- "total_flos": 5.449127300160553e+17,
9
- "train_loss": 0.3250706321844772,
10
- "train_runtime": 5438.0568,
11
- "train_samples_per_second": 25.798,
12
- "train_steps_per_second": 0.201
13
  }
 
1
  {
2
+ "epoch": 4.993125758188435,
3
+ "eval_loss": 0.16754263639450073,
4
+ "eval_runtime": 91.5328,
5
+ "eval_samples_per_second": 96.064,
6
+ "eval_steps_per_second": 6.009,
7
+ "num_input_tokens_seen": 36312064,
8
+ "total_flos": 1.474397207846191e+18,
9
+ "train_loss": 0.2081744998500571,
10
+ "train_runtime": 16798.8058,
11
+ "train_samples_per_second": 23.554,
12
+ "train_steps_per_second": 0.184
13
  }
eval_results.json CHANGED
@@ -1,8 +1,8 @@
1
  {
2
- "epoch": 4.980615735461802,
3
- "eval_loss": 0.25186342000961304,
4
- "eval_runtime": 33.1464,
5
- "eval_samples_per_second": 94.068,
6
- "eval_steps_per_second": 5.883,
7
- "num_input_tokens_seen": 13420336
8
  }
 
1
  {
2
+ "epoch": 4.993125758188435,
3
+ "eval_loss": 0.16754263639450073,
4
+ "eval_runtime": 91.5328,
5
+ "eval_samples_per_second": 96.064,
6
+ "eval_steps_per_second": 6.009,
7
+ "num_input_tokens_seen": 36312064
8
  }
model_eval_results.csv CHANGED
The diff for this file is too large to render. See raw diff
 
trainer_log.jsonl CHANGED
The diff for this file is too large to render. See raw diff
 
trainer_state.json CHANGED
The diff for this file is too large to render. See raw diff
 
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:14078fb47457c034f27fd40c42c9dc6bf714c37c87ac5d9bbfc03e5791be6b8d
3
- size 5752
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5f08c1a4cfc054ca854c70ba9851c0d18da50594db6c40b3a394d51a54beb4e9
3
+ size 5688
training_eval_loss.png CHANGED

Git LFS Details

  • SHA256: 37e8ef8061604271a849a6d35b549079a2cdd8ad8f722cbea054c46deaec4a13
  • Pointer size: 130 Bytes
  • Size of remote file: 39.5 kB

Git LFS Details

  • SHA256: 691b010491800c4f39788db026cf05b7b8af62844dbc7bdc6982b6ffbf0a75b4
  • Pointer size: 130 Bytes
  • Size of remote file: 43.2 kB
training_loss.png CHANGED

Git LFS Details

  • SHA256: 6e985d3fd5227dcc1ba82183cbc63fe7c9acc61d5565c71ff3a6837576b70775
  • Pointer size: 130 Bytes
  • Size of remote file: 29.8 kB

Git LFS Details

  • SHA256: 48563105a9525ce51f917ee1f143ca345aca28d449fb33b304cd526b87279c87
  • Pointer size: 130 Bytes
  • Size of remote file: 29.2 kB