Hugging Face's logo

abhishekyo
/
codellama2-finetuned-codex

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files
xet
 Metrics Community
abhishekyo commited on
Commit
01856cf
·
verified ·
1 Parent(s): 7702741

abhishekyo/finetune8

Browse files
Files changed (4) hide show
  1. README.md +56 -8
  2. adapter_model.safetensors +1 -1
  3. runs/Apr09_08-08-02_a1d332ecb71b/events.out.tfevents.1712650267.a1d332ecb71b.35.0 +2 -2
  4. runs/Apr09_08-08-02_a1d332ecb71b/events.out.tfevents.1712653495.a1d332ecb71b.35.1 +3 -0
README.md CHANGED
@@ -18,12 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  This model is a fine-tuned version of [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
- - eval_loss: 0.2229
22
- - eval_runtime: 13.9845
23
- - eval_samples_per_second: 1.645
24
- - eval_steps_per_second: 0.429
25
- - epoch: 12.35
26
- - step: 562
27
 
28
  ## Model description
29
 
@@ -42,7 +37,7 @@ More information needed
42
  ### Training hyperparameters
43
 
44
  The following hyperparameters were used during training:
45
- - learning_rate: 0.0001
46
  - train_batch_size: 4
47
  - eval_batch_size: 4
48
  - seed: 42
@@ -54,10 +49,63 @@ The following hyperparameters were used during training:
54
  - num_epochs: 20
55
  - mixed_precision_training: Native AMP
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ### Framework versions
58
 
59
  - PEFT 0.10.0
60
- - Transformers 4.39.2
61
  - Pytorch 2.1.2
62
  - Datasets 2.18.0
63
  - Tokenizers 0.15.2
 
18
 
19
  This model is a fine-tuned version of [codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf) on an unknown dataset.
20
  It achieves the following results on the evaluation set:
21
+ - Loss: 0.2549
 
 
 
 
 
22
 
23
  ## Model description
24
 
 
37
  ### Training hyperparameters
38
 
39
  The following hyperparameters were used during training:
40
+ - learning_rate: 4e-05
41
  - train_batch_size: 4
42
  - eval_batch_size: 4
43
  - seed: 42
 
49
  - num_epochs: 20
50
  - mixed_precision_training: Native AMP
51
 
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss |
55
+ |:-------------:|:-----:|:----:|:---------------:|
56
+ | 1.876 | 0.43 | 20 | 1.7743 |
57
+ | 1.6213 | 0.85 | 40 | 1.4426 |
58
+ | 1.0392 | 1.28 | 60 | 0.9086 |
59
+ | 0.6962 | 1.7 | 80 | 0.6664 |
60
+ | 0.529 | 2.13 | 100 | 0.5439 |
61
+ | 0.4614 | 2.55 | 120 | 0.4650 |
62
+ | 0.4264 | 2.98 | 140 | 0.4218 |
63
+ | 0.376 | 3.4 | 160 | 0.3951 |
64
+ | 0.3665 | 3.83 | 180 | 0.3722 |
65
+ | 0.3398 | 4.26 | 200 | 0.3559 |
66
+ | 0.3198 | 4.68 | 220 | 0.3389 |
67
+ | 0.3263 | 5.11 | 240 | 0.3317 |
68
+ | 0.2952 | 5.53 | 260 | 0.3223 |
69
+ | 0.2871 | 5.96 | 280 | 0.3136 |
70
+ | 0.2861 | 6.38 | 300 | 0.3084 |
71
+ | 0.2899 | 6.81 | 320 | 0.3021 |
72
+ | 0.2769 | 7.23 | 340 | 0.2982 |
73
+ | 0.2541 | 7.66 | 360 | 0.2951 |
74
+ | 0.2421 | 8.09 | 380 | 0.2914 |
75
+ | 0.2275 | 8.51 | 400 | 0.2887 |
76
+ | 0.26 | 8.94 | 420 | 0.2799 |
77
+ | 0.2275 | 9.36 | 440 | 0.2797 |
78
+ | 0.2291 | 9.79 | 460 | 0.2722 |
79
+ | 0.2222 | 10.21 | 480 | 0.2744 |
80
+ | 0.2391 | 10.64 | 500 | 0.2721 |
81
+ | 0.208 | 11.06 | 520 | 0.2671 |
82
+ | 0.2012 | 11.49 | 540 | 0.2691 |
83
+ | 0.2092 | 11.91 | 560 | 0.2619 |
84
+ | 0.1761 | 12.34 | 580 | 0.2636 |
85
+ | 0.2248 | 12.77 | 600 | 0.2596 |
86
+ | 0.1803 | 13.19 | 620 | 0.2611 |
87
+ | 0.2022 | 13.62 | 640 | 0.2597 |
88
+ | 0.2006 | 14.04 | 660 | 0.2578 |
89
+ | 0.1864 | 14.47 | 680 | 0.2561 |
90
+ | 0.1933 | 14.89 | 700 | 0.2560 |
91
+ | 0.1892 | 15.32 | 720 | 0.2570 |
92
+ | 0.192 | 15.74 | 740 | 0.2562 |
93
+ | 0.1883 | 16.17 | 760 | 0.2553 |
94
+ | 0.1781 | 16.6 | 780 | 0.2549 |
95
+ | 0.1705 | 17.02 | 800 | 0.2560 |
96
+ | 0.181 | 17.45 | 820 | 0.2566 |
97
+ | 0.1552 | 17.87 | 840 | 0.2551 |
98
+ | 0.173 | 18.3 | 860 | 0.2560 |
99
+ | 0.1934 | 18.72 | 880 | 0.2557 |
100
+ | 0.1754 | 19.15 | 900 | 0.2555 |
101
+ | 0.1796 | 19.57 | 920 | 0.2555 |
102
+ | 0.1745 | 20.0 | 940 | 0.2555 |
103
+
104
+
105
  ### Framework versions
106
 
107
  - PEFT 0.10.0
108
+ - Transformers 4.39.3
109
  - Pytorch 2.1.2
110
  - Datasets 2.18.0
111
  - Tokenizers 0.15.2
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8edf312c4ceffc647b87e7b189486782d7ce43e6750454135c0efa8f19aeae12
3
  size 1116277688
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3bd9cc97ecdd98613e8d4b015bcadea8ee9b631542be72ffbc4e668245e507f5
3
  size 1116277688
runs/Apr09_08-08-02_a1d332ecb71b/events.out.tfevents.1712650267.a1d332ecb71b.35.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b220d029c7c696fadda743a8dda31b9229e712c539f75d784813e07738d3651f
3
- size 8652
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f71a9e9803851a1369562e2836443641d2522226b4647a858df2397ce537f27
3
+ size 9006
runs/Apr09_08-08-02_a1d332ecb71b/events.out.tfevents.1712653495.a1d332ecb71b.35.1 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1092804834994fb871e33c697b7e0830a36ceec6733e005efd3e14817fd9bee9
3
+ size 359