Hugging Face's logo

Josh1207
/
codeparrot-ds-small

Text Generation
Transformers
TensorBoard
Safetensors
gpt2
Generated from Trainer
text-generation-inference
Model card Files
xet
Metrics Community
Josh1207 commited on
Commit
1891634
·
verified ·
1 Parent(s): c63935f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +112 -111
README.md CHANGED
@@ -1,111 +1,112 @@
1
- ---
2
- library_name: transformers
3
- license: mit
4
- base_model: gpt2
5
- tags:
6
- - generated_from_trainer
7
- model-index:
8
- - name: codeparrot-ds-small
9
- results: []
10
- ---
11
-
12
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
- should probably proofread and complete it, then remove this comment. -->
14
-
15
- # codeparrot-ds-small
16
-
17
- This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
18
- It achieves the following results on the evaluation set:
19
- - Loss: 3.2919
20
-
21
- ## Model description
22
-
23
- More information needed
24
-
25
- ## Intended uses & limitations
26
-
27
- More information needed
28
-
29
- ## Training and evaluation data
30
-
31
- More information needed
32
-
33
- ## Training procedure
34
-
35
- ### Training hyperparameters
36
-
37
- The following hyperparameters were used during training:
38
- - learning_rate: 0.0005
39
- - train_batch_size: 4
40
- - eval_batch_size: 4
41
- - seed: 42
42
- - gradient_accumulation_steps: 2
43
- - total_train_batch_size: 8
44
- - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
- - lr_scheduler_type: cosine
46
- - lr_scheduler_warmup_steps: 100
47
- - num_epochs: 1
48
- - mixed_precision_training: Native AMP
49
-
50
- ### Training results
51
-
52
- | Training Loss | Epoch | Step | Validation Loss |
53
- |:-------------:|:-----:|:-----:|:---------------:|
54
- | 5.9286 | 0.02 | 500 | 5.7197 |
55
- | 5.2717 | 0.04 | 1000 | 5.1488 |
56
- | 4.9972 | 0.06 | 1500 | 4.8345 |
57
- | 4.8214 | 0.08 | 2000 | 4.6251 |
58
- | 4.6272 | 0.1 | 2500 | 4.4807 |
59
- | 4.5241 | 0.12 | 3000 | 4.3484 |
60
- | 4.405 | 0.14 | 3500 | 4.2483 |
61
- | 4.3189 | 0.16 | 4000 | 4.1680 |
62
- | 4.2596 | 0.18 | 4500 | 4.0914 |
63
- | 4.2569 | 0.2 | 5000 | 4.0341 |
64
- | 4.1614 | 0.22 | 5500 | 3.9615 |
65
- | 4.1073 | 0.24 | 6000 | 3.9112 |
66
- | 4.0892 | 0.26 | 6500 | 3.8685 |
67
- | 4.0151 | 0.28 | 7000 | 3.8277 |
68
- | 3.903 | 0.3 | 7500 | 3.7787 |
69
- | 3.9248 | 0.32 | 8000 | 3.7447 |
70
- | 3.8978 | 0.34 | 8500 | 3.7189 |
71
- | 3.9231 | 0.36 | 9000 | 3.6877 |
72
- | 3.8936 | 0.38 | 9500 | 3.6479 |
73
- | 3.7649 | 0.4 | 10000 | 3.6154 |
74
- | 3.8156 | 0.42 | 10500 | 3.6069 |
75
- | 3.7588 | 0.44 | 11000 | 3.5772 |
76
- | 3.7559 | 0.46 | 11500 | 3.5517 |
77
- | 3.814 | 0.48 | 12000 | 3.5230 |
78
- | 3.7384 | 0.5 | 12500 | 3.5065 |
79
- | 3.6827 | 0.52 | 13000 | 3.4807 |
80
- | 3.6679 | 0.54 | 13500 | 3.4585 |
81
- | 3.6838 | 0.56 | 14000 | 3.4419 |
82
- | 3.7154 | 0.58 | 14500 | 3.4313 |
83
- | 3.7117 | 0.6 | 15000 | 3.4156 |
84
- | 3.6065 | 0.62 | 15500 | 3.3990 |
85
- | 3.613 | 0.64 | 16000 | 3.3820 |
86
- | 3.5824 | 0.66 | 16500 | 3.3702 |
87
- | 3.6263 | 0.68 | 17000 | 3.3645 |
88
- | 3.6073 | 0.7 | 17500 | 3.3529 |
89
- | 3.587 | 0.72 | 18000 | 3.3419 |
90
- | 3.624 | 0.74 | 18500 | 3.3340 |
91
- | 3.583 | 0.76 | 19000 | 3.3273 |
92
- | 3.573 | 0.78 | 19500 | 3.3185 |
93
- | 3.5576 | 0.8 | 20000 | 3.3123 |
94
- | 3.5623 | 0.82 | 20500 | 3.3103 |
95
- | 3.5955 | 0.84 | 21000 | 3.3053 |
96
- | 3.5947 | 0.86 | 21500 | 3.3015 |
97
- | 3.5258 | 0.88 | 22000 | 3.2989 |
98
- | 3.5985 | 0.9 | 22500 | 3.2962 |
99
- | 3.5723 | 0.92 | 23000 | 3.2938 |
100
- | 3.5863 | 0.94 | 23500 | 3.2927 |
101
- | 3.5378 | 0.96 | 24000 | 3.2921 |
102
- | 3.4952 | 0.98 | 24500 | 3.2919 |
103
- | 3.5357 | 1.0 | 25000 | 3.2919 |
104
-
105
-
106
- ### Framework versions
107
-
108
- - Transformers 4.51.3
109
- - Pytorch 2.5.1
110
- - Datasets 2.19.1
111
- - Tokenizers 0.21.1
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: mit
4
+ base_model: gpt2
5
+ tags:
6
+ - generated_from_trainer
7
+ model-index:
8
+ - name: codeparrot-ds-small
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # codeparrot-ds-small
16
+
17
+ This model is a fine-tuned version of [gpt2](https://huggingface.co/gpt2) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 3.2919
20
+
21
+ ## Model description
22
+
23
+ Due to hardware limitations and an insufficient amount of training data,
24
+ the model has only been trained for a single epoch and is currently not functional.
25
+
26
+ ## Intended uses & limitations
27
+
28
+ More information needed
29
+
30
+ ## Training and evaluation data
31
+
32
+ More information needed
33
+
34
+ ## Training procedure
35
+
36
+ ### Training hyperparameters
37
+
38
+ The following hyperparameters were used during training:
39
+ - learning_rate: 0.0005
40
+ - train_batch_size: 4
41
+ - eval_batch_size: 4
42
+ - seed: 42
43
+ - gradient_accumulation_steps: 2
44
+ - total_train_batch_size: 8
45
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
+ - lr_scheduler_type: cosine
47
+ - lr_scheduler_warmup_steps: 100
48
+ - num_epochs: 1
49
+ - mixed_precision_training: Native AMP
50
+
51
+ ### Training results
52
+
53
+ | Training Loss | Epoch | Step | Validation Loss |
54
+ |:-------------:|:-----:|:-----:|:---------------:|
55
+ | 5.9286 | 0.02 | 500 | 5.7197 |
56
+ | 5.2717 | 0.04 | 1000 | 5.1488 |
57
+ | 4.9972 | 0.06 | 1500 | 4.8345 |
58
+ | 4.8214 | 0.08 | 2000 | 4.6251 |
59
+ | 4.6272 | 0.1 | 2500 | 4.4807 |
60
+ | 4.5241 | 0.12 | 3000 | 4.3484 |
61
+ | 4.405 | 0.14 | 3500 | 4.2483 |
62
+ | 4.3189 | 0.16 | 4000 | 4.1680 |
63
+ | 4.2596 | 0.18 | 4500 | 4.0914 |
64
+ | 4.2569 | 0.2 | 5000 | 4.0341 |
65
+ | 4.1614 | 0.22 | 5500 | 3.9615 |
66
+ | 4.1073 | 0.24 | 6000 | 3.9112 |
67
+ | 4.0892 | 0.26 | 6500 | 3.8685 |
68
+ | 4.0151 | 0.28 | 7000 | 3.8277 |
69
+ | 3.903 | 0.3 | 7500 | 3.7787 |
70
+ | 3.9248 | 0.32 | 8000 | 3.7447 |
71
+ | 3.8978 | 0.34 | 8500 | 3.7189 |
72
+ | 3.9231 | 0.36 | 9000 | 3.6877 |
73
+ | 3.8936 | 0.38 | 9500 | 3.6479 |
74
+ | 3.7649 | 0.4 | 10000 | 3.6154 |
75
+ | 3.8156 | 0.42 | 10500 | 3.6069 |
76
+ | 3.7588 | 0.44 | 11000 | 3.5772 |
77
+ | 3.7559 | 0.46 | 11500 | 3.5517 |
78
+ | 3.814 | 0.48 | 12000 | 3.5230 |
79
+ | 3.7384 | 0.5 | 12500 | 3.5065 |
80
+ | 3.6827 | 0.52 | 13000 | 3.4807 |
81
+ | 3.6679 | 0.54 | 13500 | 3.4585 |
82
+ | 3.6838 | 0.56 | 14000 | 3.4419 |
83
+ | 3.7154 | 0.58 | 14500 | 3.4313 |
84
+ | 3.7117 | 0.6 | 15000 | 3.4156 |
85
+ | 3.6065 | 0.62 | 15500 | 3.3990 |
86
+ | 3.613 | 0.64 | 16000 | 3.3820 |
87
+ | 3.5824 | 0.66 | 16500 | 3.3702 |
88
+ | 3.6263 | 0.68 | 17000 | 3.3645 |
89
+ | 3.6073 | 0.7 | 17500 | 3.3529 |
90
+ | 3.587 | 0.72 | 18000 | 3.3419 |
91
+ | 3.624 | 0.74 | 18500 | 3.3340 |
92
+ | 3.583 | 0.76 | 19000 | 3.3273 |
93
+ | 3.573 | 0.78 | 19500 | 3.3185 |
94
+ | 3.5576 | 0.8 | 20000 | 3.3123 |
95
+ | 3.5623 | 0.82 | 20500 | 3.3103 |
96
+ | 3.5955 | 0.84 | 21000 | 3.3053 |
97
+ | 3.5947 | 0.86 | 21500 | 3.3015 |
98
+ | 3.5258 | 0.88 | 22000 | 3.2989 |
99
+ | 3.5985 | 0.9 | 22500 | 3.2962 |
100
+ | 3.5723 | 0.92 | 23000 | 3.2938 |
101
+ | 3.5863 | 0.94 | 23500 | 3.2927 |
102
+ | 3.5378 | 0.96 | 24000 | 3.2921 |
103
+ | 3.4952 | 0.98 | 24500 | 3.2919 |
104
+ | 3.5357 | 1.0 | 25000 | 3.2919 |
105
+
106
+
107
+ ### Framework versions
108
+
109
+ - Transformers 4.51.3
110
+ - Pytorch 2.5.1
111
+ - Datasets 2.19.1
112
+ - Tokenizers 0.21.1