Hugging Face's logo

gokulsrinivasagan
/
tinybert_train_book_v2

Fill-Mask
Transformers
TensorBoard
Safetensors
distilbert
Generated from Trainer
Eval Results (legacy)
Model card Files
xet
 Metrics Community
gokulsrinivasagan commited on
Commit
5b39191
·
verified ·
1 Parent(s): 86f5e3c

Model save

Browse files
Files changed (2) hide show
  1. README.md +96 -0
  2. logs/events.out.tfevents.1738095850.ki-g0009.123416.0 +2 -2
README.md ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
+ base_model: distilbert-base-uncased
5
+ tags:
6
+ - generated_from_trainer
7
+ metrics:
8
+ - accuracy
9
+ model-index:
10
+ - name: tinybert_train_book_v2
11
+ results: []
12
+ ---
13
+
14
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
15
+ should probably proofread and complete it, then remove this comment. -->
16
+
17
+ # tinybert_train_book_v2
18
+
19
+ This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
20
+ It achieves the following results on the evaluation set:
21
+ - Loss: 1.4239
22
+ - Accuracy: 0.6906
23
+
24
+ ## Model description
25
+
26
+ More information needed
27
+
28
+ ## Intended uses & limitations
29
+
30
+ More information needed
31
+
32
+ ## Training and evaluation data
33
+
34
+ More information needed
35
+
36
+ ## Training procedure
37
+
38
+ ### Training hyperparameters
39
+
40
+ The following hyperparameters were used during training:
41
+ - learning_rate: 0.0001
42
+ - train_batch_size: 160
43
+ - eval_batch_size: 160
44
+ - seed: 10
45
+ - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
46
+ - lr_scheduler_type: linear
47
+ - lr_scheduler_warmup_steps: 10000
48
+ - num_epochs: 25
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
53
+ |:-------------:|:-------:|:------:|:---------------:|:--------:|
54
+ | 5.6396 | 0.7025 | 10000 | 5.4784 | 0.1645 |
55
+ | 5.5513 | 1.4051 | 20000 | 5.4083 | 0.1660 |
56
+ | 2.9289 | 2.1076 | 30000 | 2.5488 | 0.5214 |
57
+ | 2.5686 | 2.8102 | 40000 | 2.2269 | 0.5650 |
58
+ | 2.3542 | 3.5127 | 50000 | 2.0268 | 0.5945 |
59
+ | 2.2044 | 4.2153 | 60000 | 1.8924 | 0.6152 |
60
+ | 2.1102 | 4.9178 | 70000 | 1.8025 | 0.6296 |
61
+ | 2.0378 | 5.6203 | 80000 | 1.7395 | 0.6393 |
62
+ | 1.9881 | 6.3229 | 90000 | 1.6945 | 0.6467 |
63
+ | 1.9469 | 7.0254 | 100000 | 1.6552 | 0.6526 |
64
+ | 1.919 | 7.7280 | 110000 | 1.6279 | 0.6572 |
65
+ | 1.8899 | 8.4305 | 120000 | 1.6012 | 0.6613 |
66
+ | 1.8634 | 9.1331 | 130000 | 1.5849 | 0.6641 |
67
+ | 1.8481 | 9.8356 | 140000 | 1.5630 | 0.6679 |
68
+ | 1.8315 | 10.5381 | 150000 | 1.5476 | 0.6703 |
69
+ | 1.8209 | 11.2407 | 160000 | 1.5394 | 0.6716 |
70
+ | 1.8038 | 11.9432 | 170000 | 1.5250 | 0.6740 |
71
+ | 1.7932 | 12.6458 | 180000 | 1.5107 | 0.6761 |
72
+ | 1.7826 | 13.3483 | 190000 | 1.5058 | 0.6770 |
73
+ | 1.7736 | 14.0509 | 200000 | 1.4952 | 0.6785 |
74
+ | 1.7635 | 14.7534 | 210000 | 1.4862 | 0.6799 |
75
+ | 1.7549 | 15.4560 | 220000 | 1.4782 | 0.6815 |
76
+ | 1.7497 | 16.1585 | 230000 | 1.4737 | 0.6821 |
77
+ | 1.7407 | 16.8610 | 240000 | 1.4675 | 0.6832 |
78
+ | 1.7334 | 17.5636 | 250000 | 1.4612 | 0.6843 |
79
+ | 1.7288 | 18.2661 | 260000 | 1.4568 | 0.6849 |
80
+ | 1.7265 | 18.9687 | 270000 | 1.4519 | 0.6858 |
81
+ | 1.7179 | 19.6712 | 280000 | 1.4469 | 0.6868 |
82
+ | 1.7156 | 20.3738 | 290000 | 1.4428 | 0.6873 |
83
+ | 1.7086 | 21.0763 | 300000 | 1.4389 | 0.6882 |
84
+ | 1.7064 | 21.7788 | 310000 | 1.4334 | 0.6892 |
85
+ | 1.7009 | 22.4814 | 320000 | 1.4309 | 0.6895 |
86
+ | 1.6989 | 23.1839 | 330000 | 1.4292 | 0.6896 |
87
+ | 1.6971 | 23.8865 | 340000 | 1.4262 | 0.6902 |
88
+ | 1.6958 | 24.5890 | 350000 | 1.4239 | 0.6906 |
89
+
90
+
91
+ ### Framework versions
92
+
93
+ - Transformers 4.46.1
94
+ - Pytorch 2.2.0+cu121
95
+ - Datasets 3.1.0
96
+ - Tokenizers 0.20.1
logs/events.out.tfevents.1738095850.ki-g0009.123416.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:aea30ec3642b1bd77205b1261065acf149438e57a57fc2300bf35f13d8962fc8
3
- size 166792
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b8329b6ca36fbb3e051a37a580c0bb16d7a4605b60f96fb314260e852290497b
3
+ size 169517