Hugging Face's logo

waelChafei
/
bertuned

Text Classification
Transformers
Safetensors
bert
Generated from Trainer
Model card Files
xet
Community
waelChafei commited on
Commit
b1fae93
·
verified ·
1 Parent(s): c1d8c51

Upload 5 files

Browse files
Files changed (5) hide show
  1. README.md +140 -0
  2. config.json +45 -0
  3. gitattributes +35 -0
  4. model.safetensors +3 -0
  5. training_args.bin +3 -0
README.md ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: bert-base-uncased
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - accuracy
8
+ - f1
9
+ - precision
10
+ - recall
11
+ model-index:
12
+ - name: TTC4900Model
13
+ results: []
14
+ ---
15
+
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
+
19
+ # TTC4900Model
20
+
21
+ This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
22
+ It achieves the following results on the evaluation set:
23
+ - Loss: 0.4669
24
+ - Accuracy: 0.8633
25
+ - F1: 0.7752
26
+ - Precision: 0.7764
27
+ - Recall: 0.7751
28
+
29
+ ## Model description
30
+
31
+ More information needed
32
+
33
+ ## Intended uses & limitations
34
+
35
+ More information needed
36
+
37
+ ## Training and evaluation data
38
+
39
+ More information needed
40
+
41
+ ## Training procedure
42
+
43
+ ### Training hyperparameters
44
+
45
+ The following hyperparameters were used during training:
46
+ - learning_rate: 5e-05
47
+ - train_batch_size: 32
48
+ - eval_batch_size: 64
49
+ - seed: 42
50
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: linear
52
+ - lr_scheduler_warmup_steps: 100
53
+ - num_epochs: 3
54
+ - mixed_precision_training: Native AMP
55
+
56
+ ### Training results
57
+
58
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy | F1 | Precision | Recall |
59
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
60
+ | 1.6784 | 0.04 | 50 | 1.2099 | 0.5995 | 0.2253 | 0.3879 | 0.2247 |
61
+ | 1.0203 | 0.08 | 100 | 0.7696 | 0.7533 | 0.4285 | 0.5258 | 0.4563 |
62
+ | 0.7322 | 0.12 | 150 | 0.6271 | 0.8152 | 0.6305 | 0.6720 | 0.6056 |
63
+ | 0.6119 | 0.16 | 200 | 0.5934 | 0.8270 | 0.6673 | 0.7649 | 0.6662 |
64
+ | 0.6203 | 0.2 | 250 | 0.5759 | 0.8281 | 0.6618 | 0.7352 | 0.6566 |
65
+ | 0.5874 | 0.24 | 300 | 0.5446 | 0.8373 | 0.7351 | 0.7633 | 0.7256 |
66
+ | 0.5507 | 0.28 | 350 | 0.5371 | 0.8394 | 0.7481 | 0.7580 | 0.7435 |
67
+ | 0.5615 | 0.33 | 400 | 0.5272 | 0.8469 | 0.7377 | 0.7961 | 0.7241 |
68
+ | 0.5371 | 0.37 | 450 | 0.5193 | 0.8431 | 0.7413 | 0.7828 | 0.7299 |
69
+ | 0.5267 | 0.41 | 500 | 0.5124 | 0.8490 | 0.7446 | 0.7714 | 0.7321 |
70
+ | 0.4971 | 0.45 | 550 | 0.5105 | 0.8509 | 0.7503 | 0.8071 | 0.7192 |
71
+ | 0.5399 | 0.49 | 600 | 0.5149 | 0.8473 | 0.7402 | 0.7849 | 0.7291 |
72
+ | 0.5229 | 0.53 | 650 | 0.5190 | 0.8447 | 0.7504 | 0.7753 | 0.7421 |
73
+ | 0.5077 | 0.57 | 700 | 0.5096 | 0.8487 | 0.7528 | 0.7764 | 0.7413 |
74
+ | 0.5073 | 0.61 | 750 | 0.4946 | 0.8511 | 0.7487 | 0.8055 | 0.7279 |
75
+ | 0.4823 | 0.65 | 800 | 0.5105 | 0.8506 | 0.7509 | 0.7918 | 0.7292 |
76
+ | 0.483 | 0.69 | 850 | 0.4887 | 0.8542 | 0.7653 | 0.7936 | 0.7502 |
77
+ | 0.55 | 0.73 | 900 | 0.4865 | 0.8556 | 0.7599 | 0.8010 | 0.7425 |
78
+ | 0.5406 | 0.77 | 950 | 0.4875 | 0.8533 | 0.7515 | 0.8018 | 0.7255 |
79
+ | 0.5078 | 0.81 | 1000 | 0.5075 | 0.8564 | 0.7528 | 0.8134 | 0.7227 |
80
+ | 0.4965 | 0.85 | 1050 | 0.4789 | 0.8560 | 0.7655 | 0.7995 | 0.7405 |
81
+ | 0.4676 | 0.89 | 1100 | 0.4806 | 0.8555 | 0.7468 | 0.7989 | 0.7244 |
82
+ | 0.4786 | 0.93 | 1150 | 0.4822 | 0.8572 | 0.7587 | 0.7762 | 0.7504 |
83
+ | 0.4928 | 0.98 | 1200 | 0.4743 | 0.8569 | 0.7553 | 0.8009 | 0.7404 |
84
+ | 0.472 | 1.02 | 1250 | 0.4917 | 0.8581 | 0.7511 | 0.8075 | 0.7278 |
85
+ | 0.395 | 1.06 | 1300 | 0.4929 | 0.8573 | 0.7657 | 0.7937 | 0.7516 |
86
+ | 0.3735 | 1.1 | 1350 | 0.4844 | 0.8558 | 0.7687 | 0.7685 | 0.7693 |
87
+ | 0.3974 | 1.14 | 1400 | 0.5005 | 0.8504 | 0.7631 | 0.7556 | 0.7732 |
88
+ | 0.4197 | 1.18 | 1450 | 0.5044 | 0.8557 | 0.7475 | 0.8096 | 0.7265 |
89
+ | 0.4149 | 1.22 | 1500 | 0.4914 | 0.8560 | 0.7556 | 0.8009 | 0.7431 |
90
+ | 0.4937 | 1.26 | 1550 | 0.4684 | 0.8583 | 0.7697 | 0.7847 | 0.7596 |
91
+ | 0.344 | 1.3 | 1600 | 0.5126 | 0.8570 | 0.7606 | 0.8084 | 0.7369 |
92
+ | 0.4399 | 1.34 | 1650 | 0.4856 | 0.8545 | 0.7608 | 0.7947 | 0.7490 |
93
+ | 0.4176 | 1.38 | 1700 | 0.4851 | 0.8578 | 0.7648 | 0.8018 | 0.7419 |
94
+ | 0.4301 | 1.42 | 1750 | 0.4725 | 0.8579 | 0.7673 | 0.7811 | 0.7611 |
95
+ | 0.4161 | 1.46 | 1800 | 0.4794 | 0.8587 | 0.7712 | 0.7954 | 0.7583 |
96
+ | 0.3632 | 1.5 | 1850 | 0.4835 | 0.8579 | 0.7716 | 0.8036 | 0.7543 |
97
+ | 0.4328 | 1.54 | 1900 | 0.4748 | 0.8574 | 0.7744 | 0.7958 | 0.7606 |
98
+ | 0.4026 | 1.59 | 1950 | 0.4733 | 0.8571 | 0.7726 | 0.7925 | 0.7591 |
99
+ | 0.3792 | 1.63 | 2000 | 0.4826 | 0.8593 | 0.7715 | 0.7912 | 0.7576 |
100
+ | 0.4087 | 1.67 | 2050 | 0.4732 | 0.8594 | 0.7704 | 0.7954 | 0.7525 |
101
+ | 0.3953 | 1.71 | 2100 | 0.4721 | 0.8592 | 0.7724 | 0.7869 | 0.7634 |
102
+ | 0.403 | 1.75 | 2150 | 0.4714 | 0.8629 | 0.7737 | 0.8055 | 0.7552 |
103
+ | 0.3987 | 1.79 | 2200 | 0.4657 | 0.8617 | 0.7729 | 0.7929 | 0.7583 |
104
+ | 0.3891 | 1.83 | 2250 | 0.4694 | 0.8609 | 0.7720 | 0.7877 | 0.7664 |
105
+ | 0.4266 | 1.87 | 2300 | 0.4716 | 0.8603 | 0.7696 | 0.7953 | 0.7625 |
106
+ | 0.3784 | 1.91 | 2350 | 0.4658 | 0.8609 | 0.7735 | 0.7968 | 0.7606 |
107
+ | 0.4108 | 1.95 | 2400 | 0.4571 | 0.8611 | 0.7746 | 0.8005 | 0.7585 |
108
+ | 0.4227 | 1.99 | 2450 | 0.4575 | 0.8634 | 0.7812 | 0.8047 | 0.7643 |
109
+ | 0.2896 | 2.03 | 2500 | 0.4835 | 0.8637 | 0.7801 | 0.7916 | 0.7730 |
110
+ | 0.3539 | 2.07 | 2550 | 0.4741 | 0.8626 | 0.7787 | 0.7974 | 0.7664 |
111
+ | 0.3657 | 2.11 | 2600 | 0.4799 | 0.8579 | 0.7737 | 0.7964 | 0.7592 |
112
+ | 0.3407 | 2.15 | 2650 | 0.4765 | 0.8604 | 0.7681 | 0.7963 | 0.7524 |
113
+ | 0.317 | 2.2 | 2700 | 0.4817 | 0.8583 | 0.7729 | 0.7797 | 0.7705 |
114
+ | 0.3166 | 2.24 | 2750 | 0.4886 | 0.8589 | 0.7653 | 0.7917 | 0.7553 |
115
+ | 0.3078 | 2.28 | 2800 | 0.4927 | 0.8574 | 0.7709 | 0.7888 | 0.7607 |
116
+ | 0.3366 | 2.32 | 2850 | 0.4948 | 0.8600 | 0.7735 | 0.7907 | 0.7609 |
117
+ | 0.2863 | 2.36 | 2900 | 0.4994 | 0.8578 | 0.7699 | 0.7784 | 0.7668 |
118
+ | 0.255 | 2.4 | 2950 | 0.5017 | 0.8601 | 0.7696 | 0.7862 | 0.7575 |
119
+ | 0.3379 | 2.44 | 3000 | 0.4824 | 0.8591 | 0.7715 | 0.7787 | 0.7671 |
120
+ | 0.2751 | 2.48 | 3050 | 0.4944 | 0.8616 | 0.7745 | 0.7965 | 0.7587 |
121
+ | 0.2902 | 2.52 | 3100 | 0.4865 | 0.8596 | 0.7751 | 0.7863 | 0.7675 |
122
+ | 0.2917 | 2.56 | 3150 | 0.4875 | 0.8608 | 0.7748 | 0.7864 | 0.7656 |
123
+ | 0.3014 | 2.6 | 3200 | 0.4872 | 0.8614 | 0.7756 | 0.7886 | 0.7663 |
124
+ | 0.3269 | 2.64 | 3250 | 0.4905 | 0.8598 | 0.7763 | 0.7884 | 0.7669 |
125
+ | 0.3245 | 2.68 | 3300 | 0.4898 | 0.8625 | 0.7778 | 0.7971 | 0.7627 |
126
+ | 0.2951 | 2.72 | 3350 | 0.4864 | 0.8599 | 0.7771 | 0.7885 | 0.7686 |
127
+ | 0.2888 | 2.76 | 3400 | 0.4906 | 0.8623 | 0.7758 | 0.7969 | 0.7609 |
128
+ | 0.3037 | 2.8 | 3450 | 0.4863 | 0.8609 | 0.7751 | 0.7939 | 0.7630 |
129
+ | 0.2855 | 2.85 | 3500 | 0.4881 | 0.8621 | 0.7773 | 0.7935 | 0.7661 |
130
+ | 0.297 | 2.89 | 3550 | 0.4880 | 0.8620 | 0.7786 | 0.7927 | 0.7687 |
131
+ | 0.2753 | 2.93 | 3600 | 0.4887 | 0.8615 | 0.7772 | 0.7902 | 0.7679 |
132
+ | 0.2922 | 2.97 | 3650 | 0.4876 | 0.8610 | 0.7770 | 0.7883 | 0.7691 |
133
+
134
+
135
+ ### Framework versions
136
+
137
+ - Transformers 4.38.2
138
+ - Pytorch 2.1.2
139
+ - Datasets 2.1.0
140
+ - Tokenizers 0.15.2
config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "bert-base-uncased",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 768,
12
+ "id2label": {
13
+ "0": "Skill",
14
+ "1": "Exp",
15
+ "2": "PI",
16
+ "3": "Sum",
17
+ "4": "Edu",
18
+ "5": "Obj",
19
+ "6": "QC"
20
+ },
21
+ "initializer_range": 0.02,
22
+ "intermediate_size": 3072,
23
+ "label2id": {
24
+ "Edu": 4,
25
+ "Exp": 1,
26
+ "Obj": 5,
27
+ "PI": 2,
28
+ "QC": 6,
29
+ "Skill": 0,
30
+ "Sum": 3
31
+ },
32
+ "layer_norm_eps": 1e-12,
33
+ "max_position_embeddings": 512,
34
+ "model_type": "bert",
35
+ "num_attention_heads": 12,
36
+ "num_hidden_layers": 12,
37
+ "pad_token_id": 0,
38
+ "position_embedding_type": "absolute",
39
+ "problem_type": "single_label_classification",
40
+ "torch_dtype": "float32",
41
+ "transformers_version": "4.38.2",
42
+ "type_vocab_size": 2,
43
+ "use_cache": true,
44
+ "vocab_size": 30522
45
+ }
gitattributes ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9656ca09afacc33714dea0908bab08f8a96eab9c8b0c75b680e400fcd1294711
3
+ size 437974028
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cb4ca599e4007cf24e2e10eb05f5d833afcc40fc29d17147a8700df7a7ea7617
3
+ size 4856