Hugging Face's logo

shawhin
/
bert-phishing-classifier_teacher

Safetensors
bert
Generated from Trainer
Model card Files
xet
 Community
shawhin commited on
Commit
0a93a9d
·
verified ·
1 Parent(s): 64f400a

End of training

Browse files
Files changed (5) hide show
  1. README.md +18 -23
  2. config.json +1 -1
  3. model.safetensors +1 -1
  4. tokenizer.json +1 -6
  5. training_args.bin +2 -2
README.md CHANGED
@@ -1,5 +1,4 @@
1
  ---
2
- library_name: transformers
3
  license: apache-2.0
4
  base_model: google-bert/bert-base-uncased
5
  tags:
@@ -9,8 +8,6 @@ metrics:
9
  model-index:
10
  - name: bert-phishing-classifier_teacher
11
  results: []
12
- datasets:
13
- - shawhin/phishing-site-classification
14
  ---
15
 
16
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -18,17 +15,15 @@ should probably proofread and complete it, then remove this comment. -->
18
 
19
  # bert-phishing-classifier_teacher
20
 
21
- This model is a fine-tuned version of [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) on the None dataset.
22
  It achieves the following results on the evaluation set:
23
- - Loss: 0.2881
24
- - Accuracy: 0.867
25
  - Auc: 0.951
26
 
27
  ## Model description
28
 
29
- Teacher model for knowledge distillation example.
30
-
31
- [Video](https://youtu.be/FLkUOkeMd5M) | [Blog](https://towardsdatascience.com/compressing-large-language-models-llms-9f406eea5b5e) | [Example code](https://github.com/ShawhinT/YouTube-Blog/tree/main/LLMs/model-compression)
32
 
33
  ## Intended uses & limitations
34
 
@@ -55,21 +50,21 @@ The following hyperparameters were used during training:
55
 
56
  | Training Loss | Epoch | Step | Validation Loss | Accuracy | Auc |
57
  |:-------------:|:-----:|:----:|:---------------:|:--------:|:-----:|
58
- | 0.4916 | 1.0 | 263 | 0.4228 | 0.784 | 0.915 |
59
- | 0.3894 | 2.0 | 526 | 0.3586 | 0.818 | 0.932 |
60
- | 0.3837 | 3.0 | 789 | 0.3144 | 0.86 | 0.939 |
61
- | 0.3574 | 4.0 | 1052 | 0.4494 | 0.807 | 0.942 |
62
- | 0.3517 | 5.0 | 1315 | 0.3287 | 0.86 | 0.947 |
63
- | 0.3518 | 6.0 | 1578 | 0.3042 | 0.871 | 0.949 |
64
- | 0.3185 | 7.0 | 1841 | 0.2900 | 0.862 | 0.949 |
65
- | 0.3267 | 8.0 | 2104 | 0.2958 | 0.876 | 0.95 |
66
- | 0.3153 | 9.0 | 2367 | 0.2881 | 0.867 | 0.951 |
67
- | 0.3061 | 10.0 | 2630 | 0.2963 | 0.873 | 0.951 |
68
 
69
 
70
  ### Framework versions
71
 
72
- - Transformers 4.44.2
73
- - Pytorch 2.2.2
74
- - Datasets 2.21.0
75
- - Tokenizers 0.19.1
 
1
  ---
 
2
  license: apache-2.0
3
  base_model: google-bert/bert-base-uncased
4
  tags:
 
8
  model-index:
9
  - name: bert-phishing-classifier_teacher
10
  results: []
 
 
11
  ---
12
 
13
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
15
 
16
  # bert-phishing-classifier_teacher
17
 
18
+ This model is a fine-tuned version of [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 0.2984
21
+ - Accuracy: 0.873
22
  - Auc: 0.951
23
 
24
  ## Model description
25
 
26
+ More information needed
 
 
27
 
28
  ## Intended uses & limitations
29
 
 
50
 
51
  | Training Loss | Epoch | Step | Validation Loss | Accuracy | Auc |
52
  |:-------------:|:-----:|:----:|:---------------:|:--------:|:-----:|
53
+ | 0.495 | 1.0 | 263 | 0.4166 | 0.78 | 0.912 |
54
+ | 0.3896 | 2.0 | 526 | 0.3570 | 0.822 | 0.931 |
55
+ | 0.3824 | 3.0 | 789 | 0.3168 | 0.858 | 0.938 |
56
+ | 0.3561 | 4.0 | 1052 | 0.4707 | 0.789 | 0.941 |
57
+ | 0.3516 | 5.0 | 1315 | 0.3298 | 0.862 | 0.946 |
58
+ | 0.354 | 6.0 | 1578 | 0.3049 | 0.869 | 0.948 |
59
+ | 0.3215 | 7.0 | 1841 | 0.2908 | 0.864 | 0.949 |
60
+ | 0.3262 | 8.0 | 2104 | 0.2987 | 0.876 | 0.95 |
61
+ | 0.3154 | 9.0 | 2367 | 0.2896 | 0.864 | 0.951 |
62
+ | 0.306 | 10.0 | 2630 | 0.2984 | 0.873 | 0.951 |
63
 
64
 
65
  ### Framework versions
66
 
67
+ - Transformers 4.43.1
68
+ - Pytorch 2.3.1
69
+ - Datasets 3.2.0
70
+ - Tokenizers 0.19.1
config.json CHANGED
@@ -28,7 +28,7 @@
28
  "position_embedding_type": "absolute",
29
  "problem_type": "single_label_classification",
30
  "torch_dtype": "float32",
31
- "transformers_version": "4.44.2",
32
  "type_vocab_size": 2,
33
  "use_cache": true,
34
  "vocab_size": 30522
 
28
  "position_embedding_type": "absolute",
29
  "problem_type": "single_label_classification",
30
  "torch_dtype": "float32",
31
+ "transformers_version": "4.43.1",
32
  "type_vocab_size": 2,
33
  "use_cache": true,
34
  "vocab_size": 30522
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f84a167f2d2459a4bf6984691bd133a11c47e171b25153668add2db492eb2a7a
3
  size 437958648
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:af108aad02608a7e6c01a820a56490634406aff5e48a49378fff48943571d50d
3
  size 437958648
tokenizer.json CHANGED
@@ -1,11 +1,6 @@
1
  {
2
  "version": "1.0",
3
- "truncation": {
4
- "direction": "Right",
5
- "max_length": 512,
6
- "strategy": "LongestFirst",
7
- "stride": 0
8
- },
9
  "padding": null,
10
  "added_tokens": [
11
  {
 
1
  {
2
  "version": "1.0",
3
+ "truncation": null,
 
 
 
 
 
4
  "padding": null,
5
  "added_tokens": [
6
  {
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:6aceaf13011825b84313dbb2f00a7029cca6a8cddd1f996c2d43c82554403c66
3
- size 5240
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6236ce2a42ccfe06196d03dff57baf9f5889bfc06ae9672175ba932d390f0ed5
3
+ size 5176