init

README.md

@@ -0,0 +1,104 @@
+---
+license: apache-2.0
+base_model: bert-large-uncased
+tags:
+- generated_from_trainer
+- phishing
+- BERT
+metrics:
+- accuracy
+- precision
+- recall
+model-index:
+- name: bert-finetuned-phishing
+  results: []
+widget:
+- text: https://www.verif22.com
+  example_title: Phishing URL
+- text: Dear colleague, An important update about your email has exceeded your
+    storage limit. You will not be able to send or receive all of your messages.
+    We will close all older versions of our Mailbox as of Friday, June 12, 2023.
+    To activate and complete the required information click here (https://ec-ec.squarespace.com). 
+    Account must be reactivated today to regenerate new space. Management Team
+  example_title: Phishing Email
+- text: You have access to FREE Video Streaming in your plan. REGISTER with your email, password and 
+    then select the monthly subscription option. https://bit.ly/3vNrU5r
+  example_title: Phishing SMS
+- text: if(data.selectedIndex > 0){$('#hidCflag').val(data.selectedData.value);};;
+    var sprypassword1 = new Spry.Widget.ValidationPassword("sprypassword1");
+    var sprytextfield1 = new Spry.Widget.ValidationTextField("sprytextfield1", "email");
+  example_title: Phishing Script
+- text: Hi, this model is really accurate :)
+  example_title: Benign message
+datasets:
+- ealvaradob/phishing-dataset
+language:
+- en
+pipeline_tag: text-classification
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# BERT FINETUNED ON PHISHING DETECTION
+
+This model is a fine-tuned version of [bert-large-uncased](https://huggingface.co/bert-large-uncased) on an [phishing dataset](https://huggingface.co/datasets/ealvaradob/phishing-dataset),
+capable of detecting phishing in its four most common forms: URLs, Emails, SMS messages and even websites. 
+
+It achieves the following results on the evaluation set:
+
+- Loss: 0.1953
+- Accuracy: 0.9717
+- Precision: 0.9658
+- Recall: 0.9670
+- False Positive Rate: 0.0249
+
+## Model description
+
+BERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. 
+This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why 
+it can use lots of publicly available data) with an automatic process to generate inputs and labels from 
+those texts.
+
+This model has the following configuration:
+
+- 24-layer
+- 1024 hidden dimension
+- 16 attention heads
+- 336M parameters
+
+## Motivation and Purpose
+
+Phishing is one of the most frequent and most expensive cyber-attacks according to several security reports. 
+This model aims to efficiently and accurately prevent phishing attacks against individuals and organizations. 
+To achieve it, BERT was trained on a diverse and robust dataset containing: URLs, SMS Messages, Emails and 
+Websites, which allows the model to extend its detection capability beyond the usual and to be used in various 
+contexts.
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 16
+- eval_batch_size: 16
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | Accuracy | Precision | Recall | False Positive Rate |
+|:-------------:|:-----:|:-----:|:---------------:|:--------:|:---------:|:------:|:-------------------:|
+| 0.1487        | 1.0   | 3866  | 0.1454          | 0.9596   | 0.9709    | 0.9320 | 0.0203              |
+| 0.0805        | 2.0   | 7732  | 0.1389          | 0.9691   | 0.9663    | 0.9601 | 0.0243              |
+| 0.0389        | 3.0   | 11598 | 0.1779          | 0.9683   | 0.9778    | 0.9461 | 0.0156              |
+| 0.0091        | 4.0   | 15464 | 0.1953          | 0.9717   | 0.9658    | 0.9670 | 0.0249              |
+
+
+### Framework versions
+
+- Transformers 4.34.1
+- Pytorch 2.1.1+cu121
+- Datasets 2.14.6
+- Tokenizers 0.14.1

config.json

@@ -0,0 +1,35 @@
+{
+  "_name_or_path": "bert-large-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "benign",
+    "1": "phishing"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "benign": 0,
+    "phishing": 1
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.34.1",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7fc8fd8ff9eb431b5876bff2e94d0ba31987fc2301942b65d1306eba9d18646
+size 1340710638

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,55 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7d104fd966c5439370d740371ebeae1a9b747a93c604762957f98ecfeec61108
+size 4536