Instructions to use specific-AI/email-agent-phishing-detection with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use specific-AI/email-agent-phishing-detection with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-classification", model="specific-AI/email-agent-phishing-detection")

# Load model directly
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("specific-AI/email-agent-phishing-detection")
model = AutoModelForSequenceClassification.from_pretrained("specific-AI/email-agent-phishing-detection", device_map="auto")

llama-cpp-python

How to use specific-AI/email-agent-phishing-detection with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="specific-AI/email-agent-phishing-detection",
	filename="bert-base-only.gguf",
)

output = llm(
	"Once upon a time,",
	max_tokens=512,
	echo=True
)
print(output)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use specific-AI/email-agent-phishing-detection with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf specific-AI/email-agent-phishing-detection
# Run inference directly in the terminal:
llama cli -hf specific-AI/email-agent-phishing-detection

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf specific-AI/email-agent-phishing-detection
# Run inference directly in the terminal:
llama cli -hf specific-AI/email-agent-phishing-detection

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf specific-AI/email-agent-phishing-detection
# Run inference directly in the terminal:
./llama-cli -hf specific-AI/email-agent-phishing-detection

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf specific-AI/email-agent-phishing-detection
# Run inference directly in the terminal:
./build/bin/llama-cli -hf specific-AI/email-agent-phishing-detection

Use Docker

docker model run hf.co/specific-AI/email-agent-phishing-detection

LM Studio
Jan
Ollama
How to use specific-AI/email-agent-phishing-detection with Ollama:
```
ollama run hf.co/specific-AI/email-agent-phishing-detection
```

Unsloth Studio

How to use specific-AI/email-agent-phishing-detection with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for specific-AI/email-agent-phishing-detection to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for specific-AI/email-agent-phishing-detection to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for specific-AI/email-agent-phishing-detection to start chatting

Atomic Chat new
Docker Model Runner
How to use specific-AI/email-agent-phishing-detection with Docker Model Runner:
```
docker model run hf.co/specific-AI/email-agent-phishing-detection
```

Lemonade

How to use specific-AI/email-agent-phishing-detection with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull specific-AI/email-agent-phishing-detection

Run and chat with the model

lemonade run user.email-agent-phishing-detection-{{QUANT_TAG}}

List all available models

lemonade list

manu-specific commited on 1 day ago

Commit

aef4ff7

verified ·

1 Parent(s): cea9e1d

Upload 16 files

Browse files

Files changed (17) hide show

.gitattributes +1 -0
LICENSE +21 -0
README.md +147 -0
bert-base-only.gguf +3 -0
classifier_b.npy +3 -0
classifier_w.npy +3 -0
config.json +34 -0
metadata.json +1 -0
model.safetensors +3 -0
pooler_b.npy +3 -0
pooler_w.npy +3 -0
special_tokens_map.json +7 -0
tokenizer.json +0 -0
tokenizer_config.json +57 -0
trainer_state.json +449 -0
training_args.bin +3 -0
vocab.txt +0 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+bert-base-only.gguf filter=lfs diff=lfs merge=lfs -text

LICENSE ADDED Viewed

	@@ -0,0 +1,21 @@

+MIT License
+Copyright (c) 2026 Specific AI Inc.
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md CHANGED Viewed

@@ -1,3 +1,150 @@
 ---
 license: mit
 ---

 ---
 license: mit
+language:
+  - en
+library_name: transformers
+pipeline_tag: text-classification
+tags:
+  - email
+  - phishing
+  - classification
+  - bert
+  - specific-ai
+  - gguf
+base_model: google-bert/bert-base-uncased
 ---
+# specific-AI/email-agent-phishing-detection
+A compact **BERT** phishing detector distilled with **[Specific AI](https://specific.ai)**.
+It classifies email content as phishing or not, for use in email agents and
+security-aware inbox workflows.
+| | |
+|---|---|
+| **Task** | Single-label text classification |
+| **Base model** | `bert-base-uncased` |
+| **Training data** | ~15,000 examples |
+| **License** | MIT |
+## Input format
+Examples were trained on emails formatted as plain text with `From`, `Subject`,
+and body (blank line between the headers and the body):
+```text
+From: <from>
+Subject: <subject>
+<body>
+```
+Pass inputs in this same shape at inference time for best results.
+## Labels
+| Label | Meaning |
+|---|---|
+| **True** | Phishing detected |
+| **False** | Phishing was not detected |
+## Evaluation
+Compared against **gpt-5.4-mini** as a teacher / baseline on the same evaluation set:
+| Metric | gpt-5.4-mini | SpecificAI |
+|---|---:|---:|
+| Accuracy | 0.971 | **0.975** |
+| Precision | 0.976 | 0.975 |
+| Recall | 0.971 | **0.975** |
+| F1 score | 0.972 | **0.975** |
+## Repository contents
+This card ships both a full Hugging Face checkpoint and GGUF-ready artifacts:
+- Full `BertForSequenceClassification` weights (`model.safetensors`) + tokenizer
+- Head layers as NumPy files (`pooler_*.npy`, `classifier_*.npy`) for GGUF / Lemonade fusion
+- Encoder GGUF: `bert-base-only.gguf` (CLS pooling; use with raw / unnormalized embeddings)
+## Quick start — Transformers
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+model_id = "specific-AI/email-agent-phishing-detection"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForSequenceClassification.from_pretrained(model_id)
+model.eval()
+text = """From: security@paypa1-support.com
+Subject: Your account will be locked
+Verify your password at http://example-phish.test/login to keep access."""
+inputs = tokenizer(text, return_tensors="pt", truncation=True)
+with torch.no_grad():
+    logits = model(**inputs).logits
+pred = model.config.id2label[int(logits.argmax(-1))]
+print(pred)  # "True" or "False"
+```
+## Quick start — Lemonade + specific-ai-tools
+When running the GGUF encoder through Lemonade Server:
+```bash
+pip install specific-ai-tools
+```
+```python
+from specific_ai_tools.embedding_heads import LemonadeEmbeddingClassifier
+classifier = LemonadeEmbeddingClassifier(
+    lemonade_model_name="user.email-agent-phishing-detection",
+    checkpoint="specific-AI/email-agent-phishing-detection:bert-base-only.gguf",
+    lemonade_base_url="http://localhost:13305",
+)
+text = """From: noreply@secure-mail-alert.com
+Subject: Reset your password now
+Click here to reset your password immediately."""
+result = classifier.predict_one(text)
+print(result.predicted_labels, result.predicted_confidences)
+```
+See the Specific AI toolkit docs for llama-cpp and other embedding backends.
+## Intended use
+- Email / inbox agents that need a fast on-device or CPU phishing signal
+- Pre-filter or assistive scoring alongside other security controls
+**Out of scope:** sole authority for blocking, quarantine, or legal determinations.
+Treat outputs as a high-throughput classifier signal and keep human / policy review
+in the loop for high-impact actions.
+## About Us
+**[Specific AI](https://specific.ai)** is the automatic SLM distillation platform
+that turns task prompts into production-grade small language models in days —
+not weeks — so your subject matter experts can ship models without waiting on
+scarce data-science bandwidth.
+We help enterprises move agentic AI from prototype to production with SLMs that
+are typically **1,000×–10,000× smaller** than teacher LLMs, run in
+**milliseconds** on CPUs or edge devices, and deliver the same or better
+task quality at a fraction of the cost — self-hosted on your cloud or
+downloaded for your own inference stack.
+**Prompt → Distill → Deploy.** Bring your prompt and data, drop them into
+Specific AI, and get a validated small model ready to test and ship.
+Ready to create SLMs at scale? Visit **[specific.ai](https://specific.ai)**.
+## License
+MIT — see [LICENSE](LICENSE).
+Copyright (C) 2026 Specific AI Inc. All rights reserved.

bert-base-only.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1adb1a65d1169a8ea677c8ba9dbb1aef69eae988a6e68f04bea6ba2f16bc9090
+size 436208736

classifier_b.npy ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dd59364bee2cee762147077cd9aba242977b9f32eb58e3dfd222680b54c2e228
+size 136

classifier_w.npy ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:336985753d38c35011b4ee6bec704290c37c7937520d016e472b1c3d1e23bcf9
+size 6272

config.json ADDED Viewed

	@@ -0,0 +1,34 @@

+{
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "False",
+    "1": "True"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "False": 0,
+    "True": 1
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "transformers_version": "4.57.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

metadata.json ADDED Viewed

	@@ -0,0 +1 @@

+ {"task_type": "ClassificationResponse", "label_names": ["False", "True"], "client_id": "40b3a00e282ee2db1338c3bba881b99125fc3ba1d7f5c29232ce1ad63fab1e75", "llm_usecase_id": "6a59bb32de331510eb8ffe24", "distillation_event_id": "6a5e4188194ffc4db094fefc", "tokenizer_files": ["tokenizer_config.json", "special_tokens_map.json", "vocab.txt", "added_tokens.json", "tokenizer.json"], "model_name": "bert-base-uncased", "input_names": ["input_ids", "attention_mask", "token_type_ids"]}

model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a62ec1ba6341895747b4ab71efdb0e162cbed09c7615a71057f7304c0a8327c
+size 437958648

pooler_b.npy ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4f0642cc5fbc74d7f49f4d0a6e580885bb930082cef47883b2c3beb0d15d3bd2
+size 3200

pooler_w.npy ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c72e03198eb5e45d958ccc9c0b5b261935ab3c0736a9d7b93c4b7340f1c5ac8
+size 2359424

special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,7 @@

+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff

tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,57 @@

+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "left",
+  "unk_token": "[UNK]"
+}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,449 @@

+{
+  "best_global_step": 7840,
+  "best_metric": 0.9680508203881729,
+  "best_model_checkpoint": "/app/checkpoints/checkpoint-7840",
+  "epoch": 5.0,
+  "eval_steps": 500,
+  "global_step": 7840,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0,
+      "eval_accuracy": 0.38692185007974483,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.38692185007974483,
+          "confusion_matrix": {
+            "fn": 1904,
+            "fp": 18,
+            "tn": 1187,
+            "tp": 26
+          },
+          "f_beta_score": 0.02634245187436677,
+          "precision": 0.5909090909090909,
+          "recall": 0.013471502590673576
+        },
+        "1": {
+          "accuracy": 0.38692185007974483,
+          "confusion_matrix": {
+            "fn": 18,
+            "fp": 1904,
+            "tn": 26,
+            "tp": 1187
+          },
+          "f_beta_score": 0.5526070763500931,
+          "precision": 0.38401811711420253,
+          "recall": 0.9850622406639005
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 1922,
+        "fp": 1922,
+        "tn": 1213,
+        "tp": 1213
+      },
+      "eval_f_beta_score": 0.22862279397747687,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            26,
+            1904
+          ],
+          [
+            18,
+            1187
+          ]
+        ]
+      },
+      "eval_loss": 0.6828452944755554,
+      "eval_precision": 0.5113864040118531,
+      "eval_recall": 0.38692185007974483,
+      "eval_runtime": 18.6915,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 167.723,
+      "eval_steps_per_second": 20.972,
+      "step": 0
+    },
+    {
+      "epoch": 1.0,
+      "grad_norm": 0.09250472486019135,
+      "hard_loss": 0.19278878086653112,
+      "learning_rate": 1.1780364852141454e-05,
+      "loss": 0.1928,
+      "step": 1568
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9559808612440192,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.9559808612440192,
+          "confusion_matrix": {
+            "fn": 94,
+            "fp": 44,
+            "tn": 1161,
+            "tp": 1836
+          },
+          "f_beta_score": 0.9637795275590552,
+          "precision": 0.9765957446808511,
+          "recall": 0.9512953367875647
+        },
+        "1": {
+          "accuracy": 0.9559808612440192,
+          "confusion_matrix": {
+            "fn": 44,
+            "fp": 94,
+            "tn": 1836,
+            "tp": 1161
+          },
+          "f_beta_score": 0.9439024390243902,
+          "precision": 0.9250996015936255,
+          "recall": 0.9634854771784233
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 138,
+        "fp": 138,
+        "tn": 2997,
+        "tp": 2997
+      },
+      "eval_f_beta_score": 0.9561393707219671,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            1836,
+            94
+          ],
+          [
+            44,
+            1161
+          ]
+        ]
+      },
+      "eval_loss": 0.20423130691051483,
+      "eval_precision": 0.956802171341104,
+      "eval_recall": 0.9559808612440192,
+      "eval_runtime": 18.3681,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 170.676,
+      "eval_steps_per_second": 21.341,
+      "step": 1568
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 0.2099187821149826,
+      "hard_loss": 0.14008313520248428,
+      "learning_rate": 1.1160366460299146e-05,
+      "loss": 0.1401,
+      "step": 3136
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.9661881977671452,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.9661881977671452,
+          "confusion_matrix": {
+            "fn": 45,
+            "fp": 61,
+            "tn": 1144,
+            "tp": 1885
+          },
+          "f_beta_score": 0.9726522187822497,
+          "precision": 0.9686536485097637,
+          "recall": 0.9766839378238342
+        },
+        "1": {
+          "accuracy": 0.9661881977671452,
+          "confusion_matrix": {
+            "fn": 61,
+            "fp": 45,
+            "tn": 1885,
+            "tp": 1144
+          },
+          "f_beta_score": 0.9557226399331662,
+          "precision": 0.9621530698065601,
+          "recall": 0.9493775933609958
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 106,
+        "fp": 106,
+        "tn": 3029,
+        "tp": 3029
+      },
+      "eval_f_beta_score": 0.9661449962900183,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            1885,
+            45
+          ],
+          [
+            61,
+            1144
+          ]
+        ]
+      },
+      "eval_loss": 0.14875319600105286,
+      "eval_precision": 0.9661550209699359,
+      "eval_recall": 0.9661881977671452,
+      "eval_runtime": 18.3189,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 171.134,
+      "eval_steps_per_second": 21.399,
+      "step": 3136
+    },
+    {
+      "epoch": 3.0,
+      "grad_norm": 0.020211078226566315,
+      "hard_loss": 0.11091282128492301,
+      "learning_rate": 1.0540368068456843e-05,
+      "loss": 0.1109,
+      "step": 4704
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.9665071770334929,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.9665071770334929,
+          "confusion_matrix": {
+            "fn": 51,
+            "fp": 54,
+            "tn": 1151,
+            "tp": 1879
+          },
+          "f_beta_score": 0.9728190525498317,
+          "precision": 0.9720641489912054,
+          "recall": 0.9735751295336788
+        },
+        "1": {
+          "accuracy": 0.9665071770334929,
+          "confusion_matrix": {
+            "fn": 54,
+            "fp": 51,
+            "tn": 1879,
+            "tp": 1151
+          },
+          "f_beta_score": 0.9563772330702118,
+          "precision": 0.9575707154742097,
+          "recall": 0.9551867219917013
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 105,
+        "fp": 105,
+        "tn": 3030,
+        "tp": 3030
+      },
+      "eval_f_beta_score": 0.9664993101342201,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            1879,
+            51
+          ],
+          [
+            54,
+            1151
+          ]
+        ]
+      },
+      "eval_loss": 0.1719985455274582,
+      "eval_precision": 0.9664933077191225,
+      "eval_recall": 0.9665071770334929,
+      "eval_runtime": 18.5606,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 168.906,
+      "eval_steps_per_second": 21.12,
+      "step": 4704
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 0.007895365357398987,
+      "hard_loss": 0.07477579451819463,
+      "learning_rate": 9.920369676614537e-06,
+      "loss": 0.0748,
+      "step": 6272
+    },
+    {
+      "epoch": 4.0,
+      "eval_accuracy": 0.9559808612440192,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.9559808612440192,
+          "confusion_matrix": {
+            "fn": 101,
+            "fp": 37,
+            "tn": 1168,
+            "tp": 1829
+          },
+          "f_beta_score": 0.9636459430979979,
+          "precision": 0.9801714898177921,
+          "recall": 0.9476683937823834
+        },
+        "1": {
+          "accuracy": 0.9559808612440192,
+          "confusion_matrix": {
+            "fn": 37,
+            "fp": 101,
+            "tn": 1829,
+            "tp": 1168
+          },
+          "f_beta_score": 0.9442198868229588,
+          "precision": 0.9204097714736013,
+          "recall": 0.9692946058091286
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 138,
+        "fp": 138,
+        "tn": 2997,
+        "tp": 2997
+      },
+      "eval_f_beta_score": 0.9561791495377358,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            1829,
+            101
+          ],
+          [
+            37,
+            1168
+          ]
+        ]
+      },
+      "eval_loss": 0.2560373842716217,
+      "eval_precision": 0.957200877184698,
+      "eval_recall": 0.9559808612440192,
+      "eval_runtime": 18.6002,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 168.546,
+      "eval_steps_per_second": 21.075,
+      "step": 6272
+    },
+    {
+      "epoch": 5.0,
+      "grad_norm": 0.028711551800370216,
+      "hard_loss": 0.04914131778617753,
+      "learning_rate": 9.300371284772232e-06,
+      "loss": 0.0491,
+      "step": 7840
+    },
+    {
+      "epoch": 5.0,
+      "eval_accuracy": 0.9681020733652312,
+      "eval_class_metrics": {
+        "0": {
+          "accuracy": 0.9681020733652312,
+          "confusion_matrix": {
+            "fn": 40,
+            "fp": 60,
+            "tn": 1145,
+            "tp": 1890
+          },
+          "f_beta_score": 0.9742268041237113,
+          "precision": 0.9692307692307692,
+          "recall": 0.9792746113989638
+        },
+        "1": {
+          "accuracy": 0.9681020733652312,
+          "confusion_matrix": {
+            "fn": 60,
+            "fp": 40,
+            "tn": 1890,
+            "tp": 1145
+          },
+          "f_beta_score": 0.9581589958158996,
+          "precision": 0.9662447257383966,
+          "recall": 0.950207468879668
+        }
+      },
+      "eval_confusion_matrix": {
+        "fn": 100,
+        "fp": 100,
+        "tn": 3035,
+        "tp": 3035
+      },
+      "eval_f_beta_score": 0.9680508203881729,
+      "eval_label_confusion_matrix": {
+        "labels": [
+          "False",
+          "True"
+        ],
+        "matrix": [
+          [
+            1890,
+            40
+          ],
+          [
+            60,
+            1145
+          ]
+        ]
+      },
+      "eval_loss": 0.24635428190231323,
+      "eval_precision": 0.9680830236459818,
+      "eval_recall": 0.9681020733652312,
+      "eval_runtime": 18.3281,
+      "eval_sample_size": 3135,
+      "eval_samples_per_second": 171.049,
+      "eval_steps_per_second": 21.388,
+      "step": 7840
+    }
+  ],
+  "logging_steps": 500,
+  "max_steps": 31360,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 20,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "EarlyStoppingCallback": {
+      "args": {
+        "early_stopping_patience": 3,
+        "early_stopping_threshold": 0.01
+      },
+      "attributes": {
+        "early_stopping_patience_counter": 3
+      }
+    },
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 1.494801327708096e+16,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5b3e8784bad28e4d48ad30e61a905f913cb182018f0244adf5b0a7e2bd4332d9
+size 5905

vocab.txt ADDED Viewed

The diff for this file is too large to render. See raw diff