Model Card for llama3-1-8b-tuned

Abstract

This repository hosts a fine‑tuned Llama 3.1 8B model that leverages LoRA (Low‑Rank Adaptation) via the Unsloth framework. The model is adapted for email‑security tasks using a curated phishing‑email training dataset and demonstrates state‑of‑the‑art performance (93.4 % accuracy, 97.4 % precision, 89.1 % recall) compared to the base model.

Model Details

Model Description

Developed by: Montimage
Model type: Large Language Model (LLM)
Language(s): English
License: Apache‑2.0
Finetuned from model: unsloth/llama‑3.1‑8b‑bnb‑4bit

Model Sources

Repository: https://huggingface.co/nosadaniel/llama3-1-8b-tuned
Demo: https://huggingface.co/spaces/nosadaniel/fined-model

How to Get Started with the Model

You can test the model using the Hugging Face Space.

Locally using Ollama

Install Ollama – https://ollama.com

Pull the base model (chooanse one):

ollama pull unsloth/llama-3.1-8b-bnb-4bit
# or
ollama pull llama-3.1-8b

Create a merged‑model manifest (no file extension) with the following content and place it in a folder of your choice:
```
FROM llama3.1:8b
ADAPTER /path/to/your/downloaded/adapter
```
Then run:
```
ollama create merged-model --from ./merged-model
```
Run the merged model:
```
ollama run merged-model
```

Training Details

Training Data

Dataset: Phishing Email Training Dataset
Link: https://huggingface.co/datasets/nosadaniel/phishing-email-training-dataset

Training Procedure

Fine‑tuning method: LoRA via Unsloth
Precision: fp16 mixed precision
Epochs: 3 (full dataset)
Learning rate: 2e‑4
Batch size: 32

Model Comparison Table

Model	Samples	Accuracy	Precision	Recall	F1‑Score	Specificity	FPR	FNR	MCC	Validity	Avg Response Time (s)	Total Input Tokens	Total Output Tokens	Avg Input Tokens	Avg Output Tokens	Quality Mean	Quality Std	Excellent (%)	Good (%)	Fair (%)	Poor (%)
llama3.1:8b	256	0.914	0.902	0.930	0.915	0.898	0.102	0.070	0.829	96.1 %	21.26	144,247	97,170	563.5	379.6	0.902	0.135	90.2	1.2	5.5	3.1
llama3.1:8b_sft (fine‑tuned)	256	0.934	0.974	0.891	0.931	0.977	0.023	0.109	0.870	100.0 %	25.65	144,247	102,171	563.5	399.1	0.943	0.108	93.4	0.0	6.6	0.0

Model Performance Analysis – llama3.1:8b_sft

Total Responses: 256
Accuracy: 93.4 % (239/256)
Valid Responses: 100 % (256/256)
Average Confidence: 0.927

Classification Metrics

Metric	Value
Accuracy	93.4 %
Precision	97.4 %
Recall	89.1 %
F1‑Score	93.1 %
Specificity	97.7 %

Confusion Matrix

	Predicted Positive	Predicted Negative
Actual Positive	114 (TP)	14 (FN)
Actual Negative	3 (FP)	125 (TN)

Additional Metrics

False Positive Rate: 2.3 %
False Negative Rate: 10.9 %
Negative Predictive Value: 89.9 %
Matthews Correlation Coefficient: 0.870

Performance Insights

✅ High Precision – Low false‑positive rate, fostering user trust.
✅ High Recall – Catches the vast majority of phishing attempts.
✅ Excellent F1‑Score – Well‑balanced precision and recall.
✅ Strong MCC – Strong overall correlation between predictions and ground truth.

Citation

BibTeX:

@dataset{llama3-1-8b-tuned,
  title={Fine‑tuned Llama 3.1 8B model for Email Phishing Detection},
  author={Montimage, Nosakhare Daniel Ahanor, Luong89},
  year={2025},
  publisher={Montimage}
}

Model Card Authors

Montimage Email Security Research Division AI/ML Engineering Team Cybersecurity Domain Experts

Framework versions

PEFT 0.17.1

Downloads last month: -

nosadaniel
/

llama3-1-8b-tuned