base_model: bert-base-uncased
library_name: peft
tags:
- mental-health
- burnout-detection
- lora
- dual-head-classifier
- workplace-wellbeing
SentinelAI BERT Filter - LoRA Adapters
LoRA fine-tuned BERT model for employee mental health classification in workplace messages. Part of the SentinelAI system for automated burnout detection via Slack message analysis.
Model Description
- Base Model: bert-base-uncased (110M parameters)
- Fine-tuning Method: LoRA (Low-Rank Adaptation)
- Task: Dual-head classification (category + severity)
- Trainable Parameters: 303,371 / 109,785,611 (0.28%)
- Developed by: Team Rocket Number One, King's College London
- License: MIT (project-specific)
Architecture
Dual-Head Classifier:
- Category Head: 7-class classification
- neutral, humor_sarcasm, stress, burnout, depression, harassment, suicidal_ideation
- Severity Head: 4-stage classification
- none, early, middle, late
- Binary Routing: 5 risk categories trigger escalation to LLM agents
Training Details
Dataset
- Total Examples: 7,000 (mixed dataset for quality + diversity)
- 5,000 from v0.1 (natural Slack-style phrasing)
- 2,000 from v0.2 (lexically diverse, synonym-enhanced)
- Splits: 80% train (5,600), 10% val (700), 10% test (700)
- Lexical Diversity: TTR 0.35 (exceeds 0.3 quality threshold)
- Clinical Grounding: Maslach Burnout Inventory (MBI), DSM-5, UK Equality Act 2010
Hyperparameters
LoRA Configuration:
r: 8
lora_alpha: 16
lora_dropout: 0.1
target_modules: ["query", "value"]
task_type: FEATURE_EXTRACTION
Training:
epochs: 3
batch_size: 16
learning_rate: 3e-4
optimizer: AdamW
scheduler: Linear warmup + decay
max_sequence_length: 128
loss_function: CrossEntropyLoss (category + severity summed)
Hardware & Performance
- GPU: NVIDIA GeForce GTX 1080 (8GB VRAM)
- Training Time: ~3 minutes (1 min/epoch)
- Training Regime: fp32
Results
Test Set Performance
| Metric | Score |
|---|---|
| Category Accuracy | 76.29% |
| Severity Accuracy | 78.29% |
| Test Loss | 1.1840 |
Performance Context:
- Category: 5.3x better than random (7-class baseline: 14.3%)
- Severity: 3.1x better than random (4-class baseline: 25%)
- Low/no overfitting: Test accuracy matches validation accuracy
Usage
Loading the Model (Production Pattern)
The repository uses a centralised Model Factory to handle architecture initialisation and weight loading. It includes Auto-Download logic that pulls the latest checkpoint from Hugging Face Hub if it is not found locally.
import torch
from services.model_factory import load_production_model
# Inference
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
# This will:
# 1. Initialise DualHeadBERTClassifier
# 2. Apply LoRA adapters
# 3. Check for 'dual_head_classifier.pt' locally
# 4. If missing, download latest from OguzhanKOG/sentinelai-bert-filter
# 5. Load trained weights and return model in eval mode
model = load_production_model(device=device)
# Model is ready for inference
message = "I'm completely overwhelmed with work and can't sleep anymore"
# ... standard tokenization using config.MODEL_NAME ...
Configuration
All parameters (LoRA rank, Alpha, Model Backbone, Paths) are centralised in filter/config.py. To change the backbone or parameters across the entire service, update this file only.
Limitations
- Synthetic Training Data: Model trained on generated examples, not real workplace messages
- English Only: No multilingual support
- Context Window: Limited to 128 tokens (Slack message-sized)
- Not a Clinical Tool: Designed for workplace wellbeing monitoring, not medical diagnosis
- Bias Risk: May reflect biases in synthetic data generation process
Intended Use
Primary Use Case: Fast, cost-effective gatekeeper filter in SentinelAI architecture. Routes high-risk messages to expensive LLM agents for detailed analysis, while filtering out low-risk neutral messages.
Architecture Position:
Slack Message → BERT Filter (this model) → [if risk] → LLM Agent Analysis → HR Alert
Not Intended For:
- Clinical diagnosis or medical decision-making
- Standalone mental health assessment
- Real-time crisis intervention (human oversight required)
- Legal or disciplinary actions without human review
Training Logs
Full training metrics available in training_log.json:
- Epoch-by-epoch train/val losses
- Category and severity accuracies per epoch
- Final test set evaluation results
Repository
Full implementation available in the project repository (Private).
Branch: feature/filter
Framework versions
- PEFT 0.18.1