README.md

---
tags:
- ml-intern
---
# HR Conversations Multi-Label Classifier

SETFit-style classifier for **20 HR topic labels** on employee–agent conversations, trained with **5,000 synthetic + 100 real samples** and evaluated via **5-fold stratified cross-validation** (no data leakage).

## Results

| Metric | Score |
|--------|-------|
| **F1-micro (5-fold CV)** | **0.7962 ± 0.0098** |
| **F1-macro (5-fold CV)** | **0.7721** |
| Fold 1 | 0.7851 |
| Fold 2 | 0.7989 |
| Fold 3 | 0.8031 |
| Fold 4 | 0.7846 |
| Fold 5 | **0.8091** |

## Model Details

| Attribute | Value |
|-----------|-------|
| Encoder | `sentence-transformers/all-MiniLM-L6-v2` (384-dim) |
| Classifier | Multi-output Logistic Regression (scikit-learn) |
| Training samples | 5,100 (5,000 synthetic + 100 real) |
| Labels | 20 HR topics |
| Validation | 5-fold stratified cross-validation |
| Framework | Sentence-Transformers + scikit-learn |

## 20 HR Topic Labels

1. Benefits  
2. Career Development  
3. Compliance & Legal  
4. Contracts  
5. Diversity, Equity & Inclusion  
6. Expense Management  
7. Harassment  
8. Health  
9. IT & Equipment  
10. Leave & Absence  
11. Mobility  
12. Offboarding  
13. Onboarding  
14. Payroll  
15. Performance Management  
16. Recruitment  
17. Safety  
18. Timetracking  
19. Training  
20. Work Arrangements

## Usage

```python
from sentence_transformers import SentenceTransformer
import pickle, json
from huggingface_hub import hf_hub_download

# Download artifacts
classifier_path = hf_hub_download("AurelPx/hr-conversations-classifier", "setfit_classifier.pkl")
label_path      = hf_hub_download("AurelPx/hr-conversations-classifier", "setfit_label_config.json")

# Load
encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
with open(classifier_path, 'rb') as f:
    classifier = pickle.load(f)
with open(label_path) as f:
    config = json.load(f)

LABELS = config['label_names']

# Classify
sample = (
    "USER: I haven't received my payslip for March yet. Could you please check what's going on?\n"
    "AGENT: Good morning. I've checked the payroll system and it appears your March payslip "
    "was generated on the 28th but there was a distribution delay. I've resent it to your "
    "registered email. You should receive it within the next hour."
)

emb   = encoder.encode([sample])
proba = classifier.predict_proba(emb)
preds = [LABELS[i] for i, p in enumerate(proba) if p[0][1] >= 0.5]
print(preds)  # ['Payroll']
```

## Interactive Demo

Try it live: [**AurelPx/hr-classifier-demo**](https://huggingface.co/spaces/AurelPx/hr-classifier-demo)

Paste any HR conversation, adjust the threshold, and see predicted labels with probabilities instantly.

## Training Approach

1. **Data augmentation** — 5,000 synthetic HR conversations generated from real conversation templates (no LLM, no external API, no data leakage).
2. **Stratified 5-fold CV** — splits by primary label, preserving label distribution in each fold.
3. **SETFit-style pipeline** — MiniLM embeddings + Logistic Regression, fast and accurate on small data.

## Files in this Repo

| File | Description |
|------|-------------|
| `setfit_classifier.pkl` | Trained Logistic Regression classifier |
| `setfit_encoder.pkl` | SentenceTransformer MiniLM encoder (optional, for offline use) |
| `setfit_cv_results.json` | Cross-validation scores per fold |
| `setfit_label_config.json` | Label names and classification threshold |
| `training_script.py` | Full training pipeline (augmentation + CV + inference) |
| `inference.py` | Standalone inference script (DistilBERT legacy — not recommended) |
| `model.safetensors` | Legacy DistilBERT checkpoint (kept for compatibility) |

## Dataset

- [AurelPx/ml-intern-a2d69eee-datasets](https://huggingface.co/datasets/AurelPx/ml-intern-a2d69eee-datasets)
- 100 English HR conversations with multi-label annotations

## License

Apache 2.0