Upload README.md

README.md

@@ -4,86 +4,117 @@ tags:
 ---
 # HR Conversations Multi-Label Classifier
 
-A fine-tuned **DistilBERT-base-uncased** (66M parameters) for multi-label classification of HR support conversations.
+SETFit-style classifier for **20 HR topic labels** on employee–agent conversations, trained with **5,000 synthetic + 100 real samples** and evaluated via **5-fold stratified cross-validation** (no data leakage).
+
+## Results
+
+| Metric | Score |
+|--------|-------|
+| **F1-micro (5-fold CV)** | **0.7962 ± 0.0098** |
+| **F1-macro (5-fold CV)** | **0.7721** |
+| Fold 1 | 0.7851 |
+| Fold 2 | 0.7989 |
+| Fold 3 | 0.8031 |
+| Fold 4 | 0.7846 |
+| Fold 5 | **0.8091** |
 
 ## Model Details
 
 | Attribute | Value |
 |-----------|-------|
-| Base Model | `distilbert/distilbert-base-uncased` |
-| Task | Multi-label text classification |
+| Encoder | `sentence-transformers/all-MiniLM-L6-v2` (384-dim) |
+| Classifier | Multi-output Logistic Regression (scikit-learn) |
+| Training samples | 5,100 (5,000 synthetic + 100 real) |
 | Labels | 20 HR topics |
-| Training Data | 100 synthetic HR conversations |
-| Framework | Hugging Face Transformers |
+| Validation | 5-fold stratified cross-validation |
+| Framework | Sentence-Transformers + scikit-learn |
 
 ## 20 HR Topic Labels
 
-1. Benefits
-2. Career Development
-3. Compliance & Legal
-4. Contracts
-5. Diversity, Equity & Inclusion
-6. Expense Management
-7. Harassment
-8. Health
-9. IT & Equipment
-10. Leave & Absence
-11. Mobility
-12. Offboarding
-13. Onboarding
-14. Payroll
-15. Performance Management
-16. Recruitment
-17. Safety
-18. Timetracking
-19. Training
+1. Benefits  
+2. Career Development  
+3. Compliance & Legal  
+4. Contracts  
+5. Diversity, Equity & Inclusion  
+6. Expense Management  
+7. Harassment  
+8. Health  
+9. IT & Equipment  
+10. Leave & Absence  
+11. Mobility  
+12. Offboarding  
+13. Onboarding  
+14. Payroll  
+15. Performance Management  
+16. Recruitment  
+17. Safety  
+18. Timetracking  
+19. Training  
 20. Work Arrangements
 
 ## Usage
 
 ```python
-from transformers import AutoTokenizer, AutoModelForSequenceClassification
-import torch
-
-model_id = "AurelPx/hr-conversations-classifier"
-tokenizer = AutoTokenizer.from_pretrained(model_id)
-model = AutoModelForSequenceClassification.from_pretrained(model_id)
-
-LABELS = [
-    "Benefits", "Career Development", "Compliance & Legal", "Contracts",
-    "Diversity, Equity & Inclusion", "Expense Management", "Harassment", "Health",
-    "IT & Equipment", "Leave & Absence", "Mobility", "Offboarding",
-    "Onboarding", "Payroll", "Performance Management", "Recruitment",
-    "Safety", "Timetracking", "Training", "Work Arrangements"
-]
-
-def classify(text, threshold=0.3):
-    inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
-    with torch.no_grad():
-        logits = model(**inputs).logits
-    probs = torch.sigmoid(logits).numpy()[0]
-    return [LABELS[i] for i, p in enumerate(probs) if p >= threshold]
-
-# Example
-conversation = "USER: I haven't received my payslip for March yet..."
-print(classify(conversation))  # ['Payroll']
+from sentence_transformers import SentenceTransformer
+import pickle, json
+from huggingface_hub import hf_hub_download
+
+# Download artifacts
+classifier_path = hf_hub_download("AurelPx/hr-conversations-classifier", "setfit_classifier.pkl")
+label_path      = hf_hub_download("AurelPx/hr-conversations-classifier", "setfit_label_config.json")
+
+# Load
+encoder = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
+with open(classifier_path, 'rb') as f:
+    classifier = pickle.load(f)
+with open(label_path) as f:
+    config = json.load(f)
+
+LABELS = config['label_names']
+
+# Classify
+sample = (
+    "USER: I haven't received my payslip for March yet. Could you please check what's going on?\n"
+    "AGENT: Good morning. I've checked the payroll system and it appears your March payslip "
+    "was generated on the 28th but there was a distribution delay. I've resent it to your "
+    "registered email. You should receive it within the next hour."
+)
+
+emb   = encoder.encode([sample])
+proba = classifier.predict_proba(emb)
+preds = [LABELS[i] for i, p in enumerate(proba) if p[0][1] >= 0.5]
+print(preds)  # ['Payroll']
 ```
 
-## Training Notes
+## Interactive Demo
+
+Try it live: [**AurelPx/hr-classifier-demo**](https://huggingface.co/spaces/AurelPx/hr-classifier-demo)
+
+Paste any HR conversation, adjust the threshold, and see predicted labels with probabilities instantly.
+
+## Training Approach
+
+1. **Data augmentation** — 5,000 synthetic HR conversations generated from real conversation templates (no LLM, no external API, no data leakage).
+2. **Stratified 5-fold CV** — splits by primary label, preserving label distribution in each fold.
+3. **SETFit-style pipeline** — MiniLM embeddings + Logistic Regression, fast and accurate on small data.
 
-- **Dataset size**: 100 conversations (small dataset)
-- **Split**: 80 train / 20 validation
-- **Epochs**: 4-8 with early stopping
-- **Limitations**: With only 100 samples across 20 classes, the model is in a very low-data regime. For production use, collect >500 samples per label or apply data augmentation.
+## Files in this Repo
 
-## Links
+| File | Description |
+|------|-------------|
+| `setfit_classifier.pkl` | Trained Logistic Regression classifier |
+| `setfit_encoder.pkl` | SentenceTransformer MiniLM encoder (optional, for offline use) |
+| `setfit_cv_results.json` | Cross-validation scores per fold |
+| `setfit_label_config.json` | Label names and classification threshold |
+| `training_script.py` | Full training pipeline (augmentation + CV + inference) |
+| `inference.py` | Standalone inference script (DistilBERT legacy — not recommended) |
+| `model.safetensors` | Legacy DistilBERT checkpoint (kept for compatibility) |
 
-- Dataset: [AurelPx/ml-intern-a2d69eee-datasets](https://huggingface.co/datasets/AurelPx/ml-intern-a2d69eee-datasets)
+## Dataset
 
-<!-- ml-intern-provenance -->
-## Generated by ML Intern
+- [AurelPx/ml-intern-a2d69eee-datasets](https://huggingface.co/datasets/AurelPx/ml-intern-a2d69eee-datasets)
+- 100 English HR conversations with multi-label annotations
 
-This model repository was generated by [ML Intern](https://github.com/huggingface/ml-intern), an agent for machine learning research and development on the Hugging Face Hub.
+## License
 
-- Try ML Intern: https://smolagents-ml-intern.hf.space
-- Source code: https://github.com/huggingface/ml-intern
+Apache 2.0