Update model card with detailed limitations and known issues

README.md

@@ -331,23 +331,57 @@ def predict_batch(texts: list, batch_size: int = 32) -> list:
 
 ---
 
-## Limitations
+## Limitations & Known Issues
 
-### Known Issues
+### ⚠️ Evaluation Limitations
 
-| Issue | Description | Mitigation |
-|-------|-------------|------------|
-| **GROOMING Boundary** | Some GROOMING cases may be classified as SEXUAL | Clear category definitions in training |
-| **Context Blindness** | Single message analysis only | Implement multi-turn context |
-| **SAFE False Positives** | Some safe content may be flagged | Threshold tuning per use case |
-| **Dialect Variation** | May underperform on regional dialects | Expanded training data |
+**Note**: Two separate evaluation sets exist:
+- **Automated Test Set**: 17,033 samples from test.csv → Macro F1: 0.9165, MCC: 0.9045
+- **Manual Edge Case Test**: 22 hand-picked samples → 86.4% accuracy (19/22 correct)
+
+| Limitation | Details | Impact |
+|------------|---------|--------|
+| **Small Manual Test Set** | Edge case validation on only 22 samples (86.4%) | Manual test not statistically significant; automated metrics (17K samples) more reliable |
+| **No Per-Class Metrics** | Only Macro F1 and MCC reported for 17K test set | Cannot assess individual category performance (e.g., SELF_HARM Precision/Recall vs SAFE) |
+| **No Confusion Matrix** | Category confusion patterns not documented | Unclear which categories are most confused beyond GROOMING/SEXUAL boundary |
+| **No PR/ROC Curves** | Precision-Recall and ROC analysis not performed | Optimal threshold selection methodology not documented |
+| **No Calibration Analysis** | Model confidence calibration not tested | Unknown if 0.7 confidence truly represents 70% probability |
+
+### ⚠️ Architectural Limitations
+
+| Limitation | Details | Impact |
+|------------|---------|--------|
+| **Short Context Window** | Max sequence length: 64 tokens | Long messages may lose critical information; truncation may remove key context |
+| **Single-Turn Only** | No conversation history analysis | GROOMING patterns often emerge across multiple messages ("Kaç yaşındasın?", "Nerelisin?", "Fotoğraf atar mısın?" may each appear SAFE individually) |
+| **No Temporal Patterns** | No escalation detection capability | Cannot detect behavior changes over time; user history not considered |
+| **Static Analysis** | Each message analyzed independently | Contextual red flags from message sequences not captured |
+
+### ⚠️ Data & Coverage Limitations
+
+| Limitation | Details | Impact |
+|------------|---------|--------|
+| **Dialect/Slang Gaps** | Regional dialects and internet slang underrepresented | Performance may degrade on: "napıon", "nbr", "slm", "mrb", regional variations |
+| **No Adversarial Testing** | Evasion techniques not systematically tested | Unknown robustness against: "S 3 x" instead of "sex", character substitution, unicode tricks |
+| **Synthetic Data Bias** | 97.5% of training data is LLM-generated | May not capture real-world linguistic patterns; potential distribution shift |
+| **Spelling Error Tolerance** | Not explicitly tested | Common typos and intentional misspellings may bypass detection |
+
+### ⚠️ Production Deployment Considerations
+
+| Consideration | Details | Recommendation |
+|---------------|---------|----------------|
+| **Threshold Selection** | Current thresholds (0.3, 0.5, 0.75) are heuristic | Perform PR curve analysis for your specific use case; adjust based on FP/FN tolerance |
+| **Confidence Calibration** | Model may be over/under-confident | Consider temperature scaling or Platt calibration before production |
+| **Category Boundaries** | GROOMING ↔ SEXUAL boundary is known issue | Review flagged content in these categories; implement human review for edge cases |
+| **Real-Time Context** | No session-level analysis | Consider implementing sliding window or conversation aggregation layer |
 
 ### Not Suitable For
 
 - Languages other than Turkish
-- Adult content moderation (different domain)
-- Sole decision-making without human review
+- Adult content moderation (requires different domain expertise)
+- Sole decision-making without human review for high-stakes situations
 - Legal evidence or court proceedings
+- Detection of sophisticated, multi-turn grooming attempts without additional context layer
+- Highly informal/slang-heavy communications without additional preprocessing
 
 ---