Update README.md
Browse files
README.md
CHANGED
|
@@ -36,61 +36,6 @@ A ModernBERT-based classifier that detects user satisfaction and dissatisfaction
|
|
| 36 |
| `WRONG_ANSWER` | System provided incorrect information | "No, that's wrong", "That's not right" |
|
| 37 |
| `WANT_DIFFERENT` | User wants alternative options | "Show me others", "What else?" |
|
| 38 |
|
| 39 |
-
## Quick Start
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
## Performance
|
| 44 |
-
|
| 45 |
-
| Metric | Score |
|
| 46 |
-
|--------|-------|
|
| 47 |
-
| Validation Accuracy | 95.47% |
|
| 48 |
-
| Validation F1 (Macro) | 95.41% |
|
| 49 |
-
| Adversarial Test Accuracy | 93.2% |
|
| 50 |
-
|
| 51 |
-
### Per-Class Performance
|
| 52 |
-
|
| 53 |
-
| Class | Adversarial Accuracy |
|
| 54 |
-
|-------|---------------------|
|
| 55 |
-
| NEED_CLARIFICATION | 94.3% |
|
| 56 |
-
| WRONG_ANSWER | 97.0% |
|
| 57 |
-
| WANT_DIFFERENT | 88.6% |
|
| 58 |
-
|
| 59 |
-
## Training Data
|
| 60 |
-
|
| 61 |
-
**Dataset**: [rootfs/user-satisfaction-dataset](https://huggingface.co/datasets/rootfs/user-satisfaction-dataset)
|
| 62 |
-
|
| 63 |
-
| Split | Examples |
|
| 64 |
-
|-------|----------|
|
| 65 |
-
| Train | 13,422 |
|
| 66 |
-
| Validation | 1,494 |
|
| 67 |
-
| Adversarial Test | 103 |
|
| 68 |
-
|
| 69 |
-
## Use Cases
|
| 70 |
-
|
| 71 |
-
### Semantic Routing
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
### Quality Monitoring
|
| 75 |
-
Track satisfaction rates across your conversational AI system.
|
| 76 |
-
|
| 77 |
-
### Feedback Loop
|
| 78 |
-
Automatically categorize user feedback for model improvement.
|
| 79 |
-
|
| 80 |
-
## Training Details
|
| 81 |
-
|
| 82 |
-
| Parameter | Value |
|
| 83 |
-
|-----------|-------|
|
| 84 |
-
| Base Model | answerdotai/ModernBERT-base |
|
| 85 |
-
| Epochs | 10 |
|
| 86 |
-
| Learning Rate | 2e-5 |
|
| 87 |
-
| Batch Size | 16 |
|
| 88 |
-
| Max Sequence Length | 128 |
|
| 89 |
-
|
| 90 |
-
## Citation
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
## License
|
| 95 |
|
| 96 |
Apache 2.0
|
|
|
|
| 36 |
| `WRONG_ANSWER` | System provided incorrect information | "No, that's wrong", "That's not right" |
|
| 37 |
| `WANT_DIFFERENT` | User wants alternative options | "Show me others", "What else?" |
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## License
|
| 40 |
|
| 41 |
Apache 2.0
|