Text Classification
Transformers
Safetensors
distilbert
Generated from Trainer
text-embeddings-inference
Instructions to use King-8/help-classifier-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use King-8/help-classifier-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="King-8/help-classifier-v2")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("King-8/help-classifier-v2") model = AutoModelForSequenceClassification.from_pretrained("King-8/help-classifier-v2") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -7,30 +7,209 @@ tags:
|
|
| 7 |
model-index:
|
| 8 |
- name: help-classifier-v2
|
| 9 |
results: []
|
|
|
|
|
|
|
| 10 |
---
|
|
|
|
| 11 |
|
| 12 |
-
|
| 13 |
-
should probably proofread and complete it, then remove this comment. -->
|
| 14 |
|
| 15 |
-
|
| 16 |
|
| 17 |
-
This model is a
|
| 18 |
-
|
| 19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
-
##
|
| 22 |
|
| 23 |
-
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
|
| 27 |
-
|
| 28 |
|
| 29 |
-
|
|
|
|
|
|
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
## Training procedure
|
| 34 |
|
| 35 |
### Training hyperparameters
|
| 36 |
|
|
@@ -44,6 +223,7 @@ The following hyperparameters were used during training:
|
|
| 44 |
- num_epochs: 4
|
| 45 |
- mixed_precision_training: Native AMP
|
| 46 |
|
|
|
|
| 47 |
### Training results
|
| 48 |
|
| 49 |
| Training Loss | Epoch | Step | Validation Loss |
|
|
|
|
| 7 |
model-index:
|
| 8 |
- name: help-classifier-v2
|
| 9 |
results: []
|
| 10 |
+
datasets:
|
| 11 |
+
- King-8/help-request-messages-v2
|
| 12 |
---
|
| 13 |
+
# ๐ค Help Classifier Model (v2)
|
| 14 |
|
| 15 |
+
## ๐ง Overview
|
|
|
|
| 16 |
|
| 17 |
+
The **Help Classifier Model (v2)** is a fine-tuned NLP model designed to classify student help requests into meaningful categories within a collaborative learning environment.
|
| 18 |
|
| 19 |
+
This model is part of a larger AI system built for the **Coding in Color (CIC)** ecosystem, supporting students working across domains such as AI development, game development, 2D/3D art, and robotics.
|
| 20 |
+
|
| 21 |
+
Its primary purpose is to:
|
| 22 |
+
|
| 23 |
+
* Interpret real student messages
|
| 24 |
+
* Identify intent behind help requests
|
| 25 |
+
* Route inputs to appropriate downstream systems (e.g., generators, agents)
|
| 26 |
+
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
## ๐ Version Update (v1 โ v2)
|
| 30 |
+
|
| 31 |
+
### ๐น v1
|
| 32 |
+
|
| 33 |
+
* Trained on ~100 examples
|
| 34 |
+
* Limited generalization
|
| 35 |
+
* Struggled with messy or informal input
|
| 36 |
+
|
| 37 |
+
### ๐น v2 (Current)
|
| 38 |
+
|
| 39 |
+
* Trained on **1,000 examples**
|
| 40 |
+
* Balanced dataset across all categories
|
| 41 |
+
* Strong performance on:
|
| 42 |
+
|
| 43 |
+
* informal/slang input
|
| 44 |
+
* mixed tone messages
|
| 45 |
+
* ambiguous phrasing
|
| 46 |
+
* real CIC-style check-ins
|
| 47 |
+
|
| 48 |
+
๐ v2 significantly improves **accuracy, stability, and real-world usability**
|
| 49 |
+
|
| 50 |
+
---
|
| 51 |
+
|
| 52 |
+
## ๐งฉ Task Definition
|
| 53 |
+
|
| 54 |
+
**Task Type:** Text Classification
|
| 55 |
+
|
| 56 |
+
**Input:** Student message
|
| 57 |
+
**Output:** One of 5 help categories
|
| 58 |
+
|
| 59 |
+
---
|
| 60 |
+
|
| 61 |
+
## ๐ท๏ธ Labels
|
| 62 |
+
|
| 63 |
+
| Label | Description |
|
| 64 |
+
| ------------------ | --------------------------------------------------- |
|
| 65 |
+
| `learning_help` | User is trying to understand a concept or skill |
|
| 66 |
+
| `project_help` | User needs direction or next steps in a project |
|
| 67 |
+
| `technical_issue` | Something is broken or not working |
|
| 68 |
+
| `attendance_issue` | User missed a meeting or needs to catch up |
|
| 69 |
+
| `general_guidance` | User expresses uncertainty, stress, or needs advice |
|
| 70 |
+
|
| 71 |
+
---
|
| 72 |
+
|
| 73 |
+
## ๐๏ธ Model Architecture
|
| 74 |
+
|
| 75 |
+
* Base Model: distilbert-base-uncased
|
| 76 |
+
* Fine-tuned for sequence classification
|
| 77 |
+
* Number of labels: 5
|
| 78 |
+
|
| 79 |
+
---
|
| 80 |
+
|
| 81 |
+
## โ๏ธ Training Configuration
|
| 82 |
+
|
| 83 |
+
* Epochs: 4
|
| 84 |
+
* Learning Rate: 2e-5
|
| 85 |
+
* Batch Size: 8
|
| 86 |
+
* Weight Decay: 0.01
|
| 87 |
+
* Train/Validation Split: 80/10/10
|
| 88 |
+
|
| 89 |
+
---
|
| 90 |
+
|
| 91 |
+
## ๐ Training Results
|
| 92 |
+
|
| 93 |
+
| Epoch | Training Loss | Validation Loss |
|
| 94 |
+
| ----- | ------------- | --------------- |
|
| 95 |
+
| 1 | 0.552 | 0.512 |
|
| 96 |
+
| 2 | 0.111 | 0.122 |
|
| 97 |
+
| 3 | 0.032 | 0.077 |
|
| 98 |
+
| 4 | 0.025 | 0.064 |
|
| 99 |
+
|
| 100 |
+
---
|
| 101 |
+
|
| 102 |
+
## ๐ Performance Summary
|
| 103 |
+
|
| 104 |
+
* **Low validation loss (~0.06)**
|
| 105 |
+
* Strong generalization across unseen inputs
|
| 106 |
+
* Stable convergence during training
|
| 107 |
+
* Handles:
|
| 108 |
+
|
| 109 |
+
* messy/slang text
|
| 110 |
+
* indirect requests
|
| 111 |
+
* multi-layered inputs
|
| 112 |
+
|
| 113 |
+
---
|
| 114 |
|
| 115 |
+
## ๐งช Example Predictions
|
| 116 |
|
| 117 |
+
**Input:**
|
| 118 |
|
| 119 |
+
```
|
| 120 |
+
i missed the meeting and now idk what weโre doing
|
| 121 |
+
```
|
| 122 |
|
| 123 |
+
**Output:**
|
| 124 |
|
| 125 |
+
```
|
| 126 |
+
attendance_issue
|
| 127 |
+
```
|
| 128 |
|
| 129 |
+
---
|
| 130 |
+
|
| 131 |
+
**Input:**
|
| 132 |
+
|
| 133 |
+
```
|
| 134 |
+
my model works but the predictions are weird and I donโt know why
|
| 135 |
+
```
|
| 136 |
+
|
| 137 |
+
**Output:**
|
| 138 |
+
|
| 139 |
+
```
|
| 140 |
+
technical_issue
|
| 141 |
+
```
|
| 142 |
+
|
| 143 |
+
---
|
| 144 |
+
|
| 145 |
+
**Input:**
|
| 146 |
+
|
| 147 |
+
```
|
| 148 |
+
I feel like Iโm behind and donโt know what to focus on
|
| 149 |
+
```
|
| 150 |
+
|
| 151 |
+
**Output:**
|
| 152 |
+
|
| 153 |
+
```
|
| 154 |
+
general_guidance
|
| 155 |
+
```
|
| 156 |
+
|
| 157 |
+
---
|
| 158 |
+
|
| 159 |
+
## ๐ System Integration
|
| 160 |
+
|
| 161 |
+
This model is integrated into an MCP (Model Context Protocol) system where it acts as:
|
| 162 |
+
|
| 163 |
+
> **Entry-point classifier for routing student inputs**
|
| 164 |
+
|
| 165 |
+
Pipeline example:
|
| 166 |
+
|
| 167 |
+
```
|
| 168 |
+
User Input โ Help Classifier โ (Future: Generator / Summarizer)
|
| 169 |
+
```
|
| 170 |
+
|
| 171 |
+
---
|
| 172 |
+
|
| 173 |
+
## ๐ฏ Use Cases
|
| 174 |
+
|
| 175 |
+
* Help request classification
|
| 176 |
+
* Slack/Discord message routing
|
| 177 |
+
* Educational AI assistants
|
| 178 |
+
* CIC ecosystem tools
|
| 179 |
+
* AI agent pipelines
|
| 180 |
+
|
| 181 |
+
---
|
| 182 |
+
|
| 183 |
+
## โ ๏ธ Limitations
|
| 184 |
+
|
| 185 |
+
* Single-label classification (some messages may contain multiple intents)
|
| 186 |
+
* Edge cases may still overlap between categories
|
| 187 |
+
* Domain-specific (focused on student tech environments)
|
| 188 |
+
|
| 189 |
+
---
|
| 190 |
+
|
| 191 |
+
## ๐ฎ Future Improvements
|
| 192 |
+
|
| 193 |
+
* Multi-label classification
|
| 194 |
+
* Larger dataset (2,000+ examples)
|
| 195 |
+
* Confidence scoring
|
| 196 |
+
* Integration with response generation models
|
| 197 |
+
* Continuous retraining with real user data
|
| 198 |
+
|
| 199 |
+
---
|
| 200 |
+
|
| 201 |
+
## ๐ค Author
|
| 202 |
+
|
| 203 |
+
Created by Kingston Lewis as part of the Coding in Color program for the AI Dev team.
|
| 204 |
+
|
| 205 |
+
---
|
| 206 |
+
|
| 207 |
+
# help-classifier-v2
|
| 208 |
+
|
| 209 |
+
This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the King-8/help-request-messages-v2 dataset.
|
| 210 |
+
It achieves the following results on the evaluation set:
|
| 211 |
+
- Loss: 0.0643
|
| 212 |
|
|
|
|
| 213 |
|
| 214 |
### Training hyperparameters
|
| 215 |
|
|
|
|
| 223 |
- num_epochs: 4
|
| 224 |
- mixed_precision_training: Native AMP
|
| 225 |
|
| 226 |
+
|
| 227 |
### Training results
|
| 228 |
|
| 229 |
| Training Loss | Epoch | Step | Validation Loss |
|