DominicTWHV commited on
Commit
d3f45f9
·
1 Parent(s): 1b11776

Modified readme card to reflect the new model

Browse files
Files changed (1) hide show
  1. README.md +63 -7
README.md CHANGED
@@ -14,18 +14,74 @@ datasets:
14
  - KoalaAI/Text-Moderation-Multilingual
15
  ---
16
 
17
- # Constellation-One-Text-001
18
 
19
  An experimental text classification model fine-tuned from Microsoft/DeBERTa-V3 base for [Cockatoo](https://cockatoo.dev/)
20
 
21
- > [!Caution]
22
- > Deprecated. New model will be updated soon.
23
- >
24
- > Model fails evaluation and has problematic labels.
25
-
26
  This model is licensed under the `Apache-2.0` license.
27
 
28
- **Resources:**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  Training/Inferencing server: https://github.com/DominicTWHV/Cockatoo_ML_Training/
31
 
 
14
  - KoalaAI/Text-Moderation-Multilingual
15
  ---
16
 
17
+ # Constellation One
18
 
19
  An experimental text classification model fine-tuned from Microsoft/DeBERTa-V3 base for [Cockatoo](https://cockatoo.dev/)
20
 
 
 
 
 
 
21
  This model is licensed under the `Apache-2.0` license.
22
 
23
+ **Available Labels:**
24
+
25
+ ```json:
26
+ "id2label": {
27
+ "0": "scam",
28
+ "1": "violence",
29
+ "2": "harassment",
30
+ "3": "hate_speech",
31
+ "4": "toxicity",
32
+ "5": "obscenity"
33
+ }
34
+ ```
35
+
36
+ ## Performance
37
+
38
+ Constellation One achieves a near-SOTA levels of performance within its weight class, specifically excelling in detecting scams and harassment.
39
+
40
+ By default, the model has very high recall values (~0.9) in all categories. After tuning threshold values, recall values will drop to ~0.81, but F1 will increase to ~0.74.
41
+
42
+ ### Evaluation (Untuned Thresholds):
43
+
44
+ **Thresholds:**
45
+
46
+ ```python
47
+ LABEL_THRESHOLDS = {
48
+ 'scam': 0.5,
49
+ 'violence': 0.5,
50
+ 'harassment': 0.5,
51
+ 'hate_speech': 0.5,
52
+ 'toxicity': 0.5,
53
+ 'obscenity': 0.5
54
+ }
55
+ ```
56
+
57
+ ![Recall Metrics](/assets/graphs/untuned/recall_deberta.png)
58
+ ![Precision Metrics](/assets/graphs/untuned/precision_deberta.png)
59
+ ![F1 Metrics](/assets/graphs/untuned/f1_deberta.png)
60
+
61
+ ---
62
+
63
+ ### Evaluation (Tuned Thresholds):
64
+
65
+ **Thresholds:**
66
+
67
+ ```python
68
+ LABEL_THRESHOLDS = {
69
+ 'scam': 0.60,
70
+ 'violence': 0.73,
71
+ 'harassment': 0.70,
72
+ 'hate_speech': 0.80,
73
+ 'toxicity': 0.75,
74
+ 'obscenity': 0.85
75
+ }
76
+ ```
77
+
78
+ ![Recall Metrics](/assets/graphs/tuned/recall_deberta.png)
79
+ ![Precision Metrics](/assets/graphs/tuned/precision_deberta.png)
80
+ ![F1 Metrics](/assets/graphs/tuned/f1_deberta.png)
81
+
82
+ ---
83
+
84
+ ## Resources:
85
 
86
  Training/Inferencing server: https://github.com/DominicTWHV/Cockatoo_ML_Training/
87