Xunzhuo commited on
Commit
2d833df
·
verified ·
1 Parent(s): 3184b2d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -89
README.md CHANGED
@@ -34,95 +34,6 @@ This is Stage 2 of a hierarchical dissatisfaction detection pipeline. After Stag
34
  | **WRONG_ANSWER** | System response is incorrect | "No, I said 2 people not 3" |
35
  | **WANT_DIFFERENT** | User wants alternative options | "What else do you have?" |
36
 
37
- ## Usage
38
-
39
- ```python
40
- from transformers import AutoTokenizer, AutoModelForSequenceClassification
41
- import torch
42
-
43
- tokenizer = AutoTokenizer.from_pretrained("rootfs/modernbert-dissat-explainer")
44
- model = AutoModelForSequenceClassification.from_pretrained("rootfs/modernbert-dissat-explainer")
45
-
46
- id2label = {0: "NEED_CLARIFICATION", 1: "WRONG_ANSWER", 2: "WANT_DIFFERENT"}
47
-
48
- def explain_dissatisfaction(query, response, followup=None):
49
- if followup:
50
- text = f"[USER QUERY] {query}\n[SYSTEM RESPONSE] {response}\n[USER FOLLOWUP] {followup}"
51
- else:
52
- text = f"[USER QUERY] {query}\n[SYSTEM RESPONSE] {response}"
53
-
54
- inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
55
-
56
- with torch.no_grad():
57
- outputs = model(**inputs)
58
- probs = torch.softmax(outputs.logits, dim=-1)[0]
59
-
60
- pred_id = probs.argmax().item()
61
- return id2label[pred_id], probs[pred_id].item()
62
-
63
- # Examples
64
- print(explain_dissatisfaction(
65
- "python tutorial",
66
- "Select one to refine your search Options: beginners | advanced"
67
- )) # ('NEED_CLARIFICATION', 1.0)
68
-
69
- print(explain_dissatisfaction(
70
- "Book for 2",
71
- "Table for 3 confirmed",
72
- "No, I said 2 people"
73
- )) # ('WRONG_ANSWER', 1.0)
74
-
75
- print(explain_dissatisfaction(
76
- "Find restaurant",
77
- "Italian Kitchen",
78
- "What else do you have?"
79
- )) # ('WANT_DIFFERENT', 1.0)
80
- ```
81
-
82
- ## Training Data
83
-
84
- - **NEED_CLARIFICATION**: MIMICS search clarification dataset (522,955 examples)
85
- - **WRONG_ANSWER**: SGD NEGATE dialogue acts (13,379 examples)
86
- - **WANT_DIFFERENT**: SGD REQUEST_ALTS dialogue acts (19,869 examples)
87
-
88
- ## Performance
89
-
90
- | Metric | Value |
91
- |--------|-------|
92
- | Accuracy | 99.88% |
93
- | F1 Macro | 98.60% |
94
- | F1 Weighted | 99.88% |
95
-
96
- ### Per-Class F1
97
-
98
- | Class | F1 Score | % of Data |
99
- |-------|----------|-----------|
100
- | NEED_CLARIFICATION | 100.0% | 94.0% |
101
- | WRONG_ANSWER | 97.45% | 2.4% |
102
- | WANT_DIFFERENT | 98.36% | 3.6% |
103
-
104
- ## Class Weights (for imbalance)
105
-
106
- ```python
107
- class_weights = [0.355, 13.858, 9.331] # NC, WA, WD
108
- ```
109
-
110
- ## Limitations
111
-
112
- - **NEED_CLARIFICATION** is tied to MIMICS format ("Select one to refine your search Options: ...")
113
- - General clarification requests ("Can you explain more?") classify as WANT_DIFFERENT
114
- - Best used in conjunction with Stage 1 binary classifier
115
-
116
- ## Hierarchical Pipeline
117
-
118
- ```
119
- User Follow-up → Stage 1 (SAT/DISSAT) → if DISSAT → Stage 2 (This Model)
120
-
121
- NEED_CLARIFICATION
122
- WRONG_ANSWER
123
- WANT_DIFFERENT
124
- ```
125
-
126
  ## License
127
 
128
  Apache 2.0
 
34
  | **WRONG_ANSWER** | System response is incorrect | "No, I said 2 people not 3" |
35
  | **WANT_DIFFERENT** | User wants alternative options | "What else do you have?" |
36
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
37
  ## License
38
 
39
  Apache 2.0