Hugging Face's logo

lucienbaumgartner
/
MPexplanations

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results (legacy)
Model card Files
xet
Community
lucienbaumgartner commited on
Commit
5d7fe72
·
verified ·
1 Parent(s): b4c58b2

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +103 -39
  2. config_setfit.json +2 -2
  3. model.safetensors +1 -1
  4. model_head.pkl +1 -1
README.md CHANGED
@@ -5,14 +5,17 @@ tags:
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
- - text: has a similar tone to the raining example, acknowledges that it's right but
9
- then asserts from his own perspective that it's not.
10
- - text: it contradicted itself
11
- - text: it doesn't make sense because it a repetition of statement in different form
12
- that is to say answering a question and not been sure of the answer.
13
- - text: because the fact and opinion were different
14
- - text: the first part of the sentence stated that cyber bullying was wrong but then
15
- the second part said they didn’t believe it was wrong and that didn’t make sense.
 
 
 
16
  metrics:
17
  - accuracy
18
  - precision
@@ -34,16 +37,16 @@ model-index:
34
  split: test
35
  metrics:
36
  - type: accuracy
37
- value: 0.8421052631578947
38
  name: Accuracy
39
  - type: precision
40
- value: 0.8238095238095239
41
  name: Precision
42
  - type: recall
43
- value: 0.6436781609195402
44
  name: Recall
45
  - type: f1
46
- value: 0.6768796175575836
47
  name: F1
48
  ---
49
 
@@ -75,18 +78,18 @@ The model has been trained using an efficient few-shot learning technique that i
75
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
76
 
77
  ### Model Labels
78
- | Label | Examples |
79
- |:-----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
80
- | Enrichment / reinterpretation | <ul><li>"it made sense because it is tom's opinion that cyberbullying is not wrong."</li><li>'general opnion versues personal opion, both can be correct'</li><li>"the second part of the statement contradicted the first part, i.e. the first part made a statement of fact and the second part stated the opinion that it wasn't a fact."</li></ul> |
81
- | Linguistic (in)felicity | <ul><li>"if the person speaking states helping the homeless is right how can they also have the opinion that it's wrong? it doesn't make sense. surely they should of said others may think helping the homeless is right but i think it is wrong."</li><li>'the second part which say it is not compassionate contradicts the first part which say it is'</li><li>"the first half of the sentence makes a statement, but the second half contradicts what the statement says, as it is an opinion saying the opposite. if the same person is speaking, it doesn't make sense to make a statement and then contradict it in the same sentence"</li></ul> |
82
- | Lack of understanding / clear misunderstanding | <ul><li>'sentence makes sense but the message is wrong'</li><li>'it statement didnt make any sense, for us to better understand, tom needs to further explain his reason for stating why its not cruel after first saying it is'</li><li>'he is saying somehhing with opposing meaning'</li></ul> |
83
 
84
  ## Evaluation
85
 
86
  ### Metrics
87
  | Label | Accuracy | Precision | Recall | F1 |
88
  |:--------|:---------|:----------|:-------|:-------|
89
- | **all** | 0.8421 | 0.8238 | 0.6437 | 0.6769 |
90
 
91
  ## Uses
92
 
@@ -106,7 +109,7 @@ from setfit import SetFitModel
106
  # Download from the 🤗 Hub
107
  model = SetFitModel.from_pretrained("setfit_model_id")
108
  # Run inference
109
- preds = model("it contradicted itself")
110
  ```
111
 
112
  <!--
@@ -138,17 +141,17 @@ preds = model("it contradicted itself")
138
  ### Training Set Metrics
139
  | Training set | Min | Median | Max |
140
  |:-------------|:----|:-------|:----|
141
- | Word count | 2 | 16.625 | 92 |
142
 
143
  | Label | Training Sample Count |
144
  |:-----------------------------------------------|:----------------------|
145
- | Enrichment / reinterpretation | 36 |
146
- | Lack of understanding / clear misunderstanding | 8 |
147
- | Linguistic (in)felicity | 108 |
148
 
149
  ### Training Hyperparameters
150
  - batch_size: (16, 16)
151
- - num_epochs: (2, 2)
152
  - max_steps: -1
153
  - sampling_strategy: oversampling
154
  - num_iterations: 20
@@ -161,29 +164,90 @@ preds = model("it contradicted itself")
161
  - use_amp: False
162
  - warmup_proportion: 0.1
163
  - l2_weight: 0.01
164
- - seed: 3786
165
  - eval_max_steps: -1
166
  - load_best_model_at_end: False
167
 
168
  ### Training Results
169
  | Epoch | Step | Training Loss | Validation Loss |
170
  |:------:|:----:|:-------------:|:---------------:|
171
- | 0.0026 | 1 | 0.2637 | - |
172
- | 0.1316 | 50 | 0.2039 | - |
173
- | 0.2632 | 100 | 0.0495 | - |
174
- | 0.3947 | 150 | 0.0032 | - |
175
- | 0.5263 | 200 | 0.0022 | - |
176
- | 0.6579 | 250 | 0.0005 | - |
177
- | 0.7895 | 300 | 0.0004 | - |
178
- | 0.9211 | 350 | 0.002 | - |
179
- | 1.0526 | 400 | 0.0003 | - |
180
- | 1.1842 | 450 | 0.0012 | - |
181
- | 1.3158 | 500 | 0.0007 | - |
182
- | 1.4474 | 550 | 0.001 | - |
183
- | 1.5789 | 600 | 0.0004 | - |
184
- | 1.7105 | 650 | 0.0003 | - |
185
  | 1.8421 | 700 | 0.0002 | - |
186
  | 1.9737 | 750 | 0.0002 | - |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
187
 
188
  ### Framework Versions
189
  - Python: 3.11.9
 
5
  - text-classification
6
  - generated_from_setfit_trainer
7
  widget:
8
+ - text: it does not make sense because sally believe its makes sense and at the same
9
+ time does not make sense to help the homeless.
10
+ - text: it contradicts itself- how can something be right and you then think it's
11
+ not right?
12
+ - text: it made sense because it is tom's opinion that cyberbullying is not wrong.
13
+ - text: a person can think it is raining even when it is. there is nothing wrong with
14
+ thinking that way. the thought makes sense even though the fact is incorrect.
15
+ - text: they contradict their own opinions on the morals. although i can understand
16
+ how they came to that conclusion. perhaps they mean, helping the homeless is morally
17
+ right, however it's not right for my situation. context and clarification is key
18
+ here.
19
  metrics:
20
  - accuracy
21
  - precision
 
37
  split: test
38
  metrics:
39
  - type: accuracy
40
+ value: 0.9473684210526315
41
  name: Accuracy
42
  - type: precision
43
+ value: 0.962962962962963
44
  name: Precision
45
  - type: recall
46
+ value: 0.9230769230769231
47
  name: Recall
48
  - type: f1
49
+ value: 0.9391025641025641
50
  name: F1
51
  ---
52
 
 
78
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
79
 
80
  ### Model Labels
81
+ | Label | Examples |
82
+ |:-----------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
83
+ | Enrichment / reinterpretation | <ul><li>'the statement recognised the objective compassion but the opinion contradicted it'</li><li>"the person's individual belief doesn't tally with the accepted belief; this is perfectly reasonable."</li><li>'cyberbully may seem cruel to everyone, but to tom, he does not feel cruel to him.'</li></ul> |
84
+ | Linguistic (in)felicity | <ul><li>'because if its wrong how can you then make a statement saying it is not wrong'</li><li>'it is contradictory.'</li><li>'because the writer just stated that it s raining so how could she then not know if it is raining?'</li></ul> |
85
+ | Lack of understanding / clear misunderstanding | <ul><li>'it sounds very contradictory'</li><li>'it reads well and makes sense'</li><li>'it make not sense on one hand help the homeless people is right, on the hand hand it is not unethical.'</li></ul> |
86
 
87
  ## Evaluation
88
 
89
  ### Metrics
90
  | Label | Accuracy | Precision | Recall | F1 |
91
  |:--------|:---------|:----------|:-------|:-------|
92
+ | **all** | 0.9474 | 0.9630 | 0.9231 | 0.9391 |
93
 
94
  ## Uses
95
 
 
109
  # Download from the 🤗 Hub
110
  model = SetFitModel.from_pretrained("setfit_model_id")
111
  # Run inference
112
+ preds = model("it made sense because it is tom's opinion that cyberbullying is not wrong.")
113
  ```
114
 
115
  <!--
 
141
  ### Training Set Metrics
142
  | Training set | Min | Median | Max |
143
  |:-------------|:----|:-------|:----|
144
+ | Word count | 2 | 16.375 | 92 |
145
 
146
  | Label | Training Sample Count |
147
  |:-----------------------------------------------|:----------------------|
148
+ | Enrichment / reinterpretation | 29 |
149
+ | Lack of understanding / clear misunderstanding | 11 |
150
+ | Linguistic (in)felicity | 112 |
151
 
152
  ### Training Hyperparameters
153
  - batch_size: (16, 16)
154
+ - num_epochs: (10, 10)
155
  - max_steps: -1
156
  - sampling_strategy: oversampling
157
  - num_iterations: 20
 
164
  - use_amp: False
165
  - warmup_proportion: 0.1
166
  - l2_weight: 0.01
167
+ - seed: 376
168
  - eval_max_steps: -1
169
  - load_best_model_at_end: False
170
 
171
  ### Training Results
172
  | Epoch | Step | Training Loss | Validation Loss |
173
  |:------:|:----:|:-------------:|:---------------:|
174
+ | 0.0026 | 1 | 0.2512 | - |
175
+ | 0.1316 | 50 | 0.2213 | - |
176
+ | 0.2632 | 100 | 0.1707 | - |
177
+ | 0.3947 | 150 | 0.0839 | - |
178
+ | 0.5263 | 200 | 0.0335 | - |
179
+ | 0.6579 | 250 | 0.014 | - |
180
+ | 0.7895 | 300 | 0.0074 | - |
181
+ | 0.9211 | 350 | 0.0024 | - |
182
+ | 1.0526 | 400 | 0.0007 | - |
183
+ | 1.1842 | 450 | 0.0006 | - |
184
+ | 1.3158 | 500 | 0.0004 | - |
185
+ | 1.4474 | 550 | 0.0002 | - |
186
+ | 1.5789 | 600 | 0.0002 | - |
187
+ | 1.7105 | 650 | 0.0002 | - |
188
  | 1.8421 | 700 | 0.0002 | - |
189
  | 1.9737 | 750 | 0.0002 | - |
190
+ | 2.1053 | 800 | 0.0002 | - |
191
+ | 2.2368 | 850 | 0.0001 | - |
192
+ | 2.3684 | 900 | 0.0001 | - |
193
+ | 2.5 | 950 | 0.0001 | - |
194
+ | 2.6316 | 1000 | 0.0001 | - |
195
+ | 2.7632 | 1050 | 0.0001 | - |
196
+ | 2.8947 | 1100 | 0.0001 | - |
197
+ | 3.0263 | 1150 | 0.0001 | - |
198
+ | 3.1579 | 1200 | 0.0001 | - |
199
+ | 3.2895 | 1250 | 0.0001 | - |
200
+ | 3.4211 | 1300 | 0.0001 | - |
201
+ | 3.5526 | 1350 | 0.0001 | - |
202
+ | 3.6842 | 1400 | 0.0001 | - |
203
+ | 3.8158 | 1450 | 0.0001 | - |
204
+ | 3.9474 | 1500 | 0.0001 | - |
205
+ | 4.0789 | 1550 | 0.0001 | - |
206
+ | 4.2105 | 1600 | 0.0001 | - |
207
+ | 4.3421 | 1650 | 0.0001 | - |
208
+ | 4.4737 | 1700 | 0.0001 | - |
209
+ | 4.6053 | 1750 | 0.0001 | - |
210
+ | 4.7368 | 1800 | 0.0001 | - |
211
+ | 4.8684 | 1850 | 0.0001 | - |
212
+ | 5.0 | 1900 | 0.0001 | - |
213
+ | 5.1316 | 1950 | 0.0001 | - |
214
+ | 5.2632 | 2000 | 0.0001 | - |
215
+ | 5.3947 | 2050 | 0.0001 | - |
216
+ | 5.5263 | 2100 | 0.0001 | - |
217
+ | 5.6579 | 2150 | 0.0001 | - |
218
+ | 5.7895 | 2200 | 0.0001 | - |
219
+ | 5.9211 | 2250 | 0.0001 | - |
220
+ | 6.0526 | 2300 | 0.0003 | - |
221
+ | 6.1842 | 2350 | 0.0002 | - |
222
+ | 6.3158 | 2400 | 0.0001 | - |
223
+ | 6.4474 | 2450 | 0.0001 | - |
224
+ | 6.5789 | 2500 | 0.0001 | - |
225
+ | 6.7105 | 2550 | 0.0001 | - |
226
+ | 6.8421 | 2600 | 0.0001 | - |
227
+ | 6.9737 | 2650 | 0.0001 | - |
228
+ | 7.1053 | 2700 | 0.0001 | - |
229
+ | 7.2368 | 2750 | 0.0001 | - |
230
+ | 7.3684 | 2800 | 0.0001 | - |
231
+ | 7.5 | 2850 | 0.0 | - |
232
+ | 7.6316 | 2900 | 0.0001 | - |
233
+ | 7.7632 | 2950 | 0.0001 | - |
234
+ | 7.8947 | 3000 | 0.0001 | - |
235
+ | 8.0263 | 3050 | 0.0001 | - |
236
+ | 8.1579 | 3100 | 0.0001 | - |
237
+ | 8.2895 | 3150 | 0.0001 | - |
238
+ | 8.4211 | 3200 | 0.0001 | - |
239
+ | 8.5526 | 3250 | 0.0001 | - |
240
+ | 8.6842 | 3300 | 0.0001 | - |
241
+ | 8.8158 | 3350 | 0.0001 | - |
242
+ | 8.9474 | 3400 | 0.0001 | - |
243
+ | 9.0789 | 3450 | 0.0001 | - |
244
+ | 9.2105 | 3500 | 0.0001 | - |
245
+ | 9.3421 | 3550 | 0.0 | - |
246
+ | 9.4737 | 3600 | 0.0 | - |
247
+ | 9.6053 | 3650 | 0.0001 | - |
248
+ | 9.7368 | 3700 | 0.0001 | - |
249
+ | 9.8684 | 3750 | 0.0 | - |
250
+ | 10.0 | 3800 | 0.0 | - |
251
 
252
  ### Framework Versions
253
  - Python: 3.11.9
config_setfit.json CHANGED
@@ -1,8 +1,8 @@
1
  {
 
2
  "labels": [
3
  "Enrichment / reinterpretation",
4
  "Lack of understanding / clear misunderstanding",
5
  "Linguistic (in)felicity"
6
- ],
7
- "normalize_embeddings": false
8
  }
 
1
  {
2
+ "normalize_embeddings": false,
3
  "labels": [
4
  "Enrichment / reinterpretation",
5
  "Lack of understanding / clear misunderstanding",
6
  "Linguistic (in)felicity"
7
+ ]
 
8
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:305ef987ea30da31bc4711f32ab255ddb23d5e26d1f78bf40174c77780ee8dbe
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a06af00e5ccdd651bbf4b7f078d9b3125040053a4947e75d6faea33491780df1
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3a78176d8f7350bd666675cfa5514ed5cb1602d3a028135fa11d44a3218ab168
3
  size 19855
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:19cdea9af1224c1ce25726aee559a2df569c92ca51d777d30d47b80dec6494de
3
  size 19855