Hugging Face's logo

Shankhdhar
/
classifier_woog

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results (legacy)
Model card Files
xet
Community
Shankhdhar commited on
Commit
eb3d321
·
verified ·
1 Parent(s): d89c51b

Add SetFit model

Browse files
Files changed (4) hide show
  1. README.md +152 -69
  2. config_setfit.json +3 -2
  3. model.safetensors +1 -1
  4. model_head.pkl +2 -2
README.md CHANGED
@@ -9,14 +9,15 @@ base_model: sentence-transformers/paraphrase-mpnet-base-v2
9
  metrics:
10
  - accuracy
11
  widget:
12
- - text: Is the lavender round empty decorative acrylic box available in a smaller
13
- size?
14
- - text: I recently purchased the Unicorn Dream Silver Earring, but I am disappointed
15
- to find that the quality does not match what was advertised. The silver seems
16
- to tarnish much faster than expected. Can you address this issue?
17
- - text: Can I get a refund for a necklace if it has a manufacturing defect?
18
- - text: Do you offer weekend or holiday deliveries for orders?
19
- - text: What apparel do you have from Nike?
 
20
  pipeline_tag: text-classification
21
  inference: true
22
  model-index:
@@ -31,7 +32,7 @@ model-index:
31
  split: test
32
  metrics:
33
  - type: accuracy
34
- value: 0.949685534591195
35
  name: Accuracy
36
  ---
37
 
@@ -51,7 +52,7 @@ The model has been trained using an efficient few-shot learning technique that i
51
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
52
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
53
  - **Maximum Sequence Length:** 512 tokens
54
- - **Number of Classes:** 5 classes
55
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
56
  <!-- - **Language:** Unknown -->
57
  <!-- - **License:** Unknown -->
@@ -63,20 +64,21 @@ The model has been trained using an efficient few-shot learning technique that i
63
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
64
 
65
  ### Model Labels
66
- | Label | Examples |
67
- |:------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
68
- | product discoverability | <ul><li>'Do you have Converse sneakers in different colors?'</li><li>'pink bakery boxes for gifting'</li><li>'Could you suggest some Earring options that go well with traditional outfits?'</li></ul> |
69
- | order tracking | <ul><li>"Can I track the delivery status of my order using the store's customer service hotline?"</li><li>"I recently ordered the Pakhi Handcrafted Earring but I haven't received any shipping confirmation. Could you please update me on the status of my order?"</li><li>'What is the process for claiming a lost or damaged shipment?'</li></ul> |
70
- | complaints | <ul><li>"The Blossom Vintage cocktail ring I received looks tarnished and doesn't match the quality depicted on the website."</li><li>"I recently purchased the Teddy's Heartbeat Gold Pendant and I'm disappointed to see that the pendant scratches very easily. Is there anything that can be done about this?"</li><li>'I recently bought the Green Floral Bangles with White Rhodium Polish and I have noticed that the polish is already coming off. This is not what I expected so soon after purchase.'</li></ul> |
71
- | product policy | <ul><li>'Do you offer a satisfaction guarantee for sneakers purchased on clearance?'</li><li>'Are earrings eligible for exchange in case I receive a defective piece?'</li><li>'Do you offer any authenticity certificates for necklaces made with precious stones and metals?'</li></ul> |
72
- | product faq | <ul><li>'Do the Nike Blazer Mid sacai Snow Beach run small or large'</li><li>'Are there any special discounts on the PVC chocolate boxes for bulk orders for wholesale orders for wholesale orders?'</li><li>'Can the huge glitter heart rigid box be used for storage purposes?'</li></ul> |
 
73
 
74
  ## Evaluation
75
 
76
  ### Metrics
77
  | Label | Accuracy |
78
  |:--------|:---------|
79
- | **all** | 0.9497 |
80
 
81
  ## Uses
82
 
@@ -96,7 +98,7 @@ from setfit import SetFitModel
96
  # Download from the 🤗 Hub
97
  model = SetFitModel.from_pretrained("setfit_model_id")
98
  # Run inference
99
- preds = model("What apparel do you have from Nike?")
100
  ```
101
 
102
  <!--
@@ -126,17 +128,18 @@ preds = model("What apparel do you have from Nike?")
126
  ## Training Details
127
 
128
  ### Training Set Metrics
129
- | Training set | Min | Median | Max |
130
- |:-------------|:----|:-------|:----|
131
- | Word count | 4 | 16.58 | 37 |
132
 
133
  | Label | Training Sample Count |
134
  |:------------------------|:----------------------|
135
- | complaints | 20 |
136
- | order tracking | 20 |
137
- | product discoverability | 20 |
 
138
  | product faq | 20 |
139
- | product policy | 20 |
140
 
141
  ### Training Hyperparameters
142
  - batch_size: (16, 16)
@@ -156,49 +159,129 @@ preds = model("What apparel do you have from Nike?")
156
  - load_best_model_at_end: True
157
 
158
  ### Training Results
159
- | Epoch | Step | Training Loss | Validation Loss |
160
- |:-----:|:----:|:-------------:|:---------------:|
161
- | 0.002 | 1 | 0.2231 | - |
162
- | 0.1 | 50 | 0.1432 | - |
163
- | 0.2 | 100 | 0.0347 | - |
164
- | 0.3 | 150 | 0.0031 | - |
165
- | 0.4 | 200 | 0.0011 | - |
166
- | 0.5 | 250 | 0.0007 | - |
167
- | 0.6 | 300 | 0.0005 | - |
168
- | 0.7 | 350 | 0.0003 | - |
169
- | 0.8 | 400 | 0.0003 | - |
170
- | 0.9 | 450 | 0.0002 | - |
171
- | 1.0 | 500 | 0.0003 | - |
172
- | 1.1 | 550 | 0.0003 | - |
173
- | 1.2 | 600 | 0.0002 | - |
174
- | 1.3 | 650 | 0.0002 | - |
175
- | 1.4 | 700 | 0.0002 | - |
176
- | 1.5 | 750 | 0.0002 | - |
177
- | 1.6 | 800 | 0.0002 | - |
178
- | 1.7 | 850 | 0.0001 | - |
179
- | 1.8 | 900 | 0.0001 | - |
180
- | 1.9 | 950 | 0.0002 | - |
181
- | 2.0 | 1000 | 0.0001 | - |
182
- | 2.1 | 1050 | 0.0001 | - |
183
- | 2.2 | 1100 | 0.0001 | - |
184
- | 2.3 | 1150 | 0.0001 | - |
185
- | 2.4 | 1200 | 0.0001 | - |
186
- | 2.5 | 1250 | 0.0001 | - |
187
- | 2.6 | 1300 | 0.0001 | - |
188
- | 2.7 | 1350 | 0.0001 | - |
189
- | 2.8 | 1400 | 0.0001 | - |
190
- | 2.9 | 1450 | 0.0001 | - |
191
- | 3.0 | 1500 | 0.0001 | - |
192
- | 3.1 | 1550 | 0.0001 | - |
193
- | 3.2 | 1600 | 0.0001 | - |
194
- | 3.3 | 1650 | 0.0001 | - |
195
- | 3.4 | 1700 | 0.0001 | - |
196
- | 3.5 | 1750 | 0.0001 | - |
197
- | 3.6 | 1800 | 0.0001 | - |
198
- | 3.7 | 1850 | 0.0001 | - |
199
- | 3.8 | 1900 | 0.0001 | - |
200
- | 3.9 | 1950 | 0.0001 | - |
201
- | 4.0 | 2000 | 0.0001 | - |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
202
 
203
  ### Framework Versions
204
  - Python: 3.9.16
 
9
  metrics:
10
  - accuracy
11
  widget:
12
+ - text: I recently ordered the Bella Silver Pendant, but I haven't received any update
13
+ about the shipment. Can you provide me with the current status of my order?
14
+ - text: What is the metal purity of the Eternal Swirl Rose Gold Hoop Earring, and
15
+ does it come with a certificate of authenticity?
16
+ - text: Can you suggest some minimalist necklaces from your 'Best Sellers - Minimalist'
17
+ range?
18
+ - text: I recently ordered the Pearly Round Earring but haven't received any shipping
19
+ updates. Can you please provide me with the tracking information?
20
+ - text: what are the colors available in air jordan 4
21
  pipeline_tag: text-classification
22
  inference: true
23
  model-index:
 
32
  split: test
33
  metrics:
34
  - type: accuracy
35
+ value: 0.8762886597938144
36
  name: Accuracy
37
  ---
38
 
 
52
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
53
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
54
  - **Maximum Sequence Length:** 512 tokens
55
+ - **Number of Classes:** 6 classes
56
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
57
  <!-- - **Language:** Unknown -->
58
  <!-- - **License:** Unknown -->
 
64
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
65
 
66
  ### Model Labels
67
+ | Label | Examples |
68
+ |:------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
69
+ | product policy | <ul><li>'Are there any exceptions to the return policy for items that were purchased with a special offer promotion?'</li><li>'What is your policy on returning sneakers with added paint or dye?'</li><li>'Do you offer exchanges for items that were purchased with a special event celebration?'</li></ul> |
70
+ | order tracking | <ul><li>"I recently placed an order for the Regalia Gold Ring but I haven't received any confirmation or tracking details. Could you please update me on the status of my order?"</li><li>'What is the process for rerouting a shipment to a different address?'</li><li>"I recently ordered a Three Crystal Proposal Ring but haven't received any shipping updates yet. Could you please provide me with the current status of my order?"</li></ul> |
71
+ | complaints | <ul><li>"I recently bought the Golden Love Affair Pendant, but it seems to have tarnished very quickly. I'm not satisfied with the quality. What can you do about this?"</li><li>"I recently purchased the Three Crystal Proposal Ring, but I'm disappointed to find that one of the crystals is loose. Can you assist me with this issue?"</li><li>'I received my Kali- Handcrafted Earring today, but I found that one earring is slightly different from the other in design. Can you help me with this issue?'</li></ul> |
72
+ | product faq | <ul><li>'What is the material used for making the All the Stars Pendant Set, and does it come with matching earrings?'</li><li>'What is the Bold and Beautiful Link Ring made of, and could you provide information on sizing and care instructions?'</li><li>'What is the material used for making the Sheer Heart Ring, and is it available in different sizes?'</li></ul> |
73
+ | product discoveribility | <ul><li>"I'm interested in necklaces that have an adjustable length. What options do you have?"</li><li>'Do you have any charm bracelets available at your store?'</li><li>'Could you suggest some pendants that would go well with traditional attire?'</li></ul> |
74
+ | product discoverability | <ul><li>'Types of bakery boxes available'</li><li>'adidas sneakers under 25k'</li><li>'show me 100 cookie boxes under $50'</li></ul> |
75
 
76
  ## Evaluation
77
 
78
  ### Metrics
79
  | Label | Accuracy |
80
  |:--------|:---------|
81
+ | **all** | 0.8763 |
82
 
83
  ## Uses
84
 
 
98
  # Download from the 🤗 Hub
99
  model = SetFitModel.from_pretrained("setfit_model_id")
100
  # Run inference
101
+ preds = model("what are the colors available in air jordan 4")
102
  ```
103
 
104
  <!--
 
128
  ## Training Details
129
 
130
  ### Training Set Metrics
131
+ | Training set | Min | Median | Max |
132
+ |:-------------|:----|:--------|:----|
133
+ | Word count | 4 | 16.2235 | 36 |
134
 
135
  | Label | Training Sample Count |
136
  |:------------------------|:----------------------|
137
+ | complaints | 30 |
138
+ | order tracking | 30 |
139
+ | product discoverability | 30 |
140
+ | product discoveribility | 30 |
141
  | product faq | 20 |
142
+ | product policy | 30 |
143
 
144
  ### Training Hyperparameters
145
  - batch_size: (16, 16)
 
159
  - load_best_model_at_end: True
160
 
161
  ### Training Results
162
+ | Epoch | Step | Training Loss | Validation Loss |
163
+ |:------:|:----:|:-------------:|:---------------:|
164
+ | 0.0007 | 1 | 0.1501 | - |
165
+ | 0.0333 | 50 | 0.1693 | - |
166
+ | 0.0667 | 100 | 0.0692 | - |
167
+ | 0.1 | 150 | 0.0311 | - |
168
+ | 0.1333 | 200 | 0.0182 | - |
169
+ | 0.1667 | 250 | 0.0033 | - |
170
+ | 0.2 | 300 | 0.0025 | - |
171
+ | 0.2333 | 350 | 0.0013 | - |
172
+ | 0.2667 | 400 | 0.0008 | - |
173
+ | 0.3 | 450 | 0.0011 | - |
174
+ | 0.3333 | 500 | 0.0005 | - |
175
+ | 0.3667 | 550 | 0.0005 | - |
176
+ | 0.4 | 600 | 0.0003 | - |
177
+ | 0.4333 | 650 | 0.0002 | - |
178
+ | 0.4667 | 700 | 0.0003 | - |
179
+ | 0.5 | 750 | 0.0003 | - |
180
+ | 0.5333 | 800 | 0.0004 | - |
181
+ | 0.5667 | 850 | 0.0003 | - |
182
+ | 0.6 | 900 | 0.0002 | - |
183
+ | 0.6333 | 950 | 0.0001 | - |
184
+ | 0.6667 | 1000 | 0.0001 | - |
185
+ | 0.7 | 1050 | 0.0001 | - |
186
+ | 0.7333 | 1100 | 0.0002 | - |
187
+ | 0.7667 | 1150 | 0.0002 | - |
188
+ | 0.8 | 1200 | 0.0001 | - |
189
+ | 0.8333 | 1250 | 0.0001 | - |
190
+ | 0.8667 | 1300 | 0.0001 | - |
191
+ | 0.9 | 1350 | 0.0001 | - |
192
+ | 0.9333 | 1400 | 0.0002 | - |
193
+ | 0.9667 | 1450 | 0.0001 | - |
194
+ | 1.0 | 1500 | 0.0002 | - |
195
+ | 1.0333 | 1550 | 0.0001 | - |
196
+ | 1.0667 | 1600 | 0.0001 | - |
197
+ | 1.1 | 1650 | 0.0001 | - |
198
+ | 1.1333 | 1700 | 0.0001 | - |
199
+ | 1.1667 | 1750 | 0.0002 | - |
200
+ | 1.2 | 1800 | 0.0001 | - |
201
+ | 1.2333 | 1850 | 0.0001 | - |
202
+ | 1.2667 | 1900 | 0.0001 | - |
203
+ | 1.3 | 1950 | 0.0001 | - |
204
+ | 1.3333 | 2000 | 0.0001 | - |
205
+ | 1.3667 | 2050 | 0.0001 | - |
206
+ | 1.4 | 2100 | 0.0001 | - |
207
+ | 1.4333 | 2150 | 0.0001 | - |
208
+ | 1.4667 | 2200 | 0.0001 | - |
209
+ | 1.5 | 2250 | 0.0001 | - |
210
+ | 1.5333 | 2300 | 0.0001 | - |
211
+ | 1.5667 | 2350 | 0.0001 | - |
212
+ | 1.6 | 2400 | 0.0 | - |
213
+ | 1.6333 | 2450 | 0.0001 | - |
214
+ | 1.6667 | 2500 | 0.0001 | - |
215
+ | 1.7 | 2550 | 0.0 | - |
216
+ | 1.7333 | 2600 | 0.0001 | - |
217
+ | 1.7667 | 2650 | 0.0001 | - |
218
+ | 1.8 | 2700 | 0.0001 | - |
219
+ | 1.8333 | 2750 | 0.0001 | - |
220
+ | 1.8667 | 2800 | 0.0001 | - |
221
+ | 1.9 | 2850 | 0.0 | - |
222
+ | 1.9333 | 2900 | 0.0001 | - |
223
+ | 1.9667 | 2950 | 0.0 | - |
224
+ | 2.0 | 3000 | 0.0 | - |
225
+ | 2.0333 | 3050 | 0.0001 | - |
226
+ | 2.0667 | 3100 | 0.0 | - |
227
+ | 2.1 | 3150 | 0.0001 | - |
228
+ | 2.1333 | 3200 | 0.0001 | - |
229
+ | 2.1667 | 3250 | 0.0 | - |
230
+ | 2.2 | 3300 | 0.0001 | - |
231
+ | 2.2333 | 3350 | 0.0001 | - |
232
+ | 2.2667 | 3400 | 0.0001 | - |
233
+ | 2.3 | 3450 | 0.0 | - |
234
+ | 2.3333 | 3500 | 0.0001 | - |
235
+ | 2.3667 | 3550 | 0.0 | - |
236
+ | 2.4 | 3600 | 0.0 | - |
237
+ | 2.4333 | 3650 | 0.0 | - |
238
+ | 2.4667 | 3700 | 0.0001 | - |
239
+ | 2.5 | 3750 | 0.0 | - |
240
+ | 2.5333 | 3800 | 0.0001 | - |
241
+ | 2.5667 | 3850 | 0.0 | - |
242
+ | 2.6 | 3900 | 0.0001 | - |
243
+ | 2.6333 | 3950 | 0.0 | - |
244
+ | 2.6667 | 4000 | 0.0001 | - |
245
+ | 2.7 | 4050 | 0.0001 | - |
246
+ | 2.7333 | 4100 | 0.0 | - |
247
+ | 2.7667 | 4150 | 0.0 | - |
248
+ | 2.8 | 4200 | 0.0 | - |
249
+ | 2.8333 | 4250 | 0.0 | - |
250
+ | 2.8667 | 4300 | 0.0 | - |
251
+ | 2.9 | 4350 | 0.0 | - |
252
+ | 2.9333 | 4400 | 0.0 | - |
253
+ | 2.9667 | 4450 | 0.0001 | - |
254
+ | 3.0 | 4500 | 0.0 | - |
255
+ | 3.0333 | 4550 | 0.0001 | - |
256
+ | 3.0667 | 4600 | 0.0 | - |
257
+ | 3.1 | 4650 | 0.0 | - |
258
+ | 3.1333 | 4700 | 0.0001 | - |
259
+ | 3.1667 | 4750 | 0.0 | - |
260
+ | 3.2 | 4800 | 0.0 | - |
261
+ | 3.2333 | 4850 | 0.0 | - |
262
+ | 3.2667 | 4900 | 0.0 | - |
263
+ | 3.3 | 4950 | 0.0001 | - |
264
+ | 3.3333 | 5000 | 0.0 | - |
265
+ | 3.3667 | 5050 | 0.0 | - |
266
+ | 3.4 | 5100 | 0.0001 | - |
267
+ | 3.4333 | 5150 | 0.0 | - |
268
+ | 3.4667 | 5200 | 0.0 | - |
269
+ | 3.5 | 5250 | 0.0001 | - |
270
+ | 3.5333 | 5300 | 0.0 | - |
271
+ | 3.5667 | 5350 | 0.0 | - |
272
+ | 3.6 | 5400 | 0.0 | - |
273
+ | 3.6333 | 5450 | 0.0 | - |
274
+ | 3.6667 | 5500 | 0.0 | - |
275
+ | 3.7 | 5550 | 0.0001 | - |
276
+ | 3.7333 | 5600 | 0.0 | - |
277
+ | 3.7667 | 5650 | 0.0 | - |
278
+ | 3.8 | 5700 | 0.0 | - |
279
+ | 3.8333 | 5750 | 0.0 | - |
280
+ | 3.8667 | 5800 | 0.0 | - |
281
+ | 3.9 | 5850 | 0.0 | - |
282
+ | 3.9333 | 5900 | 0.0 | - |
283
+ | 3.9667 | 5950 | 0.0001 | - |
284
+ | 4.0 | 6000 | 0.0 | - |
285
 
286
  ### Framework Versions
287
  - Python: 3.9.16
config_setfit.json CHANGED
@@ -1,10 +1,11 @@
1
  {
2
- "normalize_embeddings": false,
3
  "labels": [
4
  "complaints",
5
  "order tracking",
6
  "product discoverability",
 
7
  "product faq",
8
  "product policy"
9
- ]
 
10
  }
 
1
  {
 
2
  "labels": [
3
  "complaints",
4
  "order tracking",
5
  "product discoverability",
6
+ "product discoveribility",
7
  "product faq",
8
  "product policy"
9
+ ],
10
+ "normalize_embeddings": false
11
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a4edc520d5781444ec5267ed4285584850d2cf89146f93c2be858180d79c0bbc
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae03ac1345d3ba6c609f184a07acf7265b5deee425cd424ce18ec3f74561823b
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2abda5fa6b9f1099cd9b59dc380bc1a2c16a2970265cd02c2246fbeb44ccd7b2
3
- size 32063
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6c7f26962d0c6ba43552743b68ebd7babd714d75d7a6158192612125e28b2b8c
3
+ size 38311