Hugging Face's logo

Shankhdhar
/
classifier_woog

Text Classification
setfit
Safetensors
sentence-transformers
mpnet
generated_from_setfit_trainer
Eval Results (legacy)
Model card Files
xet
Community
Shankhdhar commited on
Commit
d89c51b
·
verified ·
1 Parent(s): 9fe73d3

Add SetFit model

Browse files
Files changed (5) hide show
  1. README.md +66 -86
  2. config.json +1 -1
  3. config_setfit.json +2 -3
  4. model.safetensors +1 -1
  5. model_head.pkl +2 -2
README.md CHANGED
@@ -9,12 +9,14 @@ base_model: sentence-transformers/paraphrase-mpnet-base-v2
9
  metrics:
10
  - accuracy
11
  widget:
12
- - text: What is the process for exchanging sneakers?
13
- - text: Can I track the delivery status of my order using the store's customer service
14
- hotline?
15
- - text: What are the latest choker styles available for a wedding occasion?
16
- - text: I'm interested in pendants that can be engraved. Do you provide such services?
 
17
  - text: Do you offer weekend or holiday deliveries for orders?
 
18
  pipeline_tag: text-classification
19
  inference: true
20
  model-index:
@@ -29,7 +31,7 @@ model-index:
29
  split: test
30
  metrics:
31
  - type: accuracy
32
- value: 0.9276729559748428
33
  name: Accuracy
34
  ---
35
 
@@ -49,7 +51,7 @@ The model has been trained using an efficient few-shot learning technique that i
49
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
50
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
51
  - **Maximum Sequence Length:** 512 tokens
52
- - **Number of Classes:** 6 classes
53
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
54
  <!-- - **Language:** Unknown -->
55
  <!-- - **License:** Unknown -->
@@ -61,21 +63,20 @@ The model has been trained using an efficient few-shot learning technique that i
61
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
62
 
63
  ### Model Labels
64
- | Label | Examples |
65
- |:------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
66
- | complaints | <ul><li>"I recently purchased the Teddy's Heartbeat Gold Pendant and I'm disappointed to see that the pendant scratches very easily. Is there anything that can be done about this?"</li><li>'I recently purchased the Pearly Round Earrings, but upon arrival, I noticed that the pearls are scratched and lack luster. This is not what I expected based on the product images online.'</li><li>'The Pearl Grace Ring I received has lost its shine within a week of purchase, which is disappointing. What can be done about this?'</li></ul> |
67
- | product discoverability | <ul><li>'Customized bakery boxes for specific needs'</li><li>'Suggest me some casual sneakers for women'</li><li>'Do you have Converse sneakers in different colors?'</li></ul> |
68
- | order tracking | <ul><li>"I'm concerned about the delay in the delivery of my order. Can you update me on the status?"</li><li>'I need to confirm the dispatch date for my recent purchase. Can you help me with that?'</li><li>'I want to know the status of my recent order. Can you provide me with the current shipping information?'</li></ul> |
69
- | product discoveribility | <ul><li>'Do you have any charm bracelets available at your store?'</li><li>'Do you have any statement Earrings that would be suitable for a wedding?'</li><li>'Do you have any choker-style necklaces that are trending right now?'</li></ul> |
70
- | product faq | <ul><li>'What is the material used for the red round decorative tin box for wholesale orders?'</li><li>'Are the Adidas Yeezy Foam RNNR MX Cinder unisex?'</li><li>'Is the pack of 50 popcorn boxes available in different colors?'</li></ul> |
71
- | product policy | <ul><li>'How can I find out my ring size before placing an order?'</li><li>'What is the warranty on candle supplies?'</li><li>'Can I get a refund for a necklace if it has a manufacturing defect?'</li></ul> |
72
 
73
  ## Evaluation
74
 
75
  ### Metrics
76
  | Label | Accuracy |
77
  |:--------|:---------|
78
- | **all** | 0.9277 |
79
 
80
  ## Uses
81
 
@@ -95,7 +96,7 @@ from setfit import SetFitModel
95
  # Download from the 🤗 Hub
96
  model = SetFitModel.from_pretrained("setfit_model_id")
97
  # Run inference
98
- preds = model("What is the process for exchanging sneakers?")
99
  ```
100
 
101
  <!--
@@ -125,16 +126,15 @@ preds = model("What is the process for exchanging sneakers?")
125
  ## Training Details
126
 
127
  ### Training Set Metrics
128
- | Training set | Min | Median | Max |
129
- |:-------------|:----|:--------|:----|
130
- | Word count | 4 | 15.4667 | 37 |
131
 
132
  | Label | Training Sample Count |
133
  |:------------------------|:----------------------|
134
  | complaints | 20 |
135
  | order tracking | 20 |
136
  | product discoverability | 20 |
137
- | product discoveribility | 20 |
138
  | product faq | 20 |
139
  | product policy | 20 |
140
 
@@ -156,77 +156,57 @@ preds = model("What is the process for exchanging sneakers?")
156
  - load_best_model_at_end: True
157
 
158
  ### Training Results
159
- | Epoch | Step | Training Loss | Validation Loss |
160
- |:------:|:----:|:-------------:|:---------------:|
161
- | 0.0013 | 1 | 0.2116 | - |
162
- | 0.0667 | 50 | 0.1402 | - |
163
- | 0.1333 | 100 | 0.1163 | - |
164
- | 0.2 | 150 | 0.024 | - |
165
- | 0.2667 | 200 | 0.0037 | - |
166
- | 0.3333 | 250 | 0.0016 | - |
167
- | 0.4 | 300 | 0.0011 | - |
168
- | 0.4667 | 350 | 0.0008 | - |
169
- | 0.5333 | 400 | 0.0004 | - |
170
- | 0.6 | 450 | 0.0005 | - |
171
- | 0.6667 | 500 | 0.001 | - |
172
- | 0.7333 | 550 | 0.0002 | - |
173
- | 0.8 | 600 | 0.0002 | - |
174
- | 0.8667 | 650 | 0.0003 | - |
175
- | 0.9333 | 700 | 0.0002 | - |
176
- | 1.0 | 750 | 0.0002 | - |
177
- | 1.0667 | 800 | 0.0002 | - |
178
- | 1.1333 | 850 | 0.0002 | - |
179
- | 1.2 | 900 | 0.0001 | - |
180
- | 1.2667 | 950 | 0.0001 | - |
181
- | 1.3333 | 1000 | 0.0002 | - |
182
- | 1.4 | 1050 | 0.0001 | - |
183
- | 1.4667 | 1100 | 0.0001 | - |
184
- | 1.5333 | 1150 | 0.0001 | - |
185
- | 1.6 | 1200 | 0.0002 | - |
186
- | 1.6667 | 1250 | 0.0001 | - |
187
- | 1.7333 | 1300 | 0.0001 | - |
188
- | 1.8 | 1350 | 0.0002 | - |
189
- | 1.8667 | 1400 | 0.0001 | - |
190
- | 1.9333 | 1450 | 0.0001 | - |
191
- | 2.0 | 1500 | 0.0001 | - |
192
- | 2.0667 | 1550 | 0.0001 | - |
193
- | 2.1333 | 1600 | 0.0002 | - |
194
- | 2.2 | 1650 | 0.0001 | - |
195
- | 2.2667 | 1700 | 0.0001 | - |
196
- | 2.3333 | 1750 | 0.0001 | - |
197
- | 2.4 | 1800 | 0.0001 | - |
198
- | 2.4667 | 1850 | 0.0001 | - |
199
- | 2.5333 | 1900 | 0.0001 | - |
200
- | 2.6 | 1950 | 0.0001 | - |
201
- | 2.6667 | 2000 | 0.0001 | - |
202
- | 2.7333 | 2050 | 0.0001 | - |
203
- | 2.8 | 2100 | 0.0001 | - |
204
- | 2.8667 | 2150 | 0.0001 | - |
205
- | 2.9333 | 2200 | 0.0001 | - |
206
- | 3.0 | 2250 | 0.0001 | - |
207
- | 3.0667 | 2300 | 0.0001 | - |
208
- | 3.1333 | 2350 | 0.0001 | - |
209
- | 3.2 | 2400 | 0.0001 | - |
210
- | 3.2667 | 2450 | 0.0001 | - |
211
- | 3.3333 | 2500 | 0.0001 | - |
212
- | 3.4 | 2550 | 0.0001 | - |
213
- | 3.4667 | 2600 | 0.0001 | - |
214
- | 3.5333 | 2650 | 0.0001 | - |
215
- | 3.6 | 2700 | 0.0001 | - |
216
- | 3.6667 | 2750 | 0.0001 | - |
217
- | 3.7333 | 2800 | 0.0001 | - |
218
- | 3.8 | 2850 | 0.0001 | - |
219
- | 3.8667 | 2900 | 0.0001 | - |
220
- | 3.9333 | 2950 | 0.0001 | - |
221
- | 4.0 | 3000 | 0.0001 | - |
222
 
223
  ### Framework Versions
224
  - Python: 3.9.16
225
  - SetFit: 1.0.3
226
  - Sentence Transformers: 2.7.0
227
- - Transformers: 4.40.1
228
  - PyTorch: 2.3.0
229
- - Datasets: 2.19.0
230
  - Tokenizers: 0.19.1
231
 
232
  ## Citation
 
9
  metrics:
10
  - accuracy
11
  widget:
12
+ - text: Is the lavender round empty decorative acrylic box available in a smaller
13
+ size?
14
+ - text: I recently purchased the Unicorn Dream Silver Earring, but I am disappointed
15
+ to find that the quality does not match what was advertised. The silver seems
16
+ to tarnish much faster than expected. Can you address this issue?
17
+ - text: Can I get a refund for a necklace if it has a manufacturing defect?
18
  - text: Do you offer weekend or holiday deliveries for orders?
19
+ - text: What apparel do you have from Nike?
20
  pipeline_tag: text-classification
21
  inference: true
22
  model-index:
 
31
  split: test
32
  metrics:
33
  - type: accuracy
34
+ value: 0.949685534591195
35
  name: Accuracy
36
  ---
37
 
 
51
  - **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
52
  - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
53
  - **Maximum Sequence Length:** 512 tokens
54
+ - **Number of Classes:** 5 classes
55
  <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
56
  <!-- - **Language:** Unknown -->
57
  <!-- - **License:** Unknown -->
 
63
  - **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
64
 
65
  ### Model Labels
66
+ | Label | Examples |
67
+ |:------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
68
+ | product discoverability | <ul><li>'Do you have Converse sneakers in different colors?'</li><li>'pink bakery boxes for gifting'</li><li>'Could you suggest some Earring options that go well with traditional outfits?'</li></ul> |
69
+ | order tracking | <ul><li>"Can I track the delivery status of my order using the store's customer service hotline?"</li><li>"I recently ordered the Pakhi Handcrafted Earring but I haven't received any shipping confirmation. Could you please update me on the status of my order?"</li><li>'What is the process for claiming a lost or damaged shipment?'</li></ul> |
70
+ | complaints | <ul><li>"The Blossom Vintage cocktail ring I received looks tarnished and doesn't match the quality depicted on the website."</li><li>"I recently purchased the Teddy's Heartbeat Gold Pendant and I'm disappointed to see that the pendant scratches very easily. Is there anything that can be done about this?"</li><li>'I recently bought the Green Floral Bangles with White Rhodium Polish and I have noticed that the polish is already coming off. This is not what I expected so soon after purchase.'</li></ul> |
71
+ | product policy | <ul><li>'Do you offer a satisfaction guarantee for sneakers purchased on clearance?'</li><li>'Are earrings eligible for exchange in case I receive a defective piece?'</li><li>'Do you offer any authenticity certificates for necklaces made with precious stones and metals?'</li></ul> |
72
+ | product faq | <ul><li>'Do the Nike Blazer Mid sacai Snow Beach run small or large'</li><li>'Are there any special discounts on the PVC chocolate boxes for bulk orders for wholesale orders for wholesale orders?'</li><li>'Can the huge glitter heart rigid box be used for storage purposes?'</li></ul> |
 
73
 
74
  ## Evaluation
75
 
76
  ### Metrics
77
  | Label | Accuracy |
78
  |:--------|:---------|
79
+ | **all** | 0.9497 |
80
 
81
  ## Uses
82
 
 
96
  # Download from the 🤗 Hub
97
  model = SetFitModel.from_pretrained("setfit_model_id")
98
  # Run inference
99
+ preds = model("What apparel do you have from Nike?")
100
  ```
101
 
102
  <!--
 
126
  ## Training Details
127
 
128
  ### Training Set Metrics
129
+ | Training set | Min | Median | Max |
130
+ |:-------------|:----|:-------|:----|
131
+ | Word count | 4 | 16.58 | 37 |
132
 
133
  | Label | Training Sample Count |
134
  |:------------------------|:----------------------|
135
  | complaints | 20 |
136
  | order tracking | 20 |
137
  | product discoverability | 20 |
 
138
  | product faq | 20 |
139
  | product policy | 20 |
140
 
 
156
  - load_best_model_at_end: True
157
 
158
  ### Training Results
159
+ | Epoch | Step | Training Loss | Validation Loss |
160
+ |:-----:|:----:|:-------------:|:---------------:|
161
+ | 0.002 | 1 | 0.2231 | - |
162
+ | 0.1 | 50 | 0.1432 | - |
163
+ | 0.2 | 100 | 0.0347 | - |
164
+ | 0.3 | 150 | 0.0031 | - |
165
+ | 0.4 | 200 | 0.0011 | - |
166
+ | 0.5 | 250 | 0.0007 | - |
167
+ | 0.6 | 300 | 0.0005 | - |
168
+ | 0.7 | 350 | 0.0003 | - |
169
+ | 0.8 | 400 | 0.0003 | - |
170
+ | 0.9 | 450 | 0.0002 | - |
171
+ | 1.0 | 500 | 0.0003 | - |
172
+ | 1.1 | 550 | 0.0003 | - |
173
+ | 1.2 | 600 | 0.0002 | - |
174
+ | 1.3 | 650 | 0.0002 | - |
175
+ | 1.4 | 700 | 0.0002 | - |
176
+ | 1.5 | 750 | 0.0002 | - |
177
+ | 1.6 | 800 | 0.0002 | - |
178
+ | 1.7 | 850 | 0.0001 | - |
179
+ | 1.8 | 900 | 0.0001 | - |
180
+ | 1.9 | 950 | 0.0002 | - |
181
+ | 2.0 | 1000 | 0.0001 | - |
182
+ | 2.1 | 1050 | 0.0001 | - |
183
+ | 2.2 | 1100 | 0.0001 | - |
184
+ | 2.3 | 1150 | 0.0001 | - |
185
+ | 2.4 | 1200 | 0.0001 | - |
186
+ | 2.5 | 1250 | 0.0001 | - |
187
+ | 2.6 | 1300 | 0.0001 | - |
188
+ | 2.7 | 1350 | 0.0001 | - |
189
+ | 2.8 | 1400 | 0.0001 | - |
190
+ | 2.9 | 1450 | 0.0001 | - |
191
+ | 3.0 | 1500 | 0.0001 | - |
192
+ | 3.1 | 1550 | 0.0001 | - |
193
+ | 3.2 | 1600 | 0.0001 | - |
194
+ | 3.3 | 1650 | 0.0001 | - |
195
+ | 3.4 | 1700 | 0.0001 | - |
196
+ | 3.5 | 1750 | 0.0001 | - |
197
+ | 3.6 | 1800 | 0.0001 | - |
198
+ | 3.7 | 1850 | 0.0001 | - |
199
+ | 3.8 | 1900 | 0.0001 | - |
200
+ | 3.9 | 1950 | 0.0001 | - |
201
+ | 4.0 | 2000 | 0.0001 | - |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
202
 
203
  ### Framework Versions
204
  - Python: 3.9.16
205
  - SetFit: 1.0.3
206
  - Sentence Transformers: 2.7.0
207
+ - Transformers: 4.40.2
208
  - PyTorch: 2.3.0
209
+ - Datasets: 2.19.1
210
  - Tokenizers: 0.19.1
211
 
212
  ## Citation
config.json CHANGED
@@ -19,6 +19,6 @@
19
  "pad_token_id": 1,
20
  "relative_attention_num_buckets": 32,
21
  "torch_dtype": "float32",
22
- "transformers_version": "4.40.1",
23
  "vocab_size": 30527
24
  }
 
19
  "pad_token_id": 1,
20
  "relative_attention_num_buckets": 32,
21
  "torch_dtype": "float32",
22
+ "transformers_version": "4.40.2",
23
  "vocab_size": 30527
24
  }
config_setfit.json CHANGED
@@ -1,11 +1,10 @@
1
  {
 
2
  "labels": [
3
  "complaints",
4
  "order tracking",
5
  "product discoverability",
6
- "product discoveribility",
7
  "product faq",
8
  "product policy"
9
- ],
10
- "normalize_embeddings": false
11
  }
 
1
  {
2
+ "normalize_embeddings": false,
3
  "labels": [
4
  "complaints",
5
  "order tracking",
6
  "product discoverability",
 
7
  "product faq",
8
  "product policy"
9
+ ]
 
10
  }
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9cf05a6ea6f3d6bc0260b7b8fa89947d13e691d4dffa11ee25937ff6da3fa891
3
  size 437967672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a4edc520d5781444ec5267ed4285584850d2cf89146f93c2be858180d79c0bbc
3
  size 437967672
model_head.pkl CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e6a802549e27335ab3dee1b924ba13d3f033957dbe3522c02056ba823944040d
3
- size 38311
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2abda5fa6b9f1099cd9b59dc380bc1a2c16a2970265cd02c2246fbeb44ccd7b2
3
+ size 32063