Shankhdhar/jewellery_query_setfit_classifier

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 4 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
product faq	'What are the different sizes available for the Love is in the Air Proposal Ring, and do they come at different price points?' 'What is the material of the Open Pear Cut Ring and are there different sizes available?' 'What is the material used for making the Golden Spin Hoop Earring, and does it come with any kind of warranty or guarantee?'
product discoveribility	'What are the latest choker styles available for a wedding occasion?' "I'm interested in sustainable jewelry; do you have any eco-friendly necklaces?" 'Could you recommend some necklaces with a vintage vibe to them?'
order tracking	'I recently purchased the Seher Pearl Choker Set and I would like to know the current status of my order delivery.' "I placed an order for the Tiara Silver Ring, but I haven't received any shipping updates yet. Can you provide me with the current status of my order?" 'I recently ordered the Toes Of Love Pendant but have not received any shipping confirmation. Could you please provide me with the tracking details?'
product policy	'Are there any restocking fees for bracelet returns?' "Can I exchange a ring if it doesn't fit properly?" 'Are there any care instructions included with the purchase of a ring?'

Evaluation

Metrics

Label	Accuracy
all	0.8025

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("I recently purchased the Three Crystal Proposal Ring, but I'm disappointed to find that one of the crystals is loose. Can you assist me with this issue?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	6	16.4474	30

Label	Training Sample Count
negative	0
positive	0

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (4, 4)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0016	1	0.1464	-
0.0822	50	0.0907	-
0.1645	100	0.0059	-
0.2467	150	0.0013	-
0.3289	200	0.0009	-
0.4112	250	0.0007	-
0.4934	300	0.0004	-
0.5757	350	0.0003	-
0.6579	400	0.0001	-
0.7401	450	0.0002	-
0.8224	500	0.0002	-
0.9046	550	0.0002	-
0.9868	600	0.0001	-
1.0	608	-	0.2272
1.0691	650	0.0001	-
1.1513	700	0.0001	-
1.2336	750	0.0001	-
1.3158	800	0.0001	-
1.3980	850	0.0001	-
1.4803	900	0.0001	-
1.5625	950	0.0001	-
1.6447	1000	0.0001	-
1.7270	1050	0.0001	-
1.8092	1100	0.0	-
1.8914	1150	0.0001	-
1.9737	1200	0.0001	-
2.0	1216	-	0.2807
2.0559	1250	0.0001	-
2.1382	1300	0.0001	-
2.2204	1350	0.0001	-
2.3026	1400	0.0	-
2.3849	1450	0.0001	-
2.4671	1500	0.0001	-
2.5493	1550	0.0	-
2.6316	1600	0.0001	-
2.7138	1650	0.0	-
2.7961	1700	0.0001	-
2.8783	1750	0.0	-
2.9605	1800	0.0	-
3.0	1824	-	0.3011
3.0428	1850	0.0	-
3.125	1900	0.0001	-
3.2072	1950	0.0001	-
3.2895	2000	0.0	-
3.3717	2050	0.0001	-
3.4539	2100	0.0001	-
3.5362	2150	0.0	-
3.6184	2200	0.0001	-
3.7007	2250	0.0001	-
3.7829	2300	0.0	-
3.8651	2350	0.0	-
3.9474	2400	0.0001	-
4.0	2432	-	0.311

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.9.16
• SetFit: 1.0.3
• Sentence Transformers: 2.2.2
• Transformers: 4.35.2
• PyTorch: 2.1.1
• Datasets: 2.15.0
• Tokenizers: 0.15.0

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}