Hulyyy/req-quality-setfit-2

SetFit with sentence-transformers/all-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A ClassifierChain instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
• Classification head: a ClassifierChain instance
• Maximum Sequence Length: 384 tokens

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Evaluation

Metrics

Label	Accuracy
all	0.6061

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Hulyyy/req-quality-setfit-2")
# Run inference
preds = model("The System must provide End User and Administrator functions which are easy to use and intuitive throughout.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	42.3066	1112

Training Hyperparameters

• batch_size: (32, 32)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 100
• body_learning_rate: (3e-05, 3e-05)
• head_learning_rate: 3e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: True
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0001	1	0.3717	-
0.0058	50	0.3079	-
0.0116	100	0.2539	-
0.0174	150	0.2455	-
0.0231	200	0.2352	-
0.0289	250	0.2362	-
0.0347	300	0.2309	-
0.0405	350	0.2211	-
0.0463	400	0.2072	-
0.0521	450	0.2004	-
0.0578	500	0.1837	-
0.0636	550	0.1768	-
0.0694	600	0.1702	-
0.0752	650	0.1522	-
0.0810	700	0.1438	-
0.0868	750	0.141	-
0.0925	800	0.138	-
0.0983	850	0.1319	-
0.1041	900	0.1325	-
0.1099	950	0.1205	-
0.1157	1000	0.1146	-
0.1215	1050	0.1097	-
0.1273	1100	0.1171	-
0.1330	1150	0.1009	-
0.1388	1200	0.0917	-
0.1446	1250	0.0952	-
0.1504	1300	0.0896	-
0.1562	1350	0.0874	-
0.1620	1400	0.0884	-
0.1677	1450	0.0795	-
0.1735	1500	0.0849	-
0.1793	1550	0.0764	-
0.1851	1600	0.0776	-
0.1909	1650	0.0703	-
0.1967	1700	0.0649	-
0.2025	1750	0.0677	-
0.2082	1800	0.0677	-
0.2140	1850	0.0689	-
0.2198	1900	0.0675	-
0.2256	1950	0.064	-
0.2314	2000	0.0606	-
0.2372	2050	0.0606	-
0.2429	2100	0.0625	-
0.2487	2150	0.061	-
0.2545	2200	0.0608	-
0.2603	2250	0.0556	-
0.2661	2300	0.0559	-
0.2719	2350	0.0517	-
0.2776	2400	0.0533	-
0.2834	2450	0.0511	-
0.2892	2500	0.0568	-
0.2950	2550	0.0516	-
0.3008	2600	0.0498	-
0.3066	2650	0.0464	-
0.3124	2700	0.0503	-
0.3181	2750	0.0491	-
0.3239	2800	0.0488	-
0.3297	2850	0.0505	-
0.3355	2900	0.0506	-
0.3413	2950	0.0494	-
0.3471	3000	0.048	-
0.3528	3050	0.0471	-
0.3586	3100	0.0471	-
0.3644	3150	0.046	-
0.3702	3200	0.0456	-
0.3760	3250	0.0531	-
0.3818	3300	0.0458	-
0.3876	3350	0.0442	-
0.3933	3400	0.0459	-
0.3991	3450	0.045	-
0.4049	3500	0.0435	-
0.4107	3550	0.0446	-
0.4165	3600	0.0516	-
0.4223	3650	0.0459	-
0.4280	3700	0.0469	-
0.4338	3750	0.0446	-
0.4396	3800	0.0435	-
0.4454	3850	0.0459	-
0.4512	3900	0.0444	-
0.4570	3950	0.0434	-
0.4627	4000	0.0427	-
0.4685	4050	0.0418	-
0.4743	4100	0.0423	-
0.4801	4150	0.0441	-
0.4859	4200	0.0466	-
0.4917	4250	0.0463	-
0.4975	4300	0.0455	-
0.5032	4350	0.0471	-
0.5090	4400	0.0441	-
0.5148	4450	0.0431	-
0.5206	4500	0.0415	-
0.5264	4550	0.0452	-
0.5322	4600	0.0425	-
0.5379	4650	0.0453	-
0.5437	4700	0.0444	-
0.5495	4750	0.0468	-
0.5553	4800	0.0435	-
0.5611	4850	0.0406	-
0.5669	4900	0.0434	-
0.5727	4950	0.0425	-
0.5784	5000	0.0442	-
0.5842	5050	0.0448	-
0.5900	5100	0.0395	-
0.5958	5150	0.0426	-
0.6016	5200	0.0439	-
0.6074	5250	0.0418	-
0.6131	5300	0.0407	-
0.6189	5350	0.0462	-
0.6247	5400	0.0396	-
0.6305	5450	0.0424	-
0.6363	5500	0.0417	-
0.6421	5550	0.0428	-
0.6478	5600	0.0411	-
0.6536	5650	0.0421	-
0.6594	5700	0.0426	-
0.6652	5750	0.0454	-
0.6710	5800	0.043	-
0.6768	5850	0.0418	-
0.6826	5900	0.0453	-
0.6883	5950	0.0393	-
0.6941	6000	0.0433	-
0.6999	6050	0.0448	-
0.7057	6100	0.0439	-
0.7115	6150	0.0428	-
0.7173	6200	0.0431	-
0.7230	6250	0.0443	-
0.7288	6300	0.0409	-
0.7346	6350	0.0397	-
0.7404	6400	0.0408	-
0.7462	6450	0.0443	-
0.7520	6500	0.0401	-
0.7578	6550	0.0426	-
0.7635	6600	0.0404	-
0.7693	6650	0.0414	-
0.7751	6700	0.0396	-
0.7809	6750	0.0418	-
0.7867	6800	0.0403	-
0.7925	6850	0.0416	-
0.7982	6900	0.0404	-
0.8040	6950	0.0411	-
0.8098	7000	0.0403	-
0.8156	7050	0.0388	-
0.8214	7100	0.0414	-
0.8272	7150	0.0424	-
0.8329	7200	0.0424	-
0.8387	7250	0.0437	-
0.8445	7300	0.0392	-
0.8503	7350	0.0432	-
0.8561	7400	0.0401	-
0.8619	7450	0.0392	-
0.8677	7500	0.0391	-
0.8734	7550	0.0434	-
0.8792	7600	0.0433	-
0.8850	7650	0.0436	-
0.8908	7700	0.0389	-
0.8966	7750	0.0405	-
0.9024	7800	0.0396	-
0.9081	7850	0.0423	-
0.9139	7900	0.0391	-
0.9197	7950	0.0408	-
0.9255	8000	0.0373	-
0.9313	8050	0.04	-
0.9371	8100	0.0371	-
0.9429	8150	0.0393	-
0.9486	8200	0.0408	-
0.9544	8250	0.0407	-
0.9602	8300	0.0413	-
0.9660	8350	0.038	-
0.9718	8400	0.0377	-
0.9776	8450	0.0407	-
0.9833	8500	0.0421	-
0.9891	8550	0.0408	-
0.9949	8600	0.0412	-

Framework Versions

• Python: 3.13.7
• SetFit: 1.1.3
• Sentence Transformers: 5.1.1
• Transformers: 4.57.0
• PyTorch: 2.8.0+cu129
• Datasets: 4.2.0
• Tokenizers: 0.22.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}