d31fs0/context-aware-language-classifier

SetFit with sentence-transformers/all-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 384 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	'A simple black spider is a fast face painting design that can make a big impact come Halloween.' 'A recently discovered fossil group, the Pteridospermae have characters intermediate between the Ptendophyta and the more primitive seedplants.' 'The Moral Balance model proposes that most humans operate out of a limited or flexible morality.'
1	'That fossil down the street?' 'Likewise, stores such as TJ Maxx, Ross and other discount clothing outlets often have Ralph Lauren clothing on sale, although you may have to be a bit more flexible about the color.' 'Giving some guidelines for the style, such as asking each attendant to wear matching hair pins, is fine, but being flexible will keep attendants smiling.'

Evaluation

Metrics

Label	Accuracy
all	0.9242

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("d31fs0/context-aware-language-classifier")
# Run inference
preds = model("I was 100% fossil.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	17.8011	46

Label	Training Sample Count
0	124
1	62

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0008	1	0.7262	-
0.0412	50	0.3557	-
0.0824	100	0.1985	-
0.1237	150	0.0489	-
0.1649	200	0.0019	-
0.2061	250	0.0006	-
0.2473	300	0.0004	-
0.2885	350	0.0003	-
0.3298	400	0.0002	-
0.3710	450	0.0002	-
0.4122	500	0.0002	-
0.4534	550	0.0001	-
0.4946	600	0.0001	-
0.5359	650	0.0001	-
0.5771	700	0.0001	-
0.6183	750	0.0001	-
0.6595	800	0.0001	-
0.7007	850	0.0001	-
0.7420	900	0.0001	-
0.7832	950	0.0001	-
0.8244	1000	0.0001	-
0.8656	1050	0.0001	-
0.9068	1100	0.0001	-
0.9481	1150	0.0001	-
0.9893	1200	0.0001	-
1.0	1213	-	0.1145

Framework Versions

• Python: 3.12.12
• SetFit: 1.1.3
• Sentence Transformers: 5.2.0
• Transformers: 4.57.6
• PyTorch: 2.9.0+cu126
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}