README.md

metadata

tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: You believe this old fossil?
  - text: I was 100% fossil.
  - text: >-
      Spandex blended with other fabrics like cotton or polyester can be ideal
      for exercise, because spandex is not only flexible but also durable.
  - text: >-
      The flexible schedule and departure every other day ensures that
      passengers will be able to find the ideal time for their Bahamas vacation.
  - text: >-
      Buying a new, greener refrigerator to replace an older model can be a
      great way to positively impact the environment.
metrics:
  - accuracy
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: sentence-transformers/all-mpnet-base-v2
model-index:
  - name: SetFit with sentence-transformers/all-mpnet-base-v2
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.9242424242424242
            name: Accuracy

SetFit with sentence-transformers/all-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/all-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/all-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 384 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
0	'A simple black spider is a fast face painting design that can make a big impact come Halloween.' 'A recently discovered fossil group, the Pteridospermae have characters intermediate between the Ptendophyta and the more primitive seedplants.' 'The Moral Balance model proposes that most humans operate out of a limited or flexible morality.'
1	'That fossil down the street?' 'Likewise, stores such as TJ Maxx, Ross and other discount clothing outlets often have Ralph Lauren clothing on sale, although you may have to be a bit more flexible about the color.' 'Giving some guidelines for the style, such as asking each attendant to wear matching hair pins, is fine, but being flexible will keep attendants smiling.'

Evaluation

Metrics

Label	Accuracy
all	0.9242

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("d31fs0/context-aware-language-classifier")
# Run inference
preds = model("I was 100% fossil.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	4	17.8011	46

Label	Training Sample Count
0	124
1	62

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.01
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0008	1	0.7262	-
0.0412	50	0.3557	-
0.0824	100	0.1985	-
0.1237	150	0.0489	-
0.1649	200	0.0019	-
0.2061	250	0.0006	-
0.2473	300	0.0004	-
0.2885	350	0.0003	-
0.3298	400	0.0002	-
0.3710	450	0.0002	-
0.4122	500	0.0002	-
0.4534	550	0.0001	-
0.4946	600	0.0001	-
0.5359	650	0.0001	-
0.5771	700	0.0001	-
0.6183	750	0.0001	-
0.6595	800	0.0001	-
0.7007	850	0.0001	-
0.7420	900	0.0001	-
0.7832	950	0.0001	-
0.8244	1000	0.0001	-
0.8656	1050	0.0001	-
0.9068	1100	0.0001	-
0.9481	1150	0.0001	-
0.9893	1200	0.0001	-
1.0	1213	-	0.1145

Framework Versions

• Python: 3.12.12
• SetFit: 1.1.3
• Sentence Transformers: 5.2.0
• Transformers: 4.57.6
• PyTorch: 2.9.0+cu126
• Datasets: 4.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}