SetFit with codefuse-ai/F2LLM-v2-80M

This is a SetFit model that can be used for Text Classification. This SetFit model uses codefuse-ai/F2LLM-v2-80M as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: codefuse-ai/F2LLM-v2-80M
Classification head: a LogisticRegression instance
Maximum Sequence Length: 40960 tokens
Number of Classes: 2 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
positive	'He and other thinkers anguishing over the inadequacy of global mechanisms advocate a new set of principles which would allow for intervention in a nation in crisis.' 'Plus, there are escalating concerns over cyberthreats, territorial disputes in the South China Sea and the spread of Ebola virus.' 'Now, says Urban, Russia, China and India have such strong conventional forces, and America has cut its forces so much, that in the event of a conflict "the US would be left with the choice of nuclear escalation or backing down".'
negative	'Positioning nuclear missiles in the exclave would give the Kremlin the ability to strike most central European capitals.' 'Discard the Trident missile programme.' "We're really going to choose between what I consider to be an old way of governing ourselves, of high levels of spending, high taxes, an ever more intrusive bureaucracy, or a new course, a new era if you will, and Governor Bush and I want to offer that new course of action."

Label

Examples

positive

'He and other thinkers anguishing over the inadequacy of global mechanisms advocate a new set of principles which would allow for intervention in a nation in crisis.'
'Plus, there are escalating concerns over cyberthreats, territorial disputes in the South China Sea and the spread of Ebola virus.'
'Now, says Urban, Russia, China and India have such strong conventional forces, and America has cut its forces so much, that in the event of a conflict "the US would be left with the choice of nuclear escalation or backing down".'

negative

'Positioning nuclear missiles in the exclave would give the Kremlin the ability to strike most central European capitals.'
'Discard the Trident missile programme.'
"We're really going to choose between what I consider to be an old way of governing ourselves, of high levels of spending, high taxes, an ever more intrusive bureaucracy, or a new course, a new era if you will, and Governor Bush and I want to offer that new course of action."

Evaluation

Metrics

Label	F1_Macro	F1_Binary
all	0.7436	0.6316

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("fefofico/crisis_trained_allnat")
# Run inference
preds = model("Can we really believe that the nuclear weapon will never be used again?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	24.5399	67

Label	Training Sample Count
negative	361
positive	178

Training Hyperparameters

batch_size: (256, 256)
num_epochs: (5, 5)
max_steps: -1
sampling_strategy: oversampling
num_iterations: 20
body_learning_rate: (1e-06, 1e-06)
head_learning_rate: 0.003
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0233	1	0.4268	-
0.4651	20	0.4109	-
0.0118	1	0.3813	-
0.2353	20	0.3829	-
0.4706	40	0.3552	-
0.7059	60	0.3192	-
0.9412	80	0.2901	-
1.0	85	-	0.2554
1.1765	100	0.2728	-
1.4118	120	0.2668	-
1.6471	140	0.2596	-
1.8824	160	0.2556	-
2.0	170	-	0.2423
2.1176	180	0.2489	-
2.3529	200	0.2442	-
2.5882	220	0.239	-
2.8235	240	0.2354	-
3.0	255	-	0.2356
3.0588	260	0.2281	-
3.2941	280	0.2244	-
3.5294	300	0.2227	-
3.7647	320	0.2175	-
4.0	340	0.2114	0.2313
4.2353	360	0.2103	-
4.4706	380	0.2055	-
4.7059	400	0.2048	-
4.9412	420	0.2029	-
5.0	425	-	0.2297

Framework Versions

Python: 3.12.13
SetFit: 1.1.3
Sentence Transformers: 3.4.1
Transformers: 4.57.6
PyTorch: 2.11.0+cu128
Datasets: 5.0.0
Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}