fefofico/nuclear_trained_f2llm_temp

SetFit with codefuse-ai/F2LLM-v2-80M

This is a SetFit model that can be used for Text Classification. This SetFit model uses codefuse-ai/F2LLM-v2-80M as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: codefuse-ai/F2LLM-v2-80M
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 40960 tokens
• Number of Classes: 2 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
positive	'actually, we have decided in nato at the last summit that we should explore the possibilities to integrate russian missile defence systems in our missile defence system, which i think has become even more easy after the u.s. has presented new missile defence plans.' 'and we see, of course, the risk of proliferation of nuclear weapons.' 'there are of course opportunities and we need to engage with china on issues like climate change, arms control.'
negative	'We welcome the successful achievement of a draft Chemical Weapons Convention.' 'in practice, this means that, in addition to reinforcing cooperation with our current partners, we should look to enhance our relations with countries such as australia , new zealand , japan and south korea.' 'this is not about militarizing space.'

Evaluation

Metrics

Label	F1_Macro	F1_Binary
all	0.9169	0.9065

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("fefofico/nuclear_trained_f2llm_temp")
# Run inference
preds = model("let me first of all say that we take nuclear issues extremely seriously.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	2	23.7926	132

Label	Training Sample Count
negative	1096
positive	857

Training Hyperparameters

• batch_size: (128, 128)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (5e-07, 5e-07)
• head_learning_rate: 0.0002
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• l2_weight: 0.35
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0016	1	0.3497	-
0.0327	20	0.3955	-
0.0655	40	0.3678	-
0.0982	60	0.3743	-
0.1309	80	0.3384	-
0.1637	100	0.3244	-
0.1964	120	0.3087	-
0.2291	140	0.29	-
0.2619	160	0.2675	-
0.2946	180	0.2659	-
0.3273	200	0.2584	-
0.3601	220	0.2573	-
0.3928	240	0.2544	-
0.4255	260	0.2541	-
0.4583	280	0.2498	-
0.4910	300	0.2492	-
0.5237	320	0.2471	-
0.5565	340	0.2397	-
0.5892	360	0.2356	-
0.6219	380	0.2287	-
0.6547	400	0.2277	-
0.6874	420	0.223	-
0.7201	440	0.2169	-
0.7529	460	0.2154	-
0.7856	480	0.2067	-
0.8183	500	0.2084	-
0.8511	520	0.1983	-
0.8838	540	0.199	-
0.9165	560	0.1999	-
0.9493	580	0.1939	-
0.9820	600	0.1909	-
1.0	611	-	0.1728
1.0147	620	0.1827	-
1.0475	640	0.1736	-
1.0802	660	0.1788	-
1.1129	680	0.1741	-
1.1457	700	0.1731	-
1.1784	720	0.1734	-
1.2111	740	0.1645	-
1.2439	760	0.1679	-
1.2766	780	0.1602	-
1.3093	800	0.1525	-
1.3421	820	0.1519	-
1.3748	840	0.1563	-
1.4075	860	0.1564	-
1.4403	880	0.1502	-
1.4730	900	0.144	-
1.5057	920	0.1479	-
1.5385	940	0.1472	-
1.5712	960	0.1461	-
1.6039	980	0.137	-
1.6367	1000	0.1497	-
1.6694	1020	0.1433	-
1.7021	1040	0.1426	-
1.7349	1060	0.1345	-
1.7676	1080	0.1406	-
1.8003	1100	0.135	-
1.8331	1120	0.1434	-
1.8658	1140	0.1407	-
1.8985	1160	0.1353	-
1.9313	1180	0.133	-
1.9640	1200	0.1326	-
1.9967	1220	0.1357	-
2.0	1222	-	0.1313
0.0016	1	0.1361	-
0.0327	20	0.1349	-
0.0655	40	0.1338	-
0.0982	60	0.1338	-
0.1309	80	0.1412	-
0.1637	100	0.1269	-
0.1964	120	0.1213	-
0.2291	140	0.1266	-
0.2619	160	0.1239	-
0.2946	180	0.1162	-
0.3273	200	0.1121	-
0.3601	220	0.1136	-
0.3928	240	0.111	-
0.4255	260	0.11	-
0.4583	280	0.1091	-
0.4910	300	0.1009	-
0.5237	320	0.0963	-
0.5565	340	0.094	-
0.5892	360	0.1001	-
0.6219	380	0.0956	-
0.6547	400	0.0889	-
0.6874	420	0.0895	-
0.7201	440	0.0934	-
0.7529	460	0.0857	-
0.7856	480	0.0882	-
0.8183	500	0.0878	-
0.8511	520	0.0878	-
0.8838	540	0.0909	-
0.9165	560	0.0928	-
0.9493	580	0.0903	-
0.9820	600	0.0925	-
1.0	611	-	0.1090

Framework Versions

• Python: 3.12.13
• SetFit: 1.1.3
• Sentence Transformers: 3.4.1
• Transformers: 4.57.6
• PyTorch: 2.11.0+cu128
• Datasets: 5.0.0
• Tokenizers: 0.22.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}