Hugging Face's logo

fefofico
/
nuclear_trained_f2llm_temp

Text Classification
setfit
Safetensors
sentence-transformers
qwen3
generated_from_setfit_trainer
Eval Results (legacy)
text-embeddings-inference
Model card Files
xet
 Community
nuclear_trained_f2llm_temp / README.md
fefofico's picture
fefofico
Push model using huggingface_hub.
120049e verified
preview code
|
Raw
History Blame Contribute Delete
13 kB
metadata 
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
widget:
  - text: >-
      Federal Office for Radiation Protection established a new monitoring
      network around nuclear waste storage sites.
  - text: >-
      how could we imagine these mechanisms to be implemented within a
      nato-based missile defence system.
  - text: >-
      president putin said that the precondition for a ceasefire is that ukraine
      should give up even more land, to give up all the four provinces that
      russia has annexed without controlling.
  - text: >-
      and helped protect and defend turkey’s territory and citizens against
      missile attacks.
  - text: let me first of all say that we take nuclear issues extremely seriously.
metrics:
  - f1_macro
  - f1_binary
pipeline_tag: text-classification
library_name: setfit
inference: true
base_model: codefuse-ai/F2LLM-v2-80M
model-index:
  - name: SetFit with codefuse-ai/F2LLM-v2-80M
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: f1_macro
            value: 0.9169073916737468
            name: F1_Macro
          - type: f1_binary
            value: 0.9065420560747663
            name: F1_Binary

SetFit with codefuse-ai/F2LLM-v2-80M

This is a SetFit model that can be used for Text Classification. This SetFit model uses codefuse-ai/F2LLM-v2-80M as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
positive
  • 'actually, we have decided in nato at the last summit that we should explore the possibilities to integrate russian missile defence systems in our missile defence system, which i think has become even more easy after the u.s. has presented new missile defence plans.'
  • 'and we see, of course, the risk of proliferation of nuclear weapons.'
  • 'there are of course opportunities and we need to engage with china on issues like climate change, arms control.'
negative
  • 'We welcome the successful achievement of a draft Chemical Weapons Convention.'
  • 'in practice, this means that, in addition to reinforcing cooperation with our current partners, we should look to enhance our relations with countries such as australia , new zealand , japan and south korea.'
  • 'this is not about militarizing space.'

Evaluation

Metrics

Label F1_Macro F1_Binary
all 0.9169 0.9065

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("fefofico/nuclear_trained_f2llm_temp")
# Run inference
preds = model("let me first of all say that we take nuclear issues extremely seriously.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 2 23.7926 132
Label Training Sample Count
negative 1096
positive 857

Training Hyperparameters

  • batch_size: (128, 128)
  • num_epochs: (1, 1)
  • max_steps: -1
  • sampling_strategy: oversampling
  • num_iterations: 20
  • body_learning_rate: (5e-07, 5e-07)
  • head_learning_rate: 0.0002
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • l2_weight: 0.35
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0016 1 0.3497 -
0.0327 20 0.3955 -
0.0655 40 0.3678 -
0.0982 60 0.3743 -
0.1309 80 0.3384 -
0.1637 100 0.3244 -
0.1964 120 0.3087 -
0.2291 140 0.29 -
0.2619 160 0.2675 -
0.2946 180 0.2659 -
0.3273 200 0.2584 -
0.3601 220 0.2573 -
0.3928 240 0.2544 -
0.4255 260 0.2541 -
0.4583 280 0.2498 -
0.4910 300 0.2492 -
0.5237 320 0.2471 -
0.5565 340 0.2397 -
0.5892 360 0.2356 -
0.6219 380 0.2287 -
0.6547 400 0.2277 -
0.6874 420 0.223 -
0.7201 440 0.2169 -
0.7529 460 0.2154 -
0.7856 480 0.2067 -
0.8183 500 0.2084 -
0.8511 520 0.1983 -
0.8838 540 0.199 -
0.9165 560 0.1999 -
0.9493 580 0.1939 -
0.9820 600 0.1909 -
1.0 611 - 0.1728
1.0147 620 0.1827 -
1.0475 640 0.1736 -
1.0802 660 0.1788 -
1.1129 680 0.1741 -
1.1457 700 0.1731 -
1.1784 720 0.1734 -
1.2111 740 0.1645 -
1.2439 760 0.1679 -
1.2766 780 0.1602 -
1.3093 800 0.1525 -
1.3421 820 0.1519 -
1.3748 840 0.1563 -
1.4075 860 0.1564 -
1.4403 880 0.1502 -
1.4730 900 0.144 -
1.5057 920 0.1479 -
1.5385 940 0.1472 -
1.5712 960 0.1461 -
1.6039 980 0.137 -
1.6367 1000 0.1497 -
1.6694 1020 0.1433 -
1.7021 1040 0.1426 -
1.7349 1060 0.1345 -
1.7676 1080 0.1406 -
1.8003 1100 0.135 -
1.8331 1120 0.1434 -
1.8658 1140 0.1407 -
1.8985 1160 0.1353 -
1.9313 1180 0.133 -
1.9640 1200 0.1326 -
1.9967 1220 0.1357 -
2.0 1222 - 0.1313
0.0016 1 0.1361 -
0.0327 20 0.1349 -
0.0655 40 0.1338 -
0.0982 60 0.1338 -
0.1309 80 0.1412 -
0.1637 100 0.1269 -
0.1964 120 0.1213 -
0.2291 140 0.1266 -
0.2619 160 0.1239 -
0.2946 180 0.1162 -
0.3273 200 0.1121 -
0.3601 220 0.1136 -
0.3928 240 0.111 -
0.4255 260 0.11 -
0.4583 280 0.1091 -
0.4910 300 0.1009 -
0.5237 320 0.0963 -
0.5565 340 0.094 -
0.5892 360 0.1001 -
0.6219 380 0.0956 -
0.6547 400 0.0889 -
0.6874 420 0.0895 -
0.7201 440 0.0934 -
0.7529 460 0.0857 -
0.7856 480 0.0882 -
0.8183 500 0.0878 -
0.8511 520 0.0878 -
0.8838 540 0.0909 -
0.9165 560 0.0928 -
0.9493 580 0.0903 -
0.9820 600 0.0925 -
1.0 611 - 0.1090

Framework Versions

  • Python: 3.12.13
  • SetFit: 1.1.3
  • Sentence Transformers: 3.4.1
  • Transformers: 4.57.6
  • PyTorch: 2.11.0+cu128
  • Datasets: 5.0.0
  • Tokenizers: 0.22.2

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}