Extractor_Adaptor_Qwen3_QA_websrc_second_chance

This model is a fine-tuned version of Qwen/Qwen3-0.6B on the web_finetune_train dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0949

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 8
  • eval_batch_size: 10
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 32
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 2.0

Training results

Training Loss Epoch Step Validation Loss
0.035 0.1067 50 0.1032
0.0209 0.2133 100 0.1463
0.0392 0.32 150 0.1112
0.0424 0.4267 200 0.1135
0.031 0.5333 250 0.1044
0.0338 0.64 300 0.0949
0.0478 0.7467 350 0.1124
0.0306 0.8533 400 0.1121
0.0352 0.96 450 0.1222
0.0193 1.0661 500 0.1458
0.0332 1.1728 550 0.1522
0.0264 1.2795 600 0.1254
0.0246 1.3861 650 0.1313
0.0201 1.4928 700 0.1422
0.0151 1.5995 750 0.1459
0.0206 1.7061 800 0.1486
0.0166 1.8128 850 0.1476
0.0189 1.9195 900 0.1466

Framework versions

  • PEFT 0.17.1
  • Transformers 4.57.1
  • Pytorch 2.4.1+cu124
  • Datasets 4.0.0
  • Tokenizers 0.22.1
Downloads last month
2
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for abdo-Mansour/Extractor_Adaptor_Qwen3_QA_websrc_second_chance

Finetuned
Qwen/Qwen3-0.6B
Adapter
(271)
this model