HybridModelScratch

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.0191
• Accuracy: 0.1528

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 8
• total_train_batch_size: 1024
• total_eval_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 150
• training_steps: 2000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
133.1257	0.05	100	7.7065	0.0876
107.0186	0.1	200	6.6099	0.1364
102.3462	0.15	300	6.3677	0.1448
99.0524	0.2	400	6.2145	0.1531
96.6031	0.25	500	6.1010	0.1584
95.9314	0.3	600	5.9869	0.1640
95.0101	0.35	700	5.8890	0.1683
94.2643	0.4	800	5.8284	0.1715
93.2266	0.45	900	5.7744	0.1746
92.0125	0.5	1000	5.7337	0.1763
95.8613	0.55	1100	6.0068	0.1554
98.1012	0.6	1200	6.1055	0.1491
97.152	0.65	1300	6.0546	0.1512
97.8734	0.7	1400	6.0947	0.1474
97.4964	0.75	1500	6.0419	0.1510
97.562	0.8	1600	6.0272	0.1518
97.3206	0.85	1700	6.0171	0.1529
96.8684	0.9	1800	6.0163	0.1532
96.472	0.95	1900	6.0193	0.1528
96.5051	1.0	2000	6.0191	0.1528

Framework versions

• Transformers 4.56.0
• Pytorch 2.8.0+cu129
• Datasets 5.0.0
• Tokenizers 0.22.0