08c97241ffc4d5a52943c1de619b86de

This model is a fine-tuned version of facebook/opt-6.7b on the contemmcm/trec dataset. It achieves the following results on the evaluation set:

• Loss: 0.6202
• Data Size: 1.0
• Epoch Runtime: 79.4765
• Accuracy: 0.8688
• F1 Macro: 0.8742

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Data Size	Epoch Runtime	Accuracy	F1 Macro
No log	0	0	2.1073	0	1.8039	0.1729	0.1185
No log	1	170	1.9540	0.0078	2.6466	0.3521	0.2567
No log	2	340	1.3270	0.0156	10.2601	0.6042	0.4581
No log	3	510	1.0694	0.0312	20.4705	0.6229	0.6052
No log	4	680	0.9863	0.0625	30.8447	0.5958	0.6025
0.0617	5	850	0.3352	0.125	46.2403	0.9021	0.8975
0.0617	6	1020	0.3908	0.25	63.0416	0.8896	0.8657
0.5455	7	1190	0.4489	0.5	51.3985	0.8729	0.8400
0.3711	8.0	1360	0.3894	1.0	84.7811	0.9062	0.8338
0.3905	9.0	1530	0.2843	1.0	92.5283	0.9146	0.9062
0.2889	10.0	1700	0.4974	1.0	76.5005	0.8729	0.8710
0.2083	11.0	1870	0.7143	1.0	90.5865	0.8792	0.8561
0.1796	12.0	2040	0.3693	1.0	93.0467	0.9208	0.8995
0.1077	13.0	2210	0.6202	1.0	79.4765	0.8688	0.8742

Framework versions

• Transformers 4.57.0
• Pytorch 2.8.0+cu128
• Datasets 4.3.0
• Tokenizers 0.22.1