iris-olmo-2-1b-iris-only-k10

This model is a fine-tuned version of allenai/OLMo-2-0425-1B-Instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7921
• Model Preparation Time: 0.0119
• Soft Mae: 0.0623
• Soft Brier: 0.0130
• Student Prelevantmean: 0.2277
• Teacher Prelevantmean: 0.2340
• Bin F1: 0.8475
• Cal Ece: 0.0902
• Cal Brier: 0.0686
• Cal Auroc: 0.9816
• Info Ndcg@p8: 0.9445
• Info Pairwiseacc: 0.9066
• Num Questions: 30
• Num Pairs: 240

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Use paged_adamw_8bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: constant
• lr_scheduler_warmup_steps: 0.03
• num_epochs: 6.0

Training results

Training Loss	Epoch	Step	Validation Loss	Model Preparation Time	Soft Mae	Soft Brier	Student Prelevantmean	Teacher Prelevantmean	Bin F1	Cal Ece	Cal Brier	Cal Auroc	Info Ndcg@p8	Info Pairwiseacc	Num Questions	Num Pairs
1.3100	0.2519	34	1.0794	0.0119	0.1967	0.0657	0.3548	0.2340	0.7467	0.1511	0.1342	0.9104	0.8792	0.8162	30	240
0.9093	0.5037	68	0.9672	0.0119	0.1127	0.0366	0.1661	0.2340	0.6392	0.1047	0.1026	0.9553	0.9256	0.8601	30	240
1.0392	0.7556	102	0.7359	0.0119	0.1022	0.0232	0.2672	0.2340	0.8429	0.1137	0.0854	0.9742	0.9457	0.8856	30	240
0.8128	1.0074	136	0.6466	0.0119	0.0730	0.0188	0.2275	0.2340	0.8710	0.0890	0.0721	0.9717	0.9515	0.8866	30	240
0.7664	1.2593	170	0.7198	0.0119	0.0776	0.0179	0.2097	0.2340	0.7679	0.0992	0.0834	0.9700	0.9504	0.8996	30	240
0.7041	1.5111	204	0.6885	0.0119	0.0704	0.0176	0.1959	0.2340	0.8142	0.0877	0.0742	0.9743	0.9464	0.8863	30	240
0.6613	1.7630	238	0.7089	0.0119	0.0666	0.0184	0.2253	0.2340	0.8397	0.0654	0.0698	0.9705	0.9431	0.8840	30	240
0.6163	2.0148	272	0.7168	0.0119	0.0755	0.0196	0.2358	0.2340	0.8777	0.1051	0.0810	0.9780	0.9446	0.8881	30	240
0.6721	2.2667	306	0.6785	0.0119	0.0662	0.0148	0.2048	0.2340	0.8293	0.0958	0.0720	0.9787	0.9457	0.8979	30	240
0.4474	2.5185	340	0.6788	0.0119	0.0604	0.0140	0.2445	0.2340	0.8939	0.0893	0.0625	0.9757	0.9451	0.8945	30	240
0.5002	2.7704	374	0.7259	0.0119	0.0693	0.0173	0.1987	0.2340	0.7568	0.0912	0.0751	0.9757	0.9413	0.9134	30	240
0.4219	3.0222	408	0.7544	0.0119	0.0649	0.0163	0.2121	0.2340	0.8730	0.0907	0.0722	0.9787	0.9471	0.9037	30	240
0.3581	3.2741	442	0.7818	0.0119	0.0614	0.0151	0.2021	0.2340	0.8	0.0822	0.0700	0.9779	0.9452	0.8992	30	240
0.3399	3.5259	476	0.8489	0.0119	0.0658	0.0145	0.2047	0.2340	0.7748	0.1002	0.0738	0.9785	0.9454	0.9051	30	240
0.3193	3.7778	510	0.7742	0.0119	0.0600	0.0141	0.2219	0.2340	0.875	0.0847	0.0669	0.976	0.9446	0.8945	30	240
0.4288	4.0296	544	0.9937	0.0119	0.0675	0.0156	0.1968	0.2340	0.8205	0.0972	0.0764	0.9788	0.9406	0.8906	30	240
0.2931	4.2815	578	0.7975	0.0119	0.0597	0.0132	0.2224	0.2340	0.8293	0.0891	0.0703	0.9776	0.9454	0.8964	30	240
0.3902	4.5333	612	0.8645	0.0119	0.0708	0.0149	0.2143	0.2340	0.7429	0.1280	0.0779	0.9813	0.9450	0.8904	30	240
0.3831	4.7852	646	0.7459	0.0119	0.0596	0.0140	0.2342	0.2340	0.875	0.0685	0.0622	0.9776	0.9428	0.8988	30	240
0.2541	5.0370	680	0.8288	0.0119	0.0618	0.0122	0.2358	0.2340	0.8730	0.0875	0.0645	0.9779	0.9427	0.9034	30	240
0.2705	5.2889	714	0.8203	0.0119	0.0571	0.0121	0.2137	0.2340	0.85	0.1022	0.0700	0.9798	0.9414	0.8933	30	240
0.2430	5.5407	748	0.7542	0.0119	0.0560	0.0121	0.2373	0.2340	0.8837	0.0851	0.0654	0.9777	0.9400	0.8984	30	240
0.2547	5.7926	782	0.7933	0.0119	0.0592	0.0132	0.2104	0.2340	0.8403	0.1075	0.0690	0.9811	0.9415	0.9012	30	240
0.2831	6.0	810	0.7921	0.0119	0.0623	0.0130	0.2277	0.2340	0.8475	0.0902	0.0686	0.9816	0.9445	0.9066	30	240

Framework versions

• PEFT 0.19.1
• Transformers 5.14.1
• Pytorch 2.5.1+cu124
• Datasets 5.0.0
• Tokenizers 0.22.2