layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.9059
• Answer: {'precision': 0.7288693743139407, 'recall': 0.8207663782447466, 'f1': 0.772093023255814, 'number': 809}
• Header: {'precision': 0.43795620437956206, 'recall': 0.5042016806722689, 'f1': 0.46875, 'number': 119}
• Question: {'precision': 0.8137614678899082, 'recall': 0.8328638497652582, 'f1': 0.8232018561484918, 'number': 1065}
• Overall Precision: 0.7535
• Overall Recall: 0.8083
• Overall F1: 0.7800
• Overall Accuracy: 0.8069

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8292	1.0	10	1.5818	{'precision': 0.00816326530612245, 'recall': 0.007416563658838072, 'f1': 0.007772020725388602, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.22530864197530864, 'recall': 0.13708920187793427, 'f1': 0.17046117921774664, 'number': 1065}	0.1099	0.0763	0.0900	0.3514
1.4621	2.0	20	1.2540	{'precision': 0.2941659819227609, 'recall': 0.44252163164400493, 'f1': 0.3534057255676209, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.391804457225018, 'recall': 0.5117370892018779, 'f1': 0.44381107491856675, 'number': 1065}	0.3451	0.4531	0.3918	0.6061
1.0998	3.0	30	0.9167	{'precision': 0.5, 'recall': 0.6254635352286774, 'f1': 0.5557386051619989, 'number': 809}	{'precision': 0.058823529411764705, 'recall': 0.025210084033613446, 'f1': 0.03529411764705882, 'number': 119}	{'precision': 0.5253118121790169, 'recall': 0.672300469483568, 'f1': 0.5897858319604613, 'number': 1065}	0.5049	0.6147	0.5544	0.7112
0.8275	4.0	40	0.7689	{'precision': 0.5742667928098392, 'recall': 0.7503090234857849, 'f1': 0.6505894962486602, 'number': 809}	{'precision': 0.2894736842105263, 'recall': 0.18487394957983194, 'f1': 0.22564102564102564, 'number': 119}	{'precision': 0.6544315129811996, 'recall': 0.6863849765258216, 'f1': 0.6700274977085243, 'number': 1065}	0.6044	0.6824	0.6411	0.7610
0.6622	5.0	50	0.7165	{'precision': 0.6455424274973147, 'recall': 0.7428924598269468, 'f1': 0.6908045977011494, 'number': 809}	{'precision': 0.36363636363636365, 'recall': 0.23529411764705882, 'f1': 0.2857142857142857, 'number': 119}	{'precision': 0.6690647482014388, 'recall': 0.7859154929577464, 'f1': 0.7227979274611398, 'number': 1065}	0.6490	0.7356	0.6896	0.7792
0.5529	6.0	60	0.6745	{'precision': 0.6523955147808359, 'recall': 0.7911001236093943, 'f1': 0.7150837988826816, 'number': 809}	{'precision': 0.3473684210526316, 'recall': 0.2773109243697479, 'f1': 0.308411214953271, 'number': 119}	{'precision': 0.726649528706084, 'recall': 0.7962441314553991, 'f1': 0.7598566308243728, 'number': 1065}	0.6781	0.7632	0.7181	0.7876
0.4607	7.0	70	0.6568	{'precision': 0.6858359957401491, 'recall': 0.796044499381953, 'f1': 0.7368421052631579, 'number': 809}	{'precision': 0.3490566037735849, 'recall': 0.31092436974789917, 'f1': 0.32888888888888884, 'number': 119}	{'precision': 0.7552447552447552, 'recall': 0.8112676056338028, 'f1': 0.7822544137618832, 'number': 1065}	0.7058	0.7752	0.7389	0.7954
0.398	8.0	80	0.6752	{'precision': 0.6826722338204593, 'recall': 0.8084054388133498, 'f1': 0.7402376910016978, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.31932773109243695, 'f1': 0.3261802575107296, 'number': 119}	{'precision': 0.7607361963190185, 'recall': 0.8150234741784037, 'f1': 0.786944696282865, 'number': 1065}	0.7049	0.7827	0.7418	0.7957
0.3405	9.0	90	0.6934	{'precision': 0.6829015544041451, 'recall': 0.8145859085290482, 'f1': 0.7429537767756483, 'number': 809}	{'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119}	{'precision': 0.7694369973190348, 'recall': 0.8084507042253521, 'f1': 0.7884615384615383, 'number': 1065}	0.7072	0.7842	0.7438	0.7965
0.3225	10.0	100	0.7087	{'precision': 0.6884899683210137, 'recall': 0.8059332509270705, 'f1': 0.7425968109339408, 'number': 809}	{'precision': 0.35714285714285715, 'recall': 0.33613445378151263, 'f1': 0.34632034632034636, 'number': 119}	{'precision': 0.7785714285714286, 'recall': 0.8187793427230047, 'f1': 0.7981693363844393, 'number': 1065}	0.7178	0.7847	0.7498	0.7999
0.2555	11.0	110	0.7154	{'precision': 0.7044967880085653, 'recall': 0.8133498145859085, 'f1': 0.7550200803212851, 'number': 809}	{'precision': 0.40384615384615385, 'recall': 0.35294117647058826, 'f1': 0.3766816143497759, 'number': 119}	{'precision': 0.7887197851387645, 'recall': 0.8272300469483568, 'f1': 0.8075160403299725, 'number': 1065}	0.7336	0.7933	0.7623	0.8047
0.2238	12.0	120	0.7295	{'precision': 0.7291666666666666, 'recall': 0.8220024721878862, 'f1': 0.7728065078442764, 'number': 809}	{'precision': 0.3793103448275862, 'recall': 0.3697478991596639, 'f1': 0.374468085106383, 'number': 119}	{'precision': 0.7824194952132288, 'recall': 0.844131455399061, 'f1': 0.8121047877145439, 'number': 1065}	0.7386	0.8068	0.7712	0.8082
0.198	13.0	130	0.7615	{'precision': 0.7092651757188498, 'recall': 0.823238566131026, 'f1': 0.7620137299771167, 'number': 809}	{'precision': 0.41346153846153844, 'recall': 0.36134453781512604, 'f1': 0.3856502242152467, 'number': 119}	{'precision': 0.8021680216802168, 'recall': 0.8338028169014085, 'f1': 0.8176795580110497, 'number': 1065}	0.7428	0.8013	0.7709	0.8060
0.1691	14.0	140	0.7624	{'precision': 0.7232432432432433, 'recall': 0.826946847960445, 'f1': 0.7716262975778547, 'number': 809}	{'precision': 0.4090909090909091, 'recall': 0.37815126050420167, 'f1': 0.39301310043668125, 'number': 119}	{'precision': 0.7978436657681941, 'recall': 0.8338028169014085, 'f1': 0.8154269972451792, 'number': 1065}	0.7458	0.8038	0.7737	0.8108
0.1504	15.0	150	0.7685	{'precision': 0.7197802197802198, 'recall': 0.8096415327564895, 'f1': 0.7620709714950552, 'number': 809}	{'precision': 0.41228070175438597, 'recall': 0.3949579831932773, 'f1': 0.40343347639484983, 'number': 119}	{'precision': 0.8028802880288028, 'recall': 0.8375586854460094, 'f1': 0.8198529411764706, 'number': 1065}	0.7466	0.7998	0.7723	0.8104
0.1387	16.0	160	0.8119	{'precision': 0.7063740856844305, 'recall': 0.8355995055624228, 'f1': 0.7655719139297849, 'number': 809}	{'precision': 0.38405797101449274, 'recall': 0.44537815126050423, 'f1': 0.41245136186770426, 'number': 119}	{'precision': 0.7994530537830447, 'recall': 0.8234741784037559, 'f1': 0.8112858464384829, 'number': 1065}	0.7327	0.8058	0.7675	0.8045
0.1289	17.0	170	0.8040	{'precision': 0.7145922746781116, 'recall': 0.823238566131026, 'f1': 0.7650775416427341, 'number': 809}	{'precision': 0.4236111111111111, 'recall': 0.5126050420168067, 'f1': 0.4638783269961977, 'number': 119}	{'precision': 0.819718309859155, 'recall': 0.819718309859155, 'f1': 0.819718309859155, 'number': 1065}	0.7473	0.8028	0.7741	0.8078
0.1113	18.0	180	0.8194	{'precision': 0.732662192393736, 'recall': 0.8096415327564895, 'f1': 0.7692307692307693, 'number': 809}	{'precision': 0.4233576642335766, 'recall': 0.48739495798319327, 'f1': 0.453125, 'number': 119}	{'precision': 0.8119891008174387, 'recall': 0.8394366197183099, 'f1': 0.8254847645429363, 'number': 1065}	0.7538	0.8063	0.7792	0.8082
0.1034	19.0	190	0.8405	{'precision': 0.721205597416577, 'recall': 0.8281829419035847, 'f1': 0.7710011507479863, 'number': 809}	{'precision': 0.4153846153846154, 'recall': 0.453781512605042, 'f1': 0.43373493975903615, 'number': 119}	{'precision': 0.8083941605839416, 'recall': 0.831924882629108, 'f1': 0.8199907450254512, 'number': 1065}	0.7471	0.8078	0.7763	0.8076
0.0948	20.0	200	0.8530	{'precision': 0.7199124726477024, 'recall': 0.8133498145859085, 'f1': 0.7637840975043529, 'number': 809}	{'precision': 0.4117647058823529, 'recall': 0.47058823529411764, 'f1': 0.4392156862745098, 'number': 119}	{'precision': 0.8070333633904418, 'recall': 0.8403755868544601, 'f1': 0.8233670653173873, 'number': 1065}	0.7453	0.8073	0.7750	0.8092
0.0868	21.0	210	0.8617	{'precision': 0.7291666666666666, 'recall': 0.8220024721878862, 'f1': 0.7728065078442764, 'number': 809}	{'precision': 0.4198473282442748, 'recall': 0.46218487394957986, 'f1': 0.43999999999999995, 'number': 119}	{'precision': 0.8148487626031164, 'recall': 0.8347417840375587, 'f1': 0.8246753246753246, 'number': 1065}	0.7540	0.8073	0.7797	0.8083
0.0905	22.0	220	0.8748	{'precision': 0.7333333333333333, 'recall': 0.8158220024721878, 'f1': 0.7723815096547689, 'number': 809}	{'precision': 0.39215686274509803, 'recall': 0.5042016806722689, 'f1': 0.4411764705882353, 'number': 119}	{'precision': 0.813126709206928, 'recall': 0.8375586854460094, 'f1': 0.8251618871415356, 'number': 1065}	0.7498	0.8088	0.7782	0.8064
0.0809	23.0	230	0.8749	{'precision': 0.724025974025974, 'recall': 0.826946847960445, 'f1': 0.7720715522215811, 'number': 809}	{'precision': 0.44274809160305345, 'recall': 0.48739495798319327, 'f1': 0.464, 'number': 119}	{'precision': 0.8132474701011959, 'recall': 0.8300469483568075, 'f1': 0.8215613382899627, 'number': 1065}	0.7521	0.8083	0.7792	0.8087
0.073	24.0	240	0.8752	{'precision': 0.7290748898678414, 'recall': 0.8182941903584673, 'f1': 0.7711124053581829, 'number': 809}	{'precision': 0.43703703703703706, 'recall': 0.4957983193277311, 'f1': 0.4645669291338583, 'number': 119}	{'precision': 0.8156934306569343, 'recall': 0.8394366197183099, 'f1': 0.8273947246645073, 'number': 1065}	0.7550	0.8103	0.7817	0.8090
0.0694	25.0	250	0.8898	{'precision': 0.723986856516977, 'recall': 0.8170580964153276, 'f1': 0.7677119628339142, 'number': 809}	{'precision': 0.427536231884058, 'recall': 0.4957983193277311, 'f1': 0.4591439688715953, 'number': 119}	{'precision': 0.8108356290174472, 'recall': 0.8291079812206573, 'f1': 0.819870009285051, 'number': 1065}	0.7491	0.8043	0.7757	0.8070
0.0726	26.0	260	0.8944	{'precision': 0.7213656387665198, 'recall': 0.8096415327564895, 'f1': 0.762958648806057, 'number': 809}	{'precision': 0.43703703703703706, 'recall': 0.4957983193277311, 'f1': 0.4645669291338583, 'number': 119}	{'precision': 0.8170173833485819, 'recall': 0.8384976525821596, 'f1': 0.8276181649675627, 'number': 1065}	0.7523	0.8063	0.7784	0.8082
0.0674	27.0	270	0.9073	{'precision': 0.7337733773377337, 'recall': 0.8244746600741656, 'f1': 0.7764842840512223, 'number': 809}	{'precision': 0.41304347826086957, 'recall': 0.4789915966386555, 'f1': 0.443579766536965, 'number': 119}	{'precision': 0.8118721461187215, 'recall': 0.8347417840375587, 'f1': 0.8231481481481482, 'number': 1065}	0.7530	0.8093	0.7802	0.8057
0.0701	28.0	280	0.9131	{'precision': 0.7257889009793254, 'recall': 0.8244746600741656, 'f1': 0.7719907407407407, 'number': 809}	{'precision': 0.4195804195804196, 'recall': 0.5042016806722689, 'f1': 0.4580152671755725, 'number': 119}	{'precision': 0.8141674333026679, 'recall': 0.8309859154929577, 'f1': 0.8224907063197026, 'number': 1065}	0.7501	0.8088	0.7784	0.8055
0.0657	29.0	290	0.9060	{'precision': 0.7304730473047305, 'recall': 0.8207663782447466, 'f1': 0.7729918509895226, 'number': 809}	{'precision': 0.44029850746268656, 'recall': 0.4957983193277311, 'f1': 0.46640316205533594, 'number': 119}	{'precision': 0.8119266055045872, 'recall': 0.8309859154929577, 'f1': 0.8213457076566124, 'number': 1065}	0.7539	0.8068	0.7794	0.8075
0.0641	30.0	300	0.9059	{'precision': 0.7288693743139407, 'recall': 0.8207663782447466, 'f1': 0.772093023255814, 'number': 809}	{'precision': 0.43795620437956206, 'recall': 0.5042016806722689, 'f1': 0.46875, 'number': 119}	{'precision': 0.8137614678899082, 'recall': 0.8328638497652582, 'f1': 0.8232018561484918, 'number': 1065}	0.7535	0.8083	0.7800	0.8069

Framework versions

• Transformers 4.47.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0