iati-climate-multi-classifier-weighted2

This model is a fine-tuned version of alex-miller/ODABert on a subset of the alex-miller/iati-policy-markers dataset.

It achieves the following results on the evaluation set:

• Loss: 0.7080
• Accuracy: 0.8541
• F1: 0.7121
• Precision: 0.6265
• Recall: 0.8248

Model description

This model has been trained to identify both significant and principal climate mitigation and climate adaptation project titles and/or descriptions.

Intended uses & limitations

As many of the donors in the training dataset have mixed up Adaptation and Mitigation, the model's ability to differentiate the two isn't perfect. But the sigmoid of the model logits do bias toward the correct class.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-06
• train_batch_size: 24
• eval_batch_size: 24
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Accuracy	F1	Validation Loss	Precision	Recall
0.7689	1.0	1951	0.7993	0.6421	0.6477	0.5264	0.8230
0.6217	2.0	3902	0.8303	0.6737	0.6269	0.5814	0.8010
0.5834	3.0	5853	0.8266	0.6761	0.6101	0.5715	0.8276
0.5571	4.0	7804	0.8461	0.6933	0.6169	0.6144	0.7954
0.5323	5.0	9755	0.8366	0.6869	0.6050	0.5913	0.8194
0.5126	6.0	11706	0.8327	0.6867	0.6047	0.5815	0.8385
0.4968	7.0	13657	0.8408	0.6938	0.6098	0.5989	0.8244
0.4893	8.0	15608	0.6040	0.8348	0.6895	0.5854	0.8387
0.4702	9.0	17559	0.6342	0.8508	0.7050	0.6211	0.8151
0.4514	10.0	19510	0.6210	0.8383	0.6946	0.5918	0.8404
0.4323	11.0	21461	0.6340	0.8402	0.6991	0.5943	0.8487
0.4193	12.0	23412	0.6407	0.8433	0.7005	0.6020	0.8375
0.407	13.0	25363	0.6602	0.8526	0.7094	0.6237	0.8223
0.3944	14.0	27314	0.6588	0.8441	0.7026	0.6029	0.8419
0.3834	15.0	29265	0.6881	0.8529	0.7110	0.6233	0.8274
0.3738	16.0	31216	0.7029	0.8575	0.7146	0.6359	0.8155
0.3686	17.0	33167	0.6929	0.8524	0.7102	0.6224	0.8271
0.3607	18.0	35118	0.7069	0.8545	0.7127	0.6272	0.8253
0.3556	19.0	37069	0.7072	0.8543	0.7118	0.6274	0.8225
0.3523	20.0	39020	0.7080	0.8541	0.7121	0.6265	0.8248

Framework versions

• Transformers 4.41.0
• Pytorch 2.0.1
• Datasets 2.19.1
• Tokenizers 0.19.1