distilbert-base-uncased-finetuned-cola

This model is a fine-tuned version of veriga/distilbert-base-uncased-finetuned-cola on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.8100
• Accuracy: 0.9556
• F1: 0.0017

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	37	3.7942	0.9556	0.0017
No log	2.0	74	3.7810	0.9556	0.0
No log	3.0	111	3.8160	0.9557	0.0026
No log	4.0	148	3.8256	0.9557	0.0026
No log	5.0	185	3.7971	0.9558	0.0052
No log	6.0	222	3.8099	0.9557	0.0026
No log	7.0	259	3.7882	0.9556	0.0009
No log	8.0	296	3.8088	0.9556	0.0017
No log	9.0	333	3.8060	0.9557	0.0026
No log	10.0	370	3.8122	0.9557	0.0034
No log	11.0	407	3.8023	0.9556	0.0
No log	12.0	444	3.7889	0.9556	0.0017
No log	13.0	481	3.8140	0.9556	0.0
3.8738	14.0	518	3.8300	0.9556	0.0009
3.8738	15.0	555	3.8003	0.9556	0.0
3.8738	16.0	592	3.8238	0.9556	0.0017
3.8738	17.0	629	3.7945	0.9556	0.0
3.8738	18.0	666	3.8228	0.9557	0.0043
3.8738	19.0	703	3.8044	0.9556	0.0
3.8738	20.0	740	3.8049	0.9556	0.0017
3.8738	21.0	777	3.8031	0.9556	0.0017
3.8738	22.0	814	3.8092	0.9556	0.0017
3.8738	23.0	851	3.8090	0.9559	0.0086
3.8738	24.0	888	3.7972	0.9556	0.0
3.8738	25.0	925	3.8085	0.9556	0.0
3.8738	26.0	962	3.7968	0.9557	0.0043
3.8738	27.0	999	3.8240	0.9557	0.0026
3.8886	28.0	1036	3.7986	0.9556	0.0009
3.8886	29.0	1073	3.7939	0.9556	0.0009
3.8886	30.0	1110	3.8051	0.9557	0.0026
3.8886	31.0	1147	3.8014	0.9556	0.0009
3.8886	32.0	1184	3.8082	0.9556	0.0017
3.8886	33.0	1221	3.8032	0.9557	0.0026
3.8886	34.0	1258	3.8092	0.9557	0.0026
3.8886	35.0	1295	3.7875	0.9557	0.0026
3.8886	36.0	1332	3.8107	0.9557	0.0026
3.8886	37.0	1369	3.7927	0.9556	0.0009
3.8886	38.0	1406	3.7875	0.9556	0.0017
3.8886	39.0	1443	3.8008	0.9556	0.0017
3.8886	40.0	1480	3.7967	0.9556	0.0017
3.8388	41.0	1517	3.8110	0.9564	0.0197
3.8388	42.0	1554	3.8134	0.9557	0.0026
3.8388	43.0	1591	3.8047	0.9556	0.0017
3.8388	44.0	1628	3.8008	0.9556	0.0017
3.8388	45.0	1665	3.8170	0.9557	0.0026
3.8388	46.0	1702	3.8085	0.9556	0.0017
3.8388	47.0	1739	3.8142	0.9556	0.0017
3.8388	48.0	1776	3.8048	0.9557	0.0034
3.8388	49.0	1813	3.8121	0.9557	0.0034
3.8388	50.0	1850	3.7905	0.9556	0.0
3.8388	51.0	1887	3.7977	0.9556	0.0017
3.8388	52.0	1924	3.8083	0.9564	0.0197
3.8388	53.0	1961	3.7955	0.9557	0.0026
3.8388	54.0	1998	3.8105	0.9564	0.0197
3.858	55.0	2035	3.8086	0.9557	0.0043
3.858	56.0	2072	3.8025	0.9557	0.0034
3.858	57.0	2109	3.7983	0.9556	0.0009
3.858	58.0	2146	3.8122	0.9564	0.0197
3.858	59.0	2183	3.8152	0.9557	0.0043
3.858	60.0	2220	3.8014	0.9556	0.0017
3.858	61.0	2257	3.8000	0.9556	0.0017
3.858	62.0	2294	3.8049	0.9556	0.0017
3.858	63.0	2331	3.7995	0.9556	0.0017
3.858	64.0	2368	3.8088	0.9556	0.0017
3.858	65.0	2405	3.8087	0.9556	0.0009
3.858	66.0	2442	3.8093	0.9557	0.0043
3.858	67.0	2479	3.8257	0.9564	0.0197
3.8513	68.0	2516	3.8136	0.9556	0.0009
3.8513	69.0	2553	3.8211	0.9564	0.0197
3.8513	70.0	2590	3.8055	0.9556	0.0
3.8513	71.0	2627	3.8070	0.9556	0.0
3.8513	72.0	2664	3.8054	0.9556	0.0
3.8513	73.0	2701	3.7996	0.9556	0.0009
3.8513	74.0	2738	3.8220	0.9564	0.0197
3.8513	75.0	2775	3.8093	0.9564	0.0197
3.8513	76.0	2812	3.8100	0.9564	0.0197
3.8513	77.0	2849	3.8065	0.9564	0.0197
3.8513	78.0	2886	3.8056	0.9556	0.0017
3.8513	79.0	2923	3.8060	0.9564	0.0197
3.8513	80.0	2960	3.8124	0.9564	0.0197
3.8513	81.0	2997	3.8047	0.9564	0.0197
3.8615	82.0	3034	3.8096	0.9557	0.0043
3.8615	83.0	3071	3.8096	0.9556	0.0009
3.8615	84.0	3108	3.8052	0.9556	0.0009
3.8615	85.0	3145	3.8035	0.9556	0.0009
3.8615	86.0	3182	3.8077	0.9556	0.0017
3.8615	87.0	3219	3.8029	0.9556	0.0017
3.8615	88.0	3256	3.8057	0.9556	0.0017
3.8615	89.0	3293	3.8038	0.9556	0.0017
3.8615	90.0	3330	3.8043	0.9556	0.0017
3.8615	91.0	3367	3.8036	0.9556	0.0017
3.8615	92.0	3404	3.8076	0.9556	0.0017
3.8615	93.0	3441	3.8103	0.9556	0.0017
3.8615	94.0	3478	3.8104	0.9556	0.0017
3.8533	95.0	3515	3.8106	0.9556	0.0017
3.8533	96.0	3552	3.8089	0.9556	0.0017
3.8533	97.0	3589	3.8098	0.9556	0.0017
3.8533	98.0	3626	3.8097	0.9556	0.0017
3.8533	99.0	3663	3.8100	0.9556	0.0017
3.8533	100.0	3700	3.8100	0.9556	0.0017

Framework versions

• Transformers 4.36.2
• Pytorch 1.14.0.dev20221204+cu117
• Datasets 2.2.2
• Tokenizers 0.15.0