wav2vec2-tamazigh-tifinagh-test2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 11.9837
• Wer: 1.0
• Cer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7e-06
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• total_train_batch_size: 64
• total_eval_batch_size: 64
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
No log	1.0	2	19.0231	1.0	2.2099
No log	2.0	4	18.9618	1.0	2.1657
19.9962	3.0	6	18.8997	1.0	2.0552
19.9962	4.0	8	18.8410	1.0	2.0331
19.683	5.0	10	18.7896	1.0	1.9669
19.683	6.0	12	18.7418	1.0588	1.8453
19.683	7.0	14	18.6825	1.0	1.8619
19.0417	8.0	16	18.6296	1.1471	1.8785
19.0417	9.0	18	18.5717	1.4118	1.9448
19.5491	10.0	20	18.5348	1.7941	1.9724
19.5491	11.0	22	18.4854	2.0294	1.7569
19.5491	12.0	24	18.4295	2.0882	1.5856
19.0779	13.0	26	18.3714	2.2647	1.3481
19.0779	14.0	28	18.3173	2.0	1.1050
19.1881	15.0	30	18.2664	1.4706	0.9834
19.1881	16.0	32	18.2050	1.1765	0.9724
19.1881	17.0	34	18.1427	1.0588	0.9834
18.5754	18.0	36	18.0752	1.0	1.0
18.5754	19.0	38	18.0088	1.0	1.0
18.6881	20.0	40	17.9344	1.0	1.0
18.6881	21.0	42	17.8458	1.0	1.0
18.6881	22.0	44	17.7479	1.0	1.0
18.7565	23.0	46	17.6437	1.0	1.0
18.7565	24.0	48	17.5387	1.0	1.0
18.0348	25.0	50	17.4174	1.0	1.0
18.0348	26.0	52	17.3023	1.0	1.0
18.0348	27.0	54	17.1534	1.0	1.0
18.2158	28.0	56	17.0013	1.0	1.0
18.2158	29.0	58	16.8349	1.0	1.0
18.3454	30.0	60	16.6536	1.0	1.0
18.3454	31.0	62	16.4526	1.0	1.0
18.3454	32.0	64	16.2427	1.0	1.0
17.5257	33.0	66	16.0118	1.0	1.0
17.5257	34.0	68	15.7621	1.0	1.0
16.8069	35.0	70	15.5254	1.0	1.0
16.8069	36.0	72	15.3054	1.0	1.0
16.8069	37.0	74	15.0979	1.0	1.0
17.1611	38.0	76	14.9124	1.0	1.0
17.1611	39.0	78	14.7198	1.0	1.0
16.7116	40.0	80	14.5696	1.0	1.0
16.7116	41.0	82	14.4247	1.0	1.0
16.7116	42.0	84	14.2999	1.0	1.0
16.075	43.0	86	14.1993	1.0	1.0
16.075	44.0	88	14.0855	1.0	1.0
16.2366	45.0	90	13.9871	1.0	1.0
16.2366	46.0	92	13.9344	1.0	1.0
16.2366	47.0	94	13.8442	1.0	1.0
16.9955	48.0	96	13.7309	1.0	1.0
16.9955	49.0	98	13.6605	1.0	1.0
16.0942	50.0	100	13.5664	1.0	1.0
16.0942	51.0	102	13.5030	1.0	1.0
16.0942	52.0	104	13.4367	1.0	1.0
16.2248	53.0	106	13.3663	1.0	1.0
16.2248	54.0	108	13.2937	1.0	1.0
14.9747	55.0	110	13.2110	1.0	1.0
14.9747	56.0	112	13.1368	1.0	1.0
14.9747	57.0	114	13.0670	1.0	1.0
15.005	58.0	116	13.0016	1.0	1.0
15.005	59.0	118	12.9669	1.0	1.0
16.0239	60.0	120	12.9238	1.0	1.0
16.0239	61.0	122	12.8743	1.0	1.0
16.0239	62.0	124	12.8279	1.0	1.0
15.2303	63.0	126	12.7776	1.0	1.0
15.2303	64.0	128	12.7267	1.0	1.0
14.5867	65.0	130	12.6776	1.0	1.0
14.5867	66.0	132	12.6363	1.0	1.0
14.5867	67.0	134	12.6001	1.0	1.0
16.022	68.0	136	12.5613	1.0	1.0
16.022	69.0	138	12.5252	1.0	1.0
14.743	70.0	140	12.4918	1.0	1.0
14.743	71.0	142	12.4586	1.0	1.0
14.743	72.0	144	12.4280	1.0	1.0
15.2858	73.0	146	12.3960	1.0	1.0
15.2858	74.0	148	12.3630	1.0	1.0
13.7562	75.0	150	12.3318	1.0	1.0
13.7562	76.0	152	12.3014	1.0	1.0
13.7562	77.0	154	12.2739	1.0	1.0
13.6653	78.0	156	12.2473	1.0	1.0
13.6653	79.0	158	12.2232	1.0	1.0
14.5475	80.0	160	12.2013	1.0	1.0
14.5475	81.0	162	12.1812	1.0	1.0
14.5475	82.0	164	12.1590	1.0	1.0
13.8149	83.0	166	12.1395	1.0	1.0
13.8149	84.0	168	12.1212	1.0	1.0
14.0592	85.0	170	12.1042	1.0	1.0
14.0592	86.0	172	12.0901	1.0	1.0
14.0592	87.0	174	12.0770	1.0	1.0
14.9469	88.0	176	12.0636	1.0	1.0
14.9469	89.0	178	12.0505	1.0	1.0
13.8567	90.0	180	12.0382	1.0	1.0
13.8567	91.0	182	12.0272	1.0	1.0
13.8567	92.0	184	12.0174	1.0	1.0
13.8197	93.0	186	12.0102	1.0	1.0
13.8197	94.0	188	12.0044	1.0	1.0
14.5581	95.0	190	11.9984	1.0	1.0
14.5581	96.0	192	11.9939	1.0	1.0
14.5581	97.0	194	11.9898	1.0	1.0
14.5627	98.0	196	11.9866	1.0	1.0
14.5627	99.0	198	11.9845	1.0	1.0
14.1269	100.0	200	11.9837	1.0	1.0

Framework versions

• Transformers 4.48.3
• Pytorch 2.1.0+cu121
• Datasets 2.21.0
• Tokenizers 0.21.2