w2v-bert-phoneme-en

This model is a fine-tuned version of facebook/w2v-bert-2.0 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0733
• Per: 0.9993

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1318
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Per
2.8417	0.1138	500	0.1596	0.9993
2.3378	0.2276	1000	0.1446	0.9989
1.9937	0.3414	1500	0.1181	0.9989
1.9339	0.4552	2000	0.1100	0.9993
1.8203	0.5690	2500	0.1001	0.9993
1.7333	0.6828	3000	0.1014	0.9993
1.7046	0.7966	3500	0.0942	0.9993
1.6405	0.9104	4000	0.0920	0.9989
1.4132	1.0241	4500	0.0880	0.9985
1.4147	1.1379	5000	0.0869	0.9989
1.3652	1.2517	5500	0.0869	0.9993
1.3726	1.3655	6000	0.0892	0.9989
1.2921	1.4793	6500	0.0901	0.9989
1.3118	1.5931	7000	0.0807	0.9993
1.2855	1.7069	7500	0.0766	0.9985
1.2413	1.8207	8000	0.0786	0.9989
1.2402	1.9345	8500	0.0755	0.9989
1.0501	2.0483	9000	0.0835	0.9993
1.0819	2.1621	9500	0.0776	0.9989
1.0774	2.2759	10000	0.0829	0.9993
1.0551	2.3897	10500	0.0745	0.9993
1.0503	2.5035	11000	0.0745	0.9989
1.0245	2.6173	11500	0.0782	0.9993
0.9984	2.7311	12000	0.0755	0.9993
0.9773	2.8449	12500	0.0734	0.9993
1.0014	2.9587	13000	0.0733	0.9993

Framework versions

• Transformers 5.2.0
• Pytorch 2.8.0+cu128
• Datasets 4.5.0
• Tokenizers 0.22.2