iulik-pisik/horoscop_vreme_tiny

Horoscope Model Tiny - finetuned on weather

This model is a fine-tuned version of iulik-pisik/horoscope_model_tiny on the Vreme ProTV dataset. It achieves the following results on the evaluation set:

• Loss: 0.2868
• Wer: 13.9951

Model description

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.0443	6.02	1000	0.1949	15.5569
0.0023	12.05	2000	0.2601	14.1389
0.0008	18.07	3000	0.2798	14.0362
0.0006	24.1	4000	0.2868	13.9951

Framework versions

• Transformers 4.39.2
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2