DigitalUmuganda/translategemma-4b-it-Kinyarwanda

sft_en-rw-mono_stage2_bi

This model is a fine-tuned version of saves/gtranslategemma3-4b/sft_en-rw-mono_stage1_bi on the expert_bi_gemma__train dataset. It achieves the following results on the evaluation set:

• Loss: 0.4795

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• total_eval_batch_size: 8
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss
0.4772	0.0909	500	0.4936
0.5266	0.1818	1000	0.4890
0.4793	0.2727	1500	0.4885
0.4821	0.3636	2000	0.4853
0.4723	0.4545	2500	0.4827
0.4774	0.5454	3000	0.4817
0.492	0.6363	3500	0.4810
0.5257	0.7272	4000	0.4800
0.5211	0.8181	4500	0.4791
0.5196	0.9090	5000	0.4794
0.4651	0.9999	5500	0.4795

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.1+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2