| --- |
| base_model: unsloth/gemma-3-4b-it |
| library_name: peft |
| tags: |
| - lora |
| - unsloth |
| - gemma3 |
| - vision-language |
| - lithuanian |
| - weather-captioning |
| --- |
| |
| # Gemma 3 Lithuanian Weather Caption LoRA |
|
|
| This is a LoRA adapter fine-tuned for Lithuanian weather-focused image captioning. |
|
|
| The model was trained to describe the weather in an image using short, simple Lithuanian sentences. |
|
|
| ## Task |
|
|
| Given an image, the model generates a short Lithuanian caption focused on: |
|
|
| - cloudiness |
| - sunlight |
| - precipitation |
| - visibility |
| - time of day |
| - general weather conditions |
|
|
| ## Base model |
|
|
| `unsloth/gemma-3-4b-it` |
|
|
| ## Dataset |
|
|
| Dataset: `Matas5/GMM_team_task` |
|
|
| Training examples used: 103 |
|
|
| The dataset contains images with Lithuanian weather captions. Captions were standardized to focus mainly on weather conditions rather than unrelated objects. |
|
|
| ## Training setup |
|
|
| Training method: LoRA fine-tuning with Unsloth |
|
|
| Model loading: 4-bit quantized base model |
|
|
| Fine-tuning type: PEFT / LoRA adapter |
|
|
| Number of epochs: 3 |
|
|
| Total training steps: 78 |
|
|
| Per-device batch size: 1 |
|
|
| Gradient accumulation steps: 2 |
|
|
| Number of GPUs: 2 Tesla T4 GPUs |
|
|
| Effective total batch size: 4 |
|
|
| Learning rate: 2e-4 |
|
|
| Optimizer: adamw_8bit |
| |
| Gradient checkpointing: enabled |
| |
| Save strategy: save every epoch |
| |
| Trainable parameters: 1,611,776 |
| |
| Total model parameters shown during training: 2,941,163,888 |
| |
| Trainable percentage: 0.05% |
| |
| LoRA rank: 4 |
| |
| LoRA alpha: 8 |
| |
| Target modules: `q_proj`, `v_proj` |
| |
| Vision layers fine-tuned: yes |
| |
| ## Prompt used during training |
| |
| ```text |
| Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu. |
| ``` |
| |
| ## Example expected output style |
| |
| ```text |
| Dangus giedras ir ryškiai mėlynas, debesų beveik nėra. Oras saulėtas, sausas, matomumas labai geras. |
| ``` |
| |
| ## Training result summary |
| |
| The training loss decreased strongly during fine-tuning. |
| |
| Initial loss was around 4.7–5.3. |
| |
| Final loss was around 0.6–0.8. |
| |
| This suggests the adapter learned the caption format and Lithuanian weather description style from the training dataset. |
| |
| ## Intended use |
| |
| This adapter is intended for a university project demonstrating fine-tuning of a vision-language model for Lithuanian weather captioning. |
| |
| It is not intended for professional meteorological forecasting. |
| |
