---
base_model: unsloth/gemma-3-4b-it
library_name: peft
tags:
- lora
- unsloth
- gemma3
- vision-language
- lithuanian
- weather-captioning
---

# Gemma 3 Lithuanian Weather Caption LoRA

This is a LoRA adapter fine-tuned for Lithuanian weather-focused image captioning.

The model was trained to describe the weather in an image using short, simple Lithuanian sentences.

## Task

Given an image, the model generates a short Lithuanian caption focused on:

- cloudiness
- sunlight
- precipitation
- visibility
- time of day
- general weather conditions

## Base model

`unsloth/gemma-3-4b-it`

## Dataset

Dataset: `Matas5/GMM_team_task`

Training examples used: 103

The dataset contains images with Lithuanian weather captions. Captions were standardized to focus mainly on weather conditions rather than unrelated objects.

## Training setup

Training method: LoRA fine-tuning with Unsloth

Model loading: 4-bit quantized base model

Fine-tuning type: PEFT / LoRA adapter

Number of epochs: 3

Total training steps: 78

Per-device batch size: 1

Gradient accumulation steps: 2

Number of GPUs: 2 Tesla T4 GPUs

Effective total batch size: 4

Learning rate: 2e-4

Optimizer: adamw_8bit

Gradient checkpointing: enabled

Save strategy: save every epoch

Trainable parameters: 1,611,776

Total model parameters shown during training: 2,941,163,888

Trainable percentage: 0.05%

LoRA rank: 4

LoRA alpha: 8

Target modules: `q_proj`, `v_proj`

Vision layers fine-tuned: yes

## Prompt used during training

```text
Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu.
```

## Example expected output style

```text
Dangus giedras ir ryškiai mėlynas, debesų beveik nėra. Oras saulėtas, sausas, matomumas labai geras.
```

## Training result summary

The training loss decreased strongly during fine-tuning.

Initial loss was around 4.7–5.3.

Final loss was around 0.6–0.8.

This suggests the adapter learned the caption format and Lithuanian weather description style from the training dataset.

## Intended use

This adapter is intended for a university project demonstrating fine-tuning of a vision-language model for Lithuanian weather captioning.

It is not intended for professional meteorological forecasting.