Matas5 commited on
Commit
d1809ec
·
verified ·
1 Parent(s): 9c5663d

Update model card with training parameters

Browse files
Files changed (1) hide show
  1. README.md +92 -5
README.md CHANGED
@@ -12,12 +12,99 @@ tags:
12
 
13
  # Gemma 3 Lithuanian Weather Caption LoRA
14
 
15
- This is a LoRA adapter fine-tuned on Lithuanian weather image captions.
16
 
17
- Base model: `unsloth/gemma-3-4b-it`
18
 
19
- Task: given an image, generate a short Lithuanian caption focused on weather and time of day.
20
 
21
- Example prompt:
22
 
23
- `Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu.`
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
 
13
  # Gemma 3 Lithuanian Weather Caption LoRA
14
 
15
+ This is a LoRA adapter fine-tuned for Lithuanian weather-focused image captioning.
16
 
17
+ The model was trained to describe the weather in an image using short, simple Lithuanian sentences.
18
 
19
+ ## Task
20
 
21
+ Given an image, the model generates a short Lithuanian caption focused on:
22
 
23
+ - cloudiness
24
+ - sunlight
25
+ - precipitation
26
+ - visibility
27
+ - time of day
28
+ - general weather conditions
29
+
30
+ ## Base model
31
+
32
+ `unsloth/gemma-3-4b-it`
33
+
34
+ ## Dataset
35
+
36
+ Dataset: `Matas5/GMM_team_task`
37
+
38
+ Training examples used: 103
39
+
40
+ The dataset contains images with Lithuanian weather captions. Captions were standardized to focus mainly on weather conditions rather than unrelated objects.
41
+
42
+ ## Training setup
43
+
44
+ Training method: LoRA fine-tuning with Unsloth
45
+
46
+ Model loading: 4-bit quantized base model
47
+
48
+ Fine-tuning type: PEFT / LoRA adapter
49
+
50
+ Number of epochs: 3
51
+
52
+ Total training steps: 78
53
+
54
+ Per-device batch size: 1
55
+
56
+ Gradient accumulation steps: 2
57
+
58
+ Number of GPUs: 2 Tesla T4 GPUs
59
+
60
+ Effective total batch size: 4
61
+
62
+ Learning rate: 2e-4
63
+
64
+ Optimizer: adamw_8bit
65
+
66
+ Gradient checkpointing: enabled
67
+
68
+ Save strategy: save every epoch
69
+
70
+ Trainable parameters: 1,611,776
71
+
72
+ Total model parameters shown during training: 2,941,163,888
73
+
74
+ Trainable percentage: 0.05%
75
+
76
+ LoRA rank: 4
77
+
78
+ LoRA alpha: 8
79
+
80
+ Target modules: `q_proj`, `v_proj`
81
+
82
+ Vision layers fine-tuned: yes
83
+
84
+ ## Prompt used during training
85
+
86
+ ```text
87
+ Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu.
88
+ ```
89
+
90
+ ## Example expected output style
91
+
92
+ ```text
93
+ Dangus giedras ir ryškiai mėlynas, debesų beveik nėra. Oras saulėtas, sausas, matomumas labai geras.
94
+ ```
95
+
96
+ ## Training result summary
97
+
98
+ The training loss decreased strongly during fine-tuning.
99
+
100
+ Initial loss was around 4.7–5.3.
101
+
102
+ Final loss was around 0.6–0.8.
103
+
104
+ This suggests the adapter learned the caption format and Lithuanian weather description style from the training dataset.
105
+
106
+ ## Intended use
107
+
108
+ This adapter is intended for a university project demonstrating fine-tuning of a vision-language model for Lithuanian weather captioning.
109
+
110
+ It is not intended for professional meteorological forecasting.