Instructions to use Matas5/gemma-weather-lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Matas5/gemma-weather-lora with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("unsloth/gemma-3-4b-it-unsloth-bnb-4bit") model = PeftModel.from_pretrained(base_model, "Matas5/gemma-weather-lora") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Unsloth Studio
How to use Matas5/gemma-weather-lora with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Matas5/gemma-weather-lora to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for Matas5/gemma-weather-lora to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for Matas5/gemma-weather-lora to start chatting
Load model with FastModel
pip install unsloth from unsloth import FastModel model, tokenizer = FastModel.from_pretrained( model_name="Matas5/gemma-weather-lora", max_seq_length=2048, )
Update model card with training parameters
Browse files
README.md
CHANGED
|
@@ -12,12 +12,99 @@ tags:
|
|
| 12 |
|
| 13 |
# Gemma 3 Lithuanian Weather Caption LoRA
|
| 14 |
|
| 15 |
-
This is a LoRA adapter fine-tuned
|
| 16 |
|
| 17 |
-
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
|
| 13 |
# Gemma 3 Lithuanian Weather Caption LoRA
|
| 14 |
|
| 15 |
+
This is a LoRA adapter fine-tuned for Lithuanian weather-focused image captioning.
|
| 16 |
|
| 17 |
+
The model was trained to describe the weather in an image using short, simple Lithuanian sentences.
|
| 18 |
|
| 19 |
+
## Task
|
| 20 |
|
| 21 |
+
Given an image, the model generates a short Lithuanian caption focused on:
|
| 22 |
|
| 23 |
+
- cloudiness
|
| 24 |
+
- sunlight
|
| 25 |
+
- precipitation
|
| 26 |
+
- visibility
|
| 27 |
+
- time of day
|
| 28 |
+
- general weather conditions
|
| 29 |
+
|
| 30 |
+
## Base model
|
| 31 |
+
|
| 32 |
+
`unsloth/gemma-3-4b-it`
|
| 33 |
+
|
| 34 |
+
## Dataset
|
| 35 |
+
|
| 36 |
+
Dataset: `Matas5/GMM_team_task`
|
| 37 |
+
|
| 38 |
+
Training examples used: 103
|
| 39 |
+
|
| 40 |
+
The dataset contains images with Lithuanian weather captions. Captions were standardized to focus mainly on weather conditions rather than unrelated objects.
|
| 41 |
+
|
| 42 |
+
## Training setup
|
| 43 |
+
|
| 44 |
+
Training method: LoRA fine-tuning with Unsloth
|
| 45 |
+
|
| 46 |
+
Model loading: 4-bit quantized base model
|
| 47 |
+
|
| 48 |
+
Fine-tuning type: PEFT / LoRA adapter
|
| 49 |
+
|
| 50 |
+
Number of epochs: 3
|
| 51 |
+
|
| 52 |
+
Total training steps: 78
|
| 53 |
+
|
| 54 |
+
Per-device batch size: 1
|
| 55 |
+
|
| 56 |
+
Gradient accumulation steps: 2
|
| 57 |
+
|
| 58 |
+
Number of GPUs: 2 Tesla T4 GPUs
|
| 59 |
+
|
| 60 |
+
Effective total batch size: 4
|
| 61 |
+
|
| 62 |
+
Learning rate: 2e-4
|
| 63 |
+
|
| 64 |
+
Optimizer: adamw_8bit
|
| 65 |
+
|
| 66 |
+
Gradient checkpointing: enabled
|
| 67 |
+
|
| 68 |
+
Save strategy: save every epoch
|
| 69 |
+
|
| 70 |
+
Trainable parameters: 1,611,776
|
| 71 |
+
|
| 72 |
+
Total model parameters shown during training: 2,941,163,888
|
| 73 |
+
|
| 74 |
+
Trainable percentage: 0.05%
|
| 75 |
+
|
| 76 |
+
LoRA rank: 4
|
| 77 |
+
|
| 78 |
+
LoRA alpha: 8
|
| 79 |
+
|
| 80 |
+
Target modules: `q_proj`, `v_proj`
|
| 81 |
+
|
| 82 |
+
Vision layers fine-tuned: yes
|
| 83 |
+
|
| 84 |
+
## Prompt used during training
|
| 85 |
+
|
| 86 |
+
```text
|
| 87 |
+
Trumpai apibūdink orą šioje nuotraukoje lietuviškai. Atsakyk vienu paprastu sakiniu.
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
## Example expected output style
|
| 91 |
+
|
| 92 |
+
```text
|
| 93 |
+
Dangus giedras ir ryškiai mėlynas, debesų beveik nėra. Oras saulėtas, sausas, matomumas labai geras.
|
| 94 |
+
```
|
| 95 |
+
|
| 96 |
+
## Training result summary
|
| 97 |
+
|
| 98 |
+
The training loss decreased strongly during fine-tuning.
|
| 99 |
+
|
| 100 |
+
Initial loss was around 4.7–5.3.
|
| 101 |
+
|
| 102 |
+
Final loss was around 0.6–0.8.
|
| 103 |
+
|
| 104 |
+
This suggests the adapter learned the caption format and Lithuanian weather description style from the training dataset.
|
| 105 |
+
|
| 106 |
+
## Intended use
|
| 107 |
+
|
| 108 |
+
This adapter is intended for a university project demonstrating fine-tuning of a vision-language model for Lithuanian weather captioning.
|
| 109 |
+
|
| 110 |
+
It is not intended for professional meteorological forecasting.
|