# model: gemma 3 4b it

# train only the projector fn (projector layers : 2 layers)

```
model.multi_modal_projector.mm_input_projection_weight | shape=(1152, 2560) | requires_grad=True
model.multi_modal_projector.mm_soft_emb_norm.weight | shape=(1152,) | requires_grad=True
```


HyperParameters of this model

lr: 2e-5

epochs: 10

batch size: 1

grad acumulation step: 4