Instructions to use OpenLLM-Ro/RoGemma3-4B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenLLM-Ro/RoGemma3-4B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="OpenLLM-Ro/RoGemma3-4B-Instruct")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("OpenLLM-Ro/RoGemma3-4B-Instruct")
model = AutoModelForMultimodalLM.from_pretrained("OpenLLM-Ro/RoGemma3-4B-Instruct")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenLLM-Ro/RoGemma3-4B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenLLM-Ro/RoGemma3-4B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenLLM-Ro/RoGemma3-4B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/OpenLLM-Ro/RoGemma3-4B-Instruct

SGLang

How to use OpenLLM-Ro/RoGemma3-4B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenLLM-Ro/RoGemma3-4B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenLLM-Ro/RoGemma3-4B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenLLM-Ro/RoGemma3-4B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenLLM-Ro/RoGemma3-4B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use OpenLLM-Ro/RoGemma3-4B-Instruct with Docker Model Runner:
```
docker model run hf.co/OpenLLM-Ro/RoGemma3-4B-Instruct
```

RoGemma3-4B-Instruct / README.md

mihaimasala

Unify Getting Started snippet (consistency + decode slice fix)

49cc5bf verified about 1 month ago

preview code

Raw

History Blame Contribute Delete

10.2 kB

metadata

language:
  - ro
license: cc-by-nc-4.0
library_name: transformers
tags:
  - gemma3
  - gemma
  - romanian
  - vlm
  - instruct
  - multimodal
datasets:
  - OpenLLM-Ro/ro_sft_laion
  - OpenLLM-Ro/ro_sft_pixmo_cap
  - OpenLLM-Ro/ro_sft_flickr30k_cap
  - OpenLLM-Ro/ro_sft_llava_mix
  - OpenLLM-Ro/ro_sft_pixmo_aa
  - OpenLLM-Ro/ro_sft_pixmo_cap_qa
  - OpenLLM-Ro/ro_sft_flickr30k_qa
  - OpenLLM-Ro/ro_sft_cosyn
  - OpenLLM-Ro/ro_sft_finepdfs
  - OpenLLM-Ro/ro_sft_pixmo_points
  - OpenLLM-Ro/ro_sft_pixmo_count
base_model:
  - google/gemma-3-4b-it
model-index:
  - name: OpenLLM-Ro/RoGemma3-4B-Instruct
    results:
      - task:
          type: image-text-to-text
        dataset:
          name: Romanian_VLM_Benchmarks
          type: Romanian_VLM_Benchmarks
        metrics:
          - name: Micro avg.
            type: Score
            value: 59.54
          - name: Macro avg.
            type: Score
            value: 57.14
      - task:
          type: image-text-to-text
        dataset:
          name: MMBench
          type: MMBench
        metrics:
          - name: Accuracy
            type: accuracy
            value: 69.96
      - task:
          type: image-text-to-text
        dataset:
          name: MMStar
          type: MMStar
        metrics:
          - name: Accuracy
            type: accuracy
            value: 46.01
      - task:
          type: image-text-to-text
        dataset:
          name: SeedBench2
          type: SeedBench2
        metrics:
          - name: Accuracy
            type: accuracy
            value: 62.83
      - task:
          type: image-text-to-text
        dataset:
          name: MMMU
          type: MMMU
        metrics:
          - name: Accuracy
            type: accuracy
            value: 38.67
      - task:
          type: image-text-to-text
        dataset:
          name: MME
          type: MME
        metrics:
          - name: Accuracy
            type: accuracy
            value: 54.62
      - task:
          type: image-text-to-text
        dataset:
          name: CVQA
          type: CVQA
        metrics:
          - name: Accuracy
            type: accuracy
            value: 64.24
      - task:
          type: image-text-to-text
        dataset:
          name: ALM-Bench
          type: ALM-Bench
        metrics:
          - name: Score
            type: score
            value: 65.4
      - task:
          type: image-text-to-text
        dataset:
          name: RoMemes
          type: RoMemes
        metrics:
          - name: F1
            type: f1
            value: 40.78
      - task:
          type: image-text-to-text
        dataset:
          name: RoCultVLM
          type: RoCultVLM
        metrics:
          - name: Score
            type: score
            value: 52.84
      - task:
          type: image-text-to-text
        dataset:
          name: RoFlickr30k-Caption
          type: RoFlickr30k-Caption
        metrics:
          - name: BERTScore
            type: bertscore
            value: 84.35
      - task:
          type: image-text-to-text
        dataset:
          name: RoFlickr30k-QA
          type: RoFlickr30k-QA
        metrics:
          - name: Score
            type: score
            value: 84.74
      - task:
          type: image-text-to-text
        dataset:
          name: LLaVA-Wild
          type: LLaVA-Wild
        metrics:
          - name: Score
            type: score
            value: 55.71
      - task:
          type: image-text-to-text
        dataset:
          name: AyaVisionBench
          type: AyaVisionBench
        metrics:
          - name: Score
            type: score
            value: 47.04
      - task:
          type: image-text-to-text
        dataset:
          name: m-WildVision
          type: m-WildVision
        metrics:
          - name: Score
            type: score
            value: 57.6
      - task:
          type: image-text-to-text
        dataset:
          name: RoCosyn
          type: RoCosyn
        metrics:
          - name: Score
            type: score
            value: 59.06
      - task:
          type: image-text-to-text
        dataset:
          name: RoFinepdfs
          type: RoFinepdfs
        metrics:
          - name: ANLS
            type: anls
            value: 84.35
      - task:
          type: image-text-to-text
        dataset:
          name: RoMemes OCR
          type: RoMemes-OCR
        metrics:
          - name: ANLS
            type: anls
            value: 86.33
      - task:
          type: image-text-to-text
        dataset:
          name: PixmoCount
          type: PixmoCount
        metrics:
          - name: Exact match
            type: exact_match
            value: 51.8
      - task:
          type: image-text-to-text
        dataset:
          name: PixmoPoints
          type: PixmoPoints
        metrics:
          - name: F1
            type: f1
            value: 24.87

Model Card for RoGemma3-4B-Instruct

RoGemma3-4B-Instruct is a Romanian-adapted vision-language model built on top of google/gemma-3-4b-it. It was produced by continued supervised instruction tuning of the base Gemma 3 checkpoint on a Romanian multimodal SFT mixture covering general instruction following (LLaVA mix), captioning (Pixmo-Cap, Flickr30k-Cap), visual question answering (Pixmo-AA, Pixmo-Cap-QA, Flickr30k-QA), document and chart understanding (CoSyn, FinePDFs), and visual grounding (Pixmo-Points, Pixmo-Count). The model is intended for research on Romanian VLM capabilities.

Model Details

Model Description

Developed by: OpenLLM-Ro
Language(s): Romanian
License: cc-by-nc-4.0
Finetuned from model: google/gemma-3-4b-it
Trained using:

Model Sources

Repository: https://github.com/OpenLLM-Ro/LLaMA-Factory
Paper: https://arxiv.org/abs/2605.31401

Intended Use

Intended Use Cases

RoGemma3-4B-Instruct is intended for research use on Romanian vision-language tasks — captioning, visual question answering, cultural understanding, OCR / document understanding, and visual grounding — and as a starting point for further Romanian VLM adaptation.

Out-of-Scope Use

Use in any manner that violates applicable laws or regulations (including trade-compliance laws), the project's license, or use in languages other than Romanian.

How to Get Started with the Model

import torch
from PIL import Image
from transformers import AutoProcessor, Gemma3ForConditionalGeneration

model = Gemma3ForConditionalGeneration.from_pretrained(
    "OpenLLM-Ro/RoGemma3-4B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
).eval()
processor = AutoProcessor.from_pretrained("OpenLLM-Ro/RoGemma3-4B-Instruct")

image = Image.open("example.jpg").convert("RGB")
question = "Descrie imaginea în detaliu."

messages = [
    {"role": "user", "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": question},
    ]},
]
inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Benchmarks

All benchmarks below are evaluated in Romanian. Per-benchmark winners are shown in bold. Micro is the mean over individual benchmarks; Macro is the mean over capability groups.

Aggregate

Model	Micro avg.	Macro avg.
Gemma3-4B-it	55.53	52.36
RoGemma3-4B-Instruct	59.54	57.14

General Understanding

Model	MMBench	MMStar	SeedBench2
Gemma3-4B-it	59.13	41.49	57.55
RoGemma3-4B-Instruct	69.96	46.01	62.83

Knowledge & Reasoning

Model	MMMU	MME
Gemma3-4B-it	36.67	57.32
RoGemma3-4B-Instruct	38.67	54.62

Cultural

Model	CVQA	ALM-Bench	RoMemes	RoCultVLM
Gemma3-4B-it	64.90	65.97	43.24	52.19
RoGemma3-4B-Instruct	64.24	65.40	40.78	52.84

Generation & Open-ended

Model	RoFlickr30k-Caption	RoFlickr30k-QA	LLaVA-Wild	AyaVisionBench	m-WildVision
Gemma3-4B-it	70.93	81.66	54.84	52.81	60.80
RoGemma3-4B-Instruct	84.35	84.74	55.71	47.04	57.60

OCR & Documents

Model	RoCosyn	RoFinepdfs	RoMemes OCR
Gemma3-4B-it	48.40	67.12	89.47
RoGemma3-4B-Instruct	59.06	84.35	86.33

Grounding

Model	PixmoCount	PixmoPoints
Gemma3-4B-it	40.42	10.21
RoGemma3-4B-Instruct	51.80	24.87

Citation

@misc{masala2026intelegi,
      title={``\^{I}n\c{t}elegi Rom\^{a}ne\c{s}te?'' A Recipe for Romanian Vision-Language Models},
      author={Mihai Masala and Marius Leordeanu and Mihai Dascalu and Traian Rebedea},
      year={2026},
      eprint={2605.31401},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2605.31401},
}

@inproceedings{masala-etal-2024-vorbesti,
    title = "``Vorbeşti Româneşte?'' A Recipe to Train Powerful {R}omanian {LLM}s with {E}nglish Instructions",
    author = "Masala, Mihai and Ilie-Ablachim, Denis and Dima, Alexandru and Corlatescu, Dragos and Zavelca, Miruna and Olaru, Ovio and Terian, Simina and Terian, Andrei and Leordeanu, Marius and Velicu, Horia and Popescu, Marius and Dascalu, Mihai and Rebedea, Traian",
    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2024",
    month = nov,
    year = "2024",
    pages = "11632--11647"
}