| --- |
| library_name: transformers |
| tags: |
| - smolvlm |
| - vlm |
| - dpo |
| - hallucination-reduction |
| - accessibility |
| - qlora |
| - rlaif |
| license: apache-2.0 |
| language: |
| - en |
| pipeline_tag: image-text-to-text |
| base_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct |
| datasets: |
| - HuggingFaceH4/rlaif-v_formatted |
| --- |
| |
|  |
|
|
| # Solari: Hallucination-Reduced Vision Language Model |
|
|
| Solari is a 500M parameter vision-language model fine-tuned for **reduced hallucination** on real-world images. Built on [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct), Solari uses **QLoRA + Direct Preference Optimization (DPO)** on the [RLAIF-V](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) dataset to align the model toward more faithful visual descriptions. |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| Solari targets **hallucination reduction** in vision-language tasks, with a focus on improving reliability for **accessibility applications** (e.g., assisting visually impaired users). The model was trained using parameter-efficient fine-tuning (QLoRA) with DPO to learn preferences between accurate and hallucinated image descriptions, achieving improved hallucination benchmarks while preserving general VLM capabilities. |
|
|
| - **Developed by:** Cubex11 |
| - **Model type:** Vision-Language Model (Image-Text-to-Text) |
| - **Language(s):** English |
| - **License:** Apache-2.0 |
| - **Finetuned from:** [HuggingFaceTB/SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct) |
|
|
| ### Model Sources |
|
|
| - **Base Model:** [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct) |
| - **Training Dataset:** [RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — 72K AI-generated preference pairs for hallucination reduction |
|
|
| ## Uses |
|
|
| ### Direct Use |
|
|
| Solari can be used for image understanding tasks where **factual accuracy** is critical: |
|
|
| - Describing real-world scenes for visually impaired users |
| - Visual question answering with reduced hallucination |
| - Image captioning with improved object recognition reliability |
|
|
| ### Out-of-Scope Use |
|
|
| - Tasks requiring strong mathematical reasoning or code understanding (degraded from base model) |
| - Non-English language tasks |
| - Medical or safety-critical applications without additional validation |
|
|
| ## How to Get Started with the Model |
|
|
| ```python |
| import torch |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| from PIL import Image |
| import requests |
| |
| model_id = "Cubex11/Solari" |
| model = AutoModelForImageTextToText.from_pretrained( |
| model_id, |
| torch_dtype=torch.bfloat16, |
| device_map="auto", |
| ) |
| processor = AutoProcessor.from_pretrained(model_id) |
| |
| # Load an image (replace with your own image path or URL) |
| image = Image.open("your_image.jpg").convert("RGB") |
| |
| # Create prompt |
| messages = [ |
| { |
| "role": "user", |
| "content": [ |
| {"type": "image"}, |
| {"type": "text", "text": "Describe this image in detail."} |
| ] |
| } |
| ] |
| |
| text = processor.apply_chat_template(messages, add_generation_prompt=True) |
| inputs = processor(text=text, images=[[image]], return_tensors="pt").to(model.device) |
| output = model.generate(**inputs, max_new_tokens=256) |
| trimmed = output[0][len(inputs.input_ids[0]):] |
| print(processor.decode(trimmed, skip_special_tokens=True)) |
| ``` |
|
|
| ## Training Details |
|
|
| ### Training Data |
|
|
| [RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — a large-scale multimodal preference dataset containing ~72K preference pairs. Each sample includes an image, a prompt, a **chosen** response (more accurate), and a **rejected** response (more hallucinated). Preferences are generated by open-source AI models following the RLAIF-V methodology. |
|
|
| ### Training Procedure |
|
|
| **Method:** QLoRA + Direct Preference Optimization (DPO) |
|
|
| The base model was quantized to 4-bit (NF4) and fine-tuned using Low-Rank Adaptation (LoRA) with DPO to learn preferences between accurate and hallucinated responses. |
|
|
| #### Training Hyperparameters |
|
|
| | Parameter | Value | |
| |-----------|-------| |
| | **Training regime** | bf16 mixed precision | |
| | **Quantization** | 4-bit NF4 (double quantization) | |
| | **LoRA rank (r)** | 16 | |
| | **LoRA alpha** | 16 | |
| | **LoRA dropout** | 0.1 | |
| | **DoRA** | Enabled | |
| | **Target modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj | |
| | **Trainable params** | ~1.9% of total | |
| | **Learning rate** | 5e-5 | |
| | **DPO beta** | 0.1 | |
| | **Batch size** | 8 (per device) | |
| | **Gradient accumulation** | 4 (effective batch = 32) | |
| | **Epochs** | 2 (best checkpoint at ~1 epoch / step 2500) | |
| | **Warmup ratio** | 0.1 | |
| | **Optimizer** | AdamW | |
| |
| #### Speeds, Sizes, Times |
| |
| - **Training time:** ~9 hours on NVIDIA L4 (24GB) |
| - **Best checkpoint:** Step 2500 (selected by lowest validation loss) |
| - **Model size:** ~1 GB (bf16 safetensors) |
| |
| ## Evaluation |
| |
| ### Testing Data, Factors & Metrics |
| |
| Evaluated using [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) on 8 standard benchmarks covering hallucination, general VLM capability, and real-world understanding. |
| |
| #### Metrics |
| |
| - **POPE:** F1 score across random/popular/adversarial splits (object hallucination) |
| - **AMBER:** Attribute, Existence, Relation accuracy (multi-dimensional hallucination) |
| - **HallusionBench:** aAcc, fAcc, qAcc (hallucination detection) |
| - **A-OKVQA:** Accuracy on outside-knowledge VQA |
| - **MME:** Perception and Reasoning scores |
| - **MMStar:** Multi-modal reasoning accuracy |
| - **MMBench:** General multi-modal understanding |
| - **RealWorldQA:** Real-world image understanding accuracy |
| |
| ### Results |
| |
| | Benchmark | Metric | Base Model | **Solari** | Change | |
| |-----------|--------|------------|------------|--------| |
| | **POPE** | Overall | 82.67 | **85.08** | **+2.41** | |
| | **POPE** | Recall | 76.73 | **85.33** | **+8.60** | |
| | **AMBER** | Avg ACC | 79.38 | **79.77** | **+0.39** | |
| | **AMBER** | Relation | 72.36 | **75.42** | **+3.06** | |
| | **HallusionBench** | Overall | 27.58 | **28.14** | **+0.56** | |
| | **A-OKVQA** | Overall | 68.12 | **69.00** | **+0.88** | |
| | **MMStar** | Overall | 38.33 | **39.60** | **+1.27** | |
| | **MMBench** | Test | 53.14 | **53.42** | **+0.28** | |
| | **RealWorldQA** | Overall | 49.80 | **50.59** | **+0.78** | |
| | **MME** | Perception | **1216.19** | 1118.51 | -97.68 | |
| | **MME** | Reasoning | **237.50** | 211.79 | -25.71 | |
| |
| #### Summary |
| |
| Solari improves on **7 out of 8 benchmarks** compared to the base model: |
| |
| - **POPE recall +8.60%** — dramatically better at recognizing objects actually present in images |
| - **All hallucination benchmarks improved** — POPE, AMBER, and HallusionBench |
| - **General capabilities preserved or improved** — A-OKVQA, MMStar, MMBench, RealWorldQA all show gains |
| - **Trade-off on MME** — perception score dropped ~98 points, primarily on counting (-26.7), position (-26.7), and code reasoning (-27.5) subtasks due to the model becoming more conservative |
| |
| ## Bias, Risks, and Limitations |
| |
| - **Counting and spatial reasoning degraded:** The DPO alignment made the model more conservative, reducing performance on fine-grained counting and positional reasoning tasks (reflected in MME scores). |
| - **Small model capacity:** At 500M parameters, the model has inherent limitations on complex reasoning tasks. |
| - **English only:** The model was trained and evaluated only on English-language tasks. |
| - **Training data bias:** RLAIF-V preferences are AI-generated, which may introduce systematic biases. |
| |
| ### Recommendations |
| |
| - Best suited for binary object recognition tasks ("Is there a X?") and general scene description |
| - For tasks requiring precise counting or spatial reasoning, consider using the base model or a larger VLM |
| - Always validate outputs in safety-critical applications |
| |
| ## Environmental Impact |
| |
| - **Hardware Type:** NVIDIA L4 (24GB) |
| - **Hours used:** ~9 hours |
| - **Cloud Provider:** Lightning AI |
| - **Compute Region:** US |
| |
| ## Technical Specifications |
| |
| ### Model Architecture and Objective |
| |
| - **Architecture:** SmolVLM2 (ViT vision encoder + LLM decoder with multi-modal projector) |
| - **Parameters:** ~500M total |
| - **Objective:** Direct Preference Optimization (DPO) — learns to prefer accurate descriptions over hallucinated ones |
| |
| ### Compute Infrastructure |
| |
| #### Hardware |
| |
| NVIDIA L4 GPU (24GB VRAM) on Lightning AI |
| |
| #### Software |
| |
| - Transformers |
| - TRL (DPO Trainer) |
| - PEFT (QLoRA) |
| - BitsAndBytes (4-bit quantization) |
| |
| ## Citation |
| |
| **BibTeX:** |
| |
| ```bibtex |
| @misc{solari2026, |
| title={Solari: Hallucination-Reduced Vision Language Model via QLoRA DPO on RLAIF-V}, |
| author={Cubex11}, |
| year={2026}, |
| url={https://huggingface.co/Cubex11/Solari} |
| } |
| ``` |
| |
| ## Acknowledgments |
| |
| - [HuggingFace](https://huggingface.co/) for SmolVLM2 and the RLAIF-V formatted dataset |
| - [OpenBMB](https://github.com/OpenBMB) for the RLAIF-V and RLHF-V research |
| - [Lightning AI](https://lightning.ai/) for compute resources |
| - [OpenCompass](https://github.com/open-compass/VLMEvalKit) for the VLMEvalKit evaluation toolkit |
