Update README.md

README.md

@@ -1,199 +1,234 @@
 ---
 library_name: transformers
-tags: []
+tags:
+  - smolvlm
+  - vlm
+  - dpo
+  - hallucination-reduction
+  - accessibility
+  - qlora
+  - rlaif
+license: apache-2.0
+language:
+  - en
+pipeline_tag: image-text-to-text
+base_model: HuggingFaceTB/SmolVLM2-500M-Video-Instruct
+datasets:
+  - HuggingFaceH4/rlaif-v_formatted
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
+# Solari: Hallucination-Reduced Vision Language Model
 
+Solari is a 500M parameter vision-language model fine-tuned for **reduced hallucination** on real-world images. Built on [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct), Solari uses **QLoRA + Direct Preference Optimization (DPO)** on the [RLAIF-V](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) dataset to align the model toward more faithful visual descriptions.
 
 ## Model Details
 
 ### Model Description
 
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+Solari targets **hallucination reduction** in vision-language tasks, with a focus on improving reliability for **accessibility applications** (e.g., assisting visually impaired users). The model was trained using parameter-efficient fine-tuning (QLoRA) with DPO to learn preferences between accurate and hallucinated image descriptions, achieving improved hallucination benchmarks while preserving general VLM capabilities.
 
-### Model Sources [optional]
+- **Developed by:** Cubex11
+- **Model type:** Vision-Language Model (Image-Text-to-Text)
+- **Language(s):** English
+- **License:** Apache-2.0
+- **Finetuned from:** [HuggingFaceTB/SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct)
 
-<!-- Provide the basic links for the model. -->
+### Model Sources
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+- **Base Model:** [SmolVLM2-500M-Video-Instruct](https://huggingface.co/HuggingFaceTB/SmolVLM2-500M-Video-Instruct)
+- **Training Dataset:** [RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — 72K AI-generated preference pairs for hallucination reduction
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
+Solari can be used for image understanding tasks where **factual accuracy** is critical:
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+- Describing real-world scenes for visually impaired users
+- Visual question answering with reduced hallucination
+- Image captioning with improved object recognition reliability
 
 ### Out-of-Scope Use
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+- Tasks requiring strong mathematical reasoning or code understanding (degraded from base model)
+- Non-English language tasks
+- Medical or safety-critical applications without additional validation
 
 ## How to Get Started with the Model
 
-Use the code below to get started with the model.
-
-[More Information Needed]
+```python
+import torch
+from transformers import AutoModelForImageTextToText, AutoProcessor
+from PIL import Image
+import requests
+
+model_id = "Cubex11/Solari"
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    device_map="auto",
+)
+processor = AutoProcessor.from_pretrained(model_id)
+
+# Load an image
+url = "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png"
+image = Image.open(requests.get(url, stream=True).raw).convert("RGB")
+
+# Create prompt
+messages = [
+    {
+        "role": "user",
+        "content": [
+            {"type": "image"},
+            {"type": "text", "text": "Describe this image in detail."}
+        ]
+    }
+]
+
+text = processor.apply_chat_template(messages, add_generation_prompt=True)
+inputs = processor(text=text, images=[[image]], return_tensors="pt").to(model.device)
+output = model.generate(**inputs, max_new_tokens=256)
+trimmed = output[0][len(inputs.input_ids[0]):]
+print(processor.decode(trimmed, skip_special_tokens=True))
+```
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
+[RLAIF-V (Formatted)](https://huggingface.co/datasets/HuggingFaceH4/rlaif-v_formatted) — a large-scale multimodal preference dataset containing ~72K preference pairs. Each sample includes an image, a prompt, a **chosen** response (more accurate), and a **rejected** response (more hallucinated). Preferences are generated by open-source AI models following the RLAIF-V methodology.
 
 ### Training Procedure
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
+**Method:** QLoRA + Direct Preference Optimization (DPO)
 
+The base model was quantized to 4-bit (NF4) and fine-tuned using Low-Rank Adaptation (LoRA) with DPO to learn preferences between accurate and hallucinated responses.
 
 #### Training Hyperparameters
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
+| Parameter | Value |
+|-----------|-------|
+| **Training regime** | bf16 mixed precision |
+| **Quantization** | 4-bit NF4 (double quantization) |
+| **LoRA rank (r)** | 16 |
+| **LoRA alpha** | 16 |
+| **LoRA dropout** | 0.1 |
+| **DoRA** | Enabled |
+| **Target modules** | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
+| **Trainable params** | ~1.9% of total |
+| **Learning rate** | 5e-5 |
+| **DPO beta** | 0.1 |
+| **Batch size** | 8 (per device) |
+| **Gradient accumulation** | 4 (effective batch = 32) |
+| **Epochs** | 2 (best checkpoint at ~1 epoch / step 2500) |
+| **Warmup ratio** | 0.1 |
+| **Optimizer** | AdamW |
+
+#### Speeds, Sizes, Times
+
+- **Training time:** ~9 hours on NVIDIA L4 (24GB)
+- **Best checkpoint:** Step 2500 (selected by lowest validation loss)
+- **Model size:** ~1 GB (bf16 safetensors)
 
 ## Evaluation
 
-<!-- This section describes the evaluation protocols and provides the results. -->
-
 ### Testing Data, Factors & Metrics
 
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
+Evaluated using [VLMEvalKit](https://github.com/open-compass/VLMEvalKit) on 8 standard benchmarks covering hallucination, general VLM capability, and real-world understanding.
 
 #### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
+- **POPE:** F1 score across random/popular/adversarial splits (object hallucination)
+- **AMBER:** Attribute, Existence, Relation accuracy (multi-dimensional hallucination)
+- **HallusionBench:** aAcc, fAcc, qAcc (hallucination detection)
+- **A-OKVQA:** Accuracy on outside-knowledge VQA
+- **MME:** Perception and Reasoning scores
+- **MMStar:** Multi-modal reasoning accuracy
+- **MMBench:** General multi-modal understanding
+- **RealWorldQA:** Real-world image understanding accuracy
 
 ### Results
 
-[More Information Needed]
+| Benchmark | Metric | Base Model | **Solari** | Change |
+|-----------|--------|------------|------------|--------|
+| **POPE** | Overall | 82.67 | **85.08** | **+2.41** |
+| **POPE** | Recall | 76.73 | **85.33** | **+8.60** |
+| **AMBER** | Avg ACC | 79.38 | **79.77** | **+0.39** |
+| **AMBER** | Relation | 72.36 | **75.42** | **+3.06** |
+| **HallusionBench** | fAcc | 18.21 | **19.36** | **+1.16** |
+| **A-OKVQA** | Overall | 68.12% | **69.00%** | **+0.88%** |
+| **MMStar** | Overall | 38.33% | **39.60%** | **+1.27%** |
+| **MMBench** | Test | 53.14% | **53.42%** | **+0.28%** |
+| **RealWorldQA** | Overall | 49.80% | **50.59%** | **+0.78%** |
+| **MME** | Perception | **1216.19** | 1118.51 | -97.68 |
+| **MME** | Reasoning | **237.50** | 211.79 | -25.71 |
 
 #### Summary
 
+Solari improves on **7 out of 8 benchmarks** compared to the base model:
 
+- **POPE recall +8.60%** — dramatically better at recognizing objects actually present in images
+- **All hallucination benchmarks improved** — POPE, AMBER, and HallusionBench
+- **General capabilities preserved or improved** — A-OKVQA, MMStar, MMBench, RealWorldQA all show gains
+- **Trade-off on MME** — perception score dropped ~98 points, primarily on counting (-26.7), position (-26.7), and code reasoning (-27.5) subtasks due to the model becoming more conservative
 
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
+## Bias, Risks, and Limitations
 
-[More Information Needed]
+- **Counting and spatial reasoning degraded:** The DPO alignment made the model more conservative, reducing performance on fine-grained counting and positional reasoning tasks (reflected in MME scores).
+- **Small model capacity:** At 500M parameters, the model has inherent limitations on complex reasoning tasks.
+- **English only:** The model was trained and evaluated only on English-language tasks.
+- **Training data bias:** RLAIF-V preferences are AI-generated, which may introduce systematic biases.
 
-## Environmental Impact
+### Recommendations
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+- Best suited for binary object recognition tasks ("Is there a X?") and general scene description
+- For tasks requiring precise counting or spatial reasoning, consider using the base model or a larger VLM
+- Always validate outputs in safety-critical applications
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+## Environmental Impact
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+- **Hardware Type:** NVIDIA L4 (24GB)
+- **Hours used:** ~9 hours
+- **Cloud Provider:** Lightning AI
+- **Compute Region:** US
 
-## Technical Specifications [optional]
+## Technical Specifications
 
 ### Model Architecture and Objective
 
-[More Information Needed]
+- **Architecture:** SmolVLM2 (ViT vision encoder + LLM decoder with multi-modal projector)
+- **Parameters:** ~500M total
+- **Objective:** Direct Preference Optimization (DPO) — learns to prefer accurate descriptions over hallucinated ones
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
 #### Hardware
 
-[More Information Needed]
+NVIDIA L4 GPU (24GB VRAM) on Lightning AI
 
 #### Software
 
-[More Information Needed]
+- Transformers
+- TRL (DPO Trainer)
+- PEFT (QLoRA)
+- BitsAndBytes (4-bit quantization)
 
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+## Citation
 
 **BibTeX:**
 
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
+```bibtex
+@misc{solari2026,
+    title={Solari: Hallucination-Reduced Vision Language Model via QLoRA DPO on RLAIF-V},
+    author={Cubex11},
+    year={2026},
+    url={https://huggingface.co/Cubex11/Solari}
+}
+```
+
+## Acknowledgments
+
+- [HuggingFace](https://huggingface.co/) for SmolVLM2 and the RLAIF-V formatted dataset
+- [OpenBMB](https://github.com/OpenBMB) for the RLAIF-V and RLHF-V research
+- [Lightning AI](https://lightning.ai/) for compute resources
+- [OpenCompass](https://github.com/open-compass/VLMEvalKit) for the VLMEvalKit evaluation toolkit