README.md

---
license: apache-2.0
base_model:
- Qwen/Qwen3-VL-4B-Instruct
language:
- en
pipeline_tag: image-text-to-text
library_name: transformers
tags:
- text-generation-inference
- spatial-reasoning
- vision-language
- multimodal
- image-captioning
- visual-question-answering
- conditional-generation
- vision
- language-model
- sft
- fine-grained-captioning
- computer-vision
- vllm
datasets:
- prithivMLmods/OpenCaption-FineGrained
- remyxai/SpaceThinker
---

![1](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/6pW_oYFBXOEx0mHQbC4Sw.png)

# **oMEGA-4B-SpatialThink-0804**

> **oMEGA-4B-SpatialThink-0804** is a vision-language model built on top of **Qwen/Qwen3-VL-4B-Instruct** and fine-tuned for **spatial reasoning with concise notes for unfiltered vision tasks**. The model is trained to produce concise yet informative reasoning for spatial understanding while maintaining strong image captioning capabilities. Training is based on **remyxai's SpaceThinker** and **OpenCaption-FineGrained**, enabling efficient spatial reasoning and detailed image understanding across diverse visual domains.

> [!NOTE]
> This model is an experimental release and may generate unexpected behaviors or reasoning artifacts in certain scenarios.

## **Key Highlights**

* **Qwen3-VL Foundation**: Built directly on top of **Qwen/Qwen3-VL-4B-Instruct**.
* **Spatial Reasoning**: Optimized for spatial understanding with concise reasoning notes for unfiltered vision tasks.
* **Concise Reasoning**: Generates compact reasoning while preserving essential spatial information.
* **Image Captioning**: Produces detailed and context-aware image captions.
* **Vision-Language Fine-Tuning**: Trained on high-quality spatial reasoning and fine-grained image caption datasets.
* **Research-Focused Release**: Designed for multimodal reasoning, spatial understanding, and image captioning research.
* **Efficient 4B Deployment**: Suitable for local multimodal inference and research environments.

## **Quick Start with Transformers**

```python
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import torch

model = Qwen3VLForConditionalGeneration.from_pretrained(
    "prithivMLmods/oMEGA-4B-SpatialThink-0804",
    torch_dtype="auto",
    device_map="auto"
)

processor = AutoProcessor.from_pretrained(
    "prithivMLmods/oMEGA-4B-SpatialThink-0804"
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg",
            },
            {
                "type": "text",
                "text": "Provide a detailed caption and reasoning for this image."
            },
        ],
    }
]

text = processor.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

image_inputs, video_inputs = process_vision_info(messages)

inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
).to("cuda")

generated_ids = model.generate(
    **inputs,
    max_new_tokens=128
)

generated_ids_trimmed = [
    out[len(inp):]
    for inp, out in zip(inputs.input_ids, generated_ids)
]

output_text = processor.batch_decode(
    generated_ids_trimmed,
    skip_special_tokens=True,
    clean_up_tokenization_spaces=False
)

print(output_text)
```

## **Training Details**

| Setting                 | Value                                                            |
| :---------------------- | :--------------------------------------------------------------- |
| **Base Model**          | **Qwen/Qwen3-VL-4B-Instruct**                                    |
| **Training Method**     | Supervised Fine-Tuning (SFT)                                     |
| **Primary Objective**   | Spatial Reasoning with Concise Notes for Unfiltered Vision Tasks |
| **Secondary Objective** | Efficient Spatial Reasoning and Image Captioning                 |
| **Training Framework**  | TRL + Transformers                                               |
| **Training Precision**  | BF16                                                             |

## **Intended Use**

* **Spatial Reasoning**: Understanding spatial relationships, object layouts, and geometric reasoning.
* **Image Captioning**: Producing detailed and fine-grained image descriptions.
* **Multimodal Reasoning**: Combining visual understanding with concise reasoning.
* **Vision Research**: Benchmarking and evaluating vision-language reasoning capabilities.
* **Local Deployment**: Efficient inference for multimodal applications.

## **Limitations**

* **Experimental Model**: Performance may vary across different visual domains.
* **Reasoning Artifacts**: Generated reasoning may occasionally contain incorrect intermediate interpretations.
* **Vision Ambiguity**: Highly ambiguous or low-quality images may reduce reasoning accuracy.

## **Acknowledgements**

* **[Qwen/Qwen3-VL-4B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-4B-Instruct)**: Base vision-language model used for this project.

* **[SpaceThinker](https://huggingface.co/datasets/remyxai/SpaceThinker)** by remyxai: A spatial reasoning dataset used to improve concise visual reasoning capabilities.

* **[OpenCaption-FineGrained](https://huggingface.co/datasets/prithivMLmods/OpenCaption-FineGrained)**: A fine-grained image captioning dataset used to enhance detailed visual understanding and caption generation.

* **TRL - [Transformers Reinforcement Learning](https://huggingface.co/docs/trl/en/index)**: Used for supervised fine-tuning and multimodal training.

* **[Transformers](https://huggingface.co/docs/transformers/en/index)**: Provides the model architecture, training, and inference framework for multimodal transformer models.