README.md

---
license: apache-2.0
base_model: AIDC-AI/Ovis2-4B
tags:
  - quantized
  - awq
  - vision-language-model
  - vlm
---

# AIDC-AI__Ovis2-4B__awq_w4_complete

This is a **AWQ** (4-bit) quantized version of [AIDC-AI/Ovis2-4B](https://huggingface.co/AIDC-AI/Ovis2-4B).

## Quantization Details

- **Method**: AWQ
- **Bits**: 4
- **Base model**: AIDC-AI/Ovis2-4B
- **Group size**: 128
- **Quantized portion**: LLM backbone (Qwen2ForCausalLM inside Ovis2)
- **Vision tokenizer + visual embedding**: FP16 (unchanged)
- **Loading**: AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True)

## Usage

```python
from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "{REPO_ID}",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("{REPO_ID}", trust_remote_code=True)
```

Replace `{REPO_ID}` with the repo ID of this model.

## Original Model

See [AIDC-AI/Ovis2-4B](https://huggingface.co/AIDC-AI/Ovis2-4B) for the original FP16 model.