metadata
license: apache-2.0
base_model: AIDC-AI/Ovis2-4B
tags:
- quantized
- awq
- vision-language-model
- vlm
AIDC-AI__Ovis2-4B__awq_w4_complete
This is a AWQ (4-bit) quantized version of AIDC-AI/Ovis2-4B.
Quantization Details
- Method: AWQ
- Bits: 4
- Base model: AIDC-AI/Ovis2-4B
- Group size: 128
- Quantized portion: LLM backbone (Qwen2ForCausalLM inside Ovis2)
- Vision tokenizer + visual embedding: FP16 (unchanged)
- Loading: AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True)
Usage
from transformers import AutoProcessor, AutoModelForImageTextToText
import torch
model = AutoModelForImageTextToText.from_pretrained(
"{REPO_ID}",
torch_dtype=torch.float16,
device_map="auto",
trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("{REPO_ID}", trust_remote_code=True)
Replace {REPO_ID} with the repo ID of this model.
Original Model
See AIDC-AI/Ovis2-4B for the original FP16 model.