OrinVoss/qwen3.5-9b-ov

Qwen3.5-9B OpenVINO — Intel GPU Native Multimodal LLM

Run Qwen3.5-9B multimodal on Intel integrated graphics. INT4 needs only ~6GB.

中文版 / Chinese

Model download: https://huggingface.co/OrinVoss/qwen3.5-9b-ov

Source code: https://github.com/OrinVoss/qwen3.5-9b-openvino

Quick Start

# Download model
git lfs install && git clone https://huggingface.co/OrinVoss/qwen3.5-9b-ov

# Dependencies
pip install openvino openvino-genai transformers torch pillow fastapi uvicorn opencv-python-headless

# Run server
cd qwen3.5-9b-ov && python server.py  # :8000

import httpx, base64

# Text
r = httpx.post('http://localhost:8000/v1/chat/completions', json={
    'messages': [{'role': 'user', 'content': 'Hello'}], 'max_tokens': 100
})

# Image
with open('photo.jpg', 'rb') as f:
    b64 = base64.b64encode(f.read()).decode()
r = httpx.post('http://localhost:8000/v1/chat/completions', json={
    'messages': [{'role': 'user', 'content': [
        {'type': 'image_url', 'image_url': {'url': f'data:image/jpeg;base64,{b64}'}},
        {'type': 'text', 'text': 'Describe this image'}
    ]}], 'max_tokens': 500
})

# Video
with open('video.mp4', 'rb') as f:
    b64 = base64.b64encode(f.read()).decode()
r = httpx.post('http://localhost:8000/v1/chat/completions', json={
    'messages': [{'role': 'user', 'content': [
        {'type': 'video_url', 'video_url': {'url': f'data:video/mp4;base64,{b64}'}},
        {'type': 'text', 'text': 'Describe this video'}
    ]}], 'max_tokens': 500
})

Variants

	INT4	FP16
Directory	int4/	fp16/
LM Size	4.6 GB	15 GB
VRAM	~6 GB	~18 GB
Context	~7K tokens	TBD
Vision	~15s / ~25s	TBD

Architecture

Module	Lines	Role
server.py	110	HTTP routing
engine.py	170	Lazy-loading inference
vision.py	60	PyTorch CPU vision
generation.py	60	Sampling & streaming
position_ids.py	44	MRoPE 3D position encoding

• Lazy loading: text first, vision on demand
• Graceful degradation: vision failure doesn't affect text
• Think mode: enable_thinking=False (works correctly)
• GPU pollution: known OpenVINO xe driver bug; restart or use /reset

Detailed technical documentation: TECHNICAL.md (English) | TECHNICAL_zh.md (中文)

---

Why

QAgent — a local-only QQ bot. Built for people without dedicated GPUs.

I have zero AI background and can't code. Built this in one afternoon with Claude Code.

License

Apache 2.0. Derivative of Qwen3.5-9B. Original weights (c) Alibaba Cloud.