Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Paper • 2605.30280 • Published • 146
Younes - Overshoot PRO
younes-ovs
AI & ML interests
None yet
Recent Activity
upvoted a paper about 1 month ago
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments liked a model about 2 months ago
RedHatAI/gemma-4-31B-it-FP8-block updated a dataset 3 months ago
younes-ovs/inference-video-benchmark-v1Organizations
upvoted a paper about 1 month ago
liked a
model about 2 months ago
updated 2
datasets 3 months ago
published 2
datasets 3 months ago
liked 7
models 3 months ago
google/gemma-4-26B-A4B
Image-Text-to-Text • 27B • Updated • 721k • 331
moondream/moondream3-preview
Image-Text-to-Text • 9B • Updated • 307k • 661
datalab-to/chandra-ocr-2
Image-Text-to-Text • 5B • Updated • 1.34M • 433
moonshotai/Kimi-VL-A3B-Thinking
Image-Text-to-Text • 16B • Updated • 145k • 449
moonshotai/Kimi-VL-A3B-Thinking-2506
Image-Text-to-Text • 16B • Updated • 4.33k • 366
Hcompany/Holo3-35B-A3B
Image-Text-to-Text • 35B • Updated • 19.1k • 369
baidu/Qianfan-OCR
Image-Text-to-Text • 5B • Updated • 259k • 1.19k
Benchmarks, tradeoffs, and live video inference across five Qwen 3.5 models
🔥 6
#43 opened 4 months ago
by younes-ovs
reacted to nightmedia's post with 🚀 4 months ago
Post
3017
Qwen3.5 Performance Metrics
With the 3.5 architecture, a lot of the old quanting methods don't work as before. I noticed this when benchmarking Deckard(qx) quants and by mistake ran a q8 that was better. That only happens if the qx sucked--and it did--enhancing layers just because they look interesting doesn't work anymore, so until I get a clear understanding of the architecture, I will publish mxfp4 and mxfp8 of the 3.5 models, that seem very stable and high performant
I will start posting here the metrics I gather from the series, starting with the smallest. If I have numbers from previous or similar models, I will post them in comparison
Qwen3.5-0.8B
Detailed metrics by model
nightmedia/Qwen3.5-0.8B-mxfp8-mlx
nightmedia/Qwen3.5-2B-mxfp8-mlx
nightmedia/Qwen3.5-4B-mxfp8-mlx
nightmedia/Qwen3.5-9B-mxfp8-mlx
nightmedia/Qwen3.5-27B-Text
nightmedia/Qwen3.5-122B-A10B-Text-mxfp4-mlx
More metrics coming soon.
I am running these on my Mac, an M4Max with 128GB RAM. Some performance numbers like tokens/second reflect the performance on my box.
This post will be updated with every model that gets tested. The larger models take hours, the 27B a couple days, so it will be a long process.
-G
With the 3.5 architecture, a lot of the old quanting methods don't work as before. I noticed this when benchmarking Deckard(qx) quants and by mistake ran a q8 that was better. That only happens if the qx sucked--and it did--enhancing layers just because they look interesting doesn't work anymore, so until I get a clear understanding of the architecture, I will publish mxfp4 and mxfp8 of the 3.5 models, that seem very stable and high performant
I will start posting here the metrics I gather from the series, starting with the smallest. If I have numbers from previous or similar models, I will post them in comparison
Qwen3.5-0.8B
quant arc arc/e boolq hswag obkqa piqa wino
mxfp8 0.351,0.501,0.733,0.462,0.348,0.682,0.573
mxfp4 0.339,0.489,0.738,0.433,0.330,0.672,0.553
Old model performance
Qwen3-0.6B
bf16 0.298,0.354,0.378,0.415,0.344,0.649,0.534
q8-hi 0.296,0.355,0.378,0.416,0.348,0.652,0.529
q8 0.299,0.354,0.378,0.414,0.346,0.650,0.535
q6-hi 0.301,0.356,0.378,0.415,0.350,0.651,0.541
q6 0.300,0.367,0.378,0.416,0.344,0.647,0.524
mxfp4 0.286,0.364,0.609,0.404,0.316,0.626,0.531
Quant Perplexity Peak memory
mxfp8 6.611 ± 0.049 7.65 GB
mxfp4 7.455 ± 0.057 6.33 GBDetailed metrics by model
nightmedia/Qwen3.5-0.8B-mxfp8-mlx
nightmedia/Qwen3.5-2B-mxfp8-mlx
nightmedia/Qwen3.5-4B-mxfp8-mlx
nightmedia/Qwen3.5-9B-mxfp8-mlx
nightmedia/Qwen3.5-27B-Text
nightmedia/Qwen3.5-122B-A10B-Text-mxfp4-mlx
More metrics coming soon.
I am running these on my Mac, an M4Max with 128GB RAM. Some performance numbers like tokens/second reflect the performance on my box.
This post will be updated with every model that gets tested. The larger models take hours, the 27B a couple days, so it will be a long process.
-G
- 21 replies
upvoted a collection 4 months ago
liked a
model 4 months ago
liked a
model 7 months ago
liked a
model 8 months ago
liked a
Space 9 months ago