Hunmin-397B-A17B-CUA

cua

A general-purpose 397B-parameter vision-language model (17B active) based on Qwen3.5-397B-A17B, developed through low-rank capability transfer followed by SFT and Agent RL (GRPO) for GUI grounding and multi-step desktop/browser tasks โ€” substantially improving computer-use and fine-grained GUI-grounding performance while keeping Korean benchmark results close to the base model.

Technical Blog: Hunmin-CUA โ€” Post-Training a 397B MoE for Computer Use

Overview

  • Author: Hojun Lee (์ดํ˜ธ์ค€) ยท LinkedIn
  • Role: Project Lead & Primary Researcher
  • Affiliation: GenON
  • Base model: Qwen3.5-397B-A17B (MoE, 397B total / ~17B active)
  • Post-training hardware: B200 x 8

Highlights

  • Strong GUI grounding. Hunmin achieves 75.6 on ScreenSpot-Pro with single-view inference (76.6 with zoom-in inference), 96.2 on ScreenSpot-v2, and 86.8 on OSWorld-G-R, leading all models compared under the same reproduced evaluation setup.
  • Large computer-use gains with Korean capability preserved. Compared with the base model, Hunmin improves by +22.3 on OSWorld and +9.1 on WindowsAgentArena. Across the Korean benchmarks reported below, performance remains within approximately 2 points of the base model.
  • Practical computer-use setting. All computer-use evaluations use image_max = 5, matching the training/evaluation setup. We intentionally use a smaller visual history because image_max โ‰ˆ 20 can introduce substantial inference cost and memory overhead in real-world computer-use deployments.

Hunmin improves computer-use and GUI-grounding performance while keeping Korean benchmark results close to the base model.

Benchmarks

GUI Grounding

Benchmark Base Hunmin (ours) Qwen-CUA
ScreenSpot-Pro 72.7 75.6 62.2
ScreenSpot-Pro (zoom-in) 75.1 76.6 71.9
ScreenSpot-v2 95.2 96.2 95.0
OSWorld-G 73.2 79.9 75.7
OSWorld-G-R 79.2 86.8 83.8

Author-reproduced results: baseline models were independently evaluated in the authors' environment rather than copied from model providers' reports.

Computer-Use Agents

All agent evaluations below use image_max = 5.

Benchmark Base Hunmin (ours) Qwen-CUA
OSWorld (360) 48.2 70.5 77.1
WindowsAgentArena (153) 41.8 50.9 57.3

Author-reproduced results: baseline models were independently evaluated in the authors' environment rather than copied from model providers' reports.

Note: These agent scores should not be directly compared with results obtained using substantially different visual-history settings such as image_max โ‰ˆ 20.

Korean

Benchmark Base Hunmin (ours) Qwen-CUA
KMMLU-Redux 82.6 81.4 78.1
KMMLU-Pro 77.9 76.1 71.4
IFEval-Ko 84.7 85.1 81.0
K-MMBench 92.5 92.5 90.3
K-SEED 81.3 81.5 80.9
K-MMStar 81.1 80.3 75.0
K-DTCBench 98.3 98.2 97.5
K-LLaVA-W (judge) 126.0 128.0 130.1

Across the reported Korean benchmark suite, Hunmin remains close to the Qwen3.5-397B-A17B base model while substantially improving computer-use capability.

Gains over Base

Benchmark Base Ours ฮ”
OSWorld (360) 48.2 70.5 +22.3
WindowsAgentArena (153) 41.8 50.9 +9.1
OSWorld-G-R 79.2 86.8 +7.6
ScreenSpot-Pro (single) 72.7 75.6 +2.9

The largest improvements appear on long-horizon agent tasks, while GUI grounding also improves and Korean performance remains close to the base model.

On OSWorld, the capability-transferred checkpoint achieved 66.2, and subsequent SFT and Agent RL (GRPO) further improved performance to 70.5 โ€” an additional +4.3 points under the same evaluation setup.

Comparison with Qwen-CUA

Both models were evaluated under the same settings.

Benchmark Qwen-CUA Ours ฮ” (Ours - Qwen-CUA)
ScreenSpot-Pro 62.2 75.6 +13.4
OSWorld-G-R 83.8 86.8 +3.0
KMMLU-Pro 71.4 76.1 +4.7
K-MMStar 75.0 80.3 +5.3
OSWorld (360) 77.1 70.5 -6.6

Developed through Qwen-CUA-derived low-rank capability transfer and subsequent post-training, Hunmin achieves a distinct capability profile. Under the same reproduced evaluation setup, it outperforms Qwen-CUA by 13.4 points on ScreenSpot-Pro and 3.0 points on OSWorld-G-R, while also achieving stronger results on several Korean benchmarks. Qwen-CUA remains stronger on long-horizon computer-use tasks.

For more detail about our evaluations, see our report.

Best Practices

  • Coordinates: outputs are normalized to [0, 1000]; scale using /1000 ร— (width, height).
  • Grounding: use native-resolution screenshots (up to ~16.7M pixels).
  • Agents: use image_max = 5 to match the training and evaluation configuration.

Limitations

  • Agent scores are configuration-dependent. Results reported here use image_max = 5 and should not be directly compared with evaluations using substantially different screenshot-history settings.
  • Performance may vary with screenshot resolution, reasoning configuration, interaction limits, serving stack, and benchmark harness versions.

Quickstart (vLLM)

vllm serve <path>/Hunmin-397B-A17B-CUA \
  --tensor-parallel-size 8 \
  --max-model-len 131072 \
  --gpu-memory-utilization 0.90 \
  --enable-prefix-caching

Training

Hunmin combines low-rank capability transfer from Qwen-CUA with subsequent SFT and Agent RL (GRPO). We computed the parameter delta between Qwen-CUA and the corresponding Qwen3.5 base checkpoint, approximated it using truncated SVD, and represented it as LoRA-style adapters. These adapters were first merged into Qwen3.5-397B-A17B BF16.

The resulting capability-transferred checkpoint was quantized to FP8 using the same fine-grained block-wise quantization scheme as the official Qwen3.5-397B-A17B-FP8 release. Further post-training was performed on this checkpoint using FP8 QLoRA.

The post-training adapters were merged into the BF16 counterpart of the capability-transferred checkpoint to produce the final model for release and evaluation.

Acknowledgements

Built on Qwen3.5-397B-A17B, with Qwen-CUA providing the source checkpoint for low-rank capability transfer and serving as a CUA-specialized reference model in our reproduced evaluations.

Downloads last month
85
Safetensors
Model size
397B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mncai/Hunmin-397B-A17B-CUA

Finetuned
(43)
this model
Quantizations
1 model

Article mentioning mncai/Hunmin-397B-A17B-CUA

Evaluation results