README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-9B
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - computer-use-agent
  - gui-agent
  - reward-model
  - llm-as-a-judge
  - reinforcement-learning
---

# SeekJudge-9B

SeekJudge-9B is the trained judge model of **SeekJudge**, a reward framework for
reinforcement learning in computer-use agents. Given a task goal and a recorded
trajectory (screenshots plus actions), it decides whether the goal was achieved
and emits per-step quality labels in the `<answer_overall>` / `<answer_step>`
format.

The model is a judge, not an agent. It does not produce GUI actions.

## Code and resources

| | |
|---|---|
| Code | [github.com/ZJUSCL/SeekJudge](https://github.com/ZJUSCL/SeekJudge) |
| Benchmark | [ZJUSCL/CUAStepBench](https://huggingface.co/datasets/ZJUSCL/CUAStepBench) |
| Leaderboard | [github.com/ZJUSCL/CUAStepBench](https://github.com/ZJUSCL/CUAStepBench) |

The repository README covers serving the model, running it as a batch judge
over trajectory datasets, and hosting it as an HTTP reward server for online RL
training. This checkpoint requires `seek.trained: True` in the SeekJudge
configuration, which selects the parser for the trained output format.

## Model size

| | |
|---|---|
| Total parameters | 9.41 B (9,409.81 M) |
| Trainable during SFT | 8.95 B (8,953.80 M, 95.15 %) |
| Frozen | vision encoder and aligner (456 M) |
| Precision | bfloat16 |
| Checkpoint on disk | 18.8 GB (17.5 GiB), 4 safetensors shards |
| Context length | 262,144 (trained at 32,000) |

Architecture follows the Qwen3.5 multimodal stack: 32 language layers mixing
linear attention with full attention every 4th layer, hidden size 4096, 16
attention heads with 4 KV heads, plus a 27-layer SigLIP-style vision tower
(hidden size 1152, patch size 16) projecting to 4096.

## Training cost

| | |
|---|---|
| Hardware | 8 x NVIDIA RTX A6000 (48 GB), single node |
| Wall-clock time | 41.4 h (149,071 s) |
| GPU-hours | 331 A6000-hours |
| Throughput | 52.5 s per optimizer step, 0.61 samples/s |
| Peak memory | 25.4 GiB per GPU |

Peak memory stays low because DeepSpeed ZeRO-3 offloads both optimizer states
and parameters to CPU, which trades memory for the step time above.

## Training configuration

Full-parameter supervised fine-tuning of the language model on top of
[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), with the vision
encoder and the aligner frozen. Run with
[ms-swift](https://github.com/modelscope/ms-swift) 4.1.0.dev0 on transformers
5.3.0.

| Parameter | Value |
|---|---|
| Tuning type | full (`freeze_vit=True`, `freeze_aligner=True`, `freeze_llm=False`) |
| Epochs | 2 |
| Optimizer steps | 2,840 (1,420 per epoch) |
| Global batch size | 32 sequences (1 per device x 4 accumulation x 8 GPUs) |
| Learning rate | 1e-5, cosine schedule, warmup ratio 0.05 |
| Optimizer | `adamw_torch_fused`, betas (0.9, 0.95), weight decay 0.01 |
| Gradient clipping | 1.0 |
| Max sequence length | 32,000 |
| Precision | bfloat16 |
| Attention | FlashAttention |
| Memory | DeepSpeed ZeRO-3, CPU offload of optimizer and parameters, gradient checkpointing |
| Packing / padding-free | off |
| Template | `qwen3_5`, non-thinking prefix enabled |
| Seed | 42 |

### Data volume

| | |
|---|---|
| Samples | 45,416 |
| Tokens per sample | 3,335 mean, 2,971 std, 597 min, 31,769 max |
| Tokens per epoch | approximately 151 M |
| Tokens seen | approximately 303 M over 2 epochs |

The mixture combines trajectory-level judging targets, per-step analysis
targets, and tool-query targets, so that one model serves every stage of the
SeekJudge pipeline.

### Final training metrics

Training loss 0.246 and token accuracy 0.918 at step 2,840; mean loss over the
whole run 0.380.

## License

Apache-2.0, inherited from the Qwen3.5-9B base model. The SeekJudge codebase is
MIT-licensed.