Create README.md

README.md

@@ -0,0 +1,112 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3.5-9B
+library_name: transformers
+pipeline_tag: image-text-to-text
+tags:
+  - computer-use-agent
+  - gui-agent
+  - reward-model
+  - llm-as-a-judge
+  - reinforcement-learning
+---
+
+# SeekJudge-9B
+
+SeekJudge-9B is the trained judge model of **SeekJudge**, a reward framework for
+reinforcement learning in computer-use agents. Given a task goal and a recorded
+trajectory (screenshots plus actions), it decides whether the goal was achieved
+and emits per-step quality labels in the `<answer_overall>` / `<answer_step>`
+format.
+
+The model is a judge, not an agent. It does not produce GUI actions.
+
+## Code and resources
+
+| | |
+|---|---|
+| Code | [github.com/ZJUSCL/SeekJudge](https://github.com/ZJUSCL/SeekJudge) |
+| Benchmark | [ZJUSCL/CUAStepBench](https://huggingface.co/datasets/ZJUSCL/CUAStepBench) |
+| Leaderboard | [github.com/ZJUSCL/CUAStepBench](https://github.com/ZJUSCL/CUAStepBench) |
+
+The repository README covers serving the model, running it as a batch judge
+over trajectory datasets, and hosting it as an HTTP reward server for online RL
+training. This checkpoint requires `seek.trained: True` in the SeekJudge
+configuration, which selects the parser for the trained output format.
+
+## Model size
+
+| | |
+|---|---|
+| Total parameters | 9.41 B (9,409.81 M) |
+| Trainable during SFT | 8.95 B (8,953.80 M, 95.15 %) |
+| Frozen | vision encoder and aligner (456 M) |
+| Precision | bfloat16 |
+| Checkpoint on disk | 18.8 GB (17.5 GiB), 4 safetensors shards |
+| Context length | 262,144 (trained at 32,000) |
+
+Architecture follows the Qwen3.5 multimodal stack: 32 language layers mixing
+linear attention with full attention every 4th layer, hidden size 4096, 16
+attention heads with 4 KV heads, plus a 27-layer SigLIP-style vision tower
+(hidden size 1152, patch size 16) projecting to 4096.
+
+## Training cost
+
+| | |
+|---|---|
+| Hardware | 8 x NVIDIA RTX A6000 (48 GB), single node |
+| Wall-clock time | 41.4 h (149,071 s) |
+| GPU-hours | 331 A6000-hours |
+| Throughput | 52.5 s per optimizer step, 0.61 samples/s |
+| Peak memory | 25.4 GiB per GPU |
+
+Peak memory stays low because DeepSpeed ZeRO-3 offloads both optimizer states
+and parameters to CPU, which trades memory for the step time above.
+
+## Training configuration
+
+Full-parameter supervised fine-tuning of the language model on top of
+[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), with the vision
+encoder and the aligner frozen. Run with
+[ms-swift](https://github.com/modelscope/ms-swift) 4.1.0.dev0 on transformers
+5.3.0.
+
+| Parameter | Value |
+|---|---|
+| Tuning type | full (`freeze_vit=True`, `freeze_aligner=True`, `freeze_llm=False`) |
+| Epochs | 2 |
+| Optimizer steps | 2,840 (1,420 per epoch) |
+| Global batch size | 32 sequences (1 per device x 4 accumulation x 8 GPUs) |
+| Learning rate | 1e-5, cosine schedule, warmup ratio 0.05 |
+| Optimizer | `adamw_torch_fused`, betas (0.9, 0.95), weight decay 0.01 |
+| Gradient clipping | 1.0 |
+| Max sequence length | 32,000 |
+| Precision | bfloat16 |
+| Attention | FlashAttention |
+| Memory | DeepSpeed ZeRO-3, CPU offload of optimizer and parameters, gradient checkpointing |
+| Packing / padding-free | off |
+| Template | `qwen3_5`, non-thinking prefix enabled |
+| Seed | 42 |
+
+### Data volume
+
+| | |
+|---|---|
+| Samples | 45,416 |
+| Tokens per sample | 3,335 mean, 2,971 std, 597 min, 31,769 max |
+| Tokens per epoch | approximately 151 M |
+| Tokens seen | approximately 303 M over 2 epochs |
+
+The mixture combines trajectory-level judging targets, per-step analysis
+targets, and tool-query targets, so that one model serves every stage of the
+SeekJudge pipeline.
+
+### Final training metrics
+
+Training loss 0.246 and token accuracy 0.918 at step 2,840; mean loss over the
+whole run 0.380.
+
+## License
+
+Apache-2.0, inherited from the Qwen3.5-9B base model. The SeekJudge codebase is
+MIT-licensed.