| --- |
| base_model: Qwen/Qwen3.5-9B |
| base_model_relation: finetune |
| library_name: transformers |
| pipeline_tag: image-text-to-text |
| license: apache-2.0 |
| tags: |
| - qwen3.5 |
| - game2player |
| - visual-agent |
| - game-playing |
| - sft |
| model_name: Game2Player 9B |
| --- |
| |
| # Game2Player 9B |
|
|
| Game2Player 9B is a visual game-playing model fine-tuned from |
| [Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B). It takes game |
| screenshots, a task description and recent actions as input, then predicts the |
| next keyboard or mouse action. |
|
|
| This repository contains the complete model weights. Load it directly with |
| vLLM or Transformers. |
|
|
| **Game2Player: Scaling Gameplay Experience for Visual Agents through |
| Environment–Player Co-Development** |
|
|
| [Project](https://game2player.github.io/) · |
| [Code](https://github.com/game2player/Game2Player) · |
| [PlayerBench](https://github.com/game2player/Game2Player/tree/main/benchmark/playerbench) |
|
|
| ## Training |
|
|
| The model was trained on gameplay collected from code policies and VLM players |
| with Game2Player. Training updates the language model while keeping the vision |
| encoder and visual projector frozen. |
|
|
| | Setting | Value | |
| | --- | --- | |
| | Training examples | 1,564,236 | |
| | Training duration | One epoch, 9,237 optimizer steps | |
| | Maximum training sequence length | 25,000 tokens | |
| | Learning rate | 0.00001 with a cosine schedule | |
| | Weight precision | BF16 | |
|
|
| Dataset coming soon. |
|
|
| ## Input and output |
|
|
| Provide the game rules, available controls and task goal in the system message. |
| The user message contains the current screenshot and, when available, up to |
| 50 previous actions with their short notes and six previous screenshots. |
| Use at most seven images in total and about 1,024 visual tokens per image. |
|
|
| The model returns a `computer_use` action inside `<tool_call>` tags. For example: |
|
|
| ```text |
| <tool_call> |
| {"name":"computer_use","arguments":{"action":"press_key","key":"ArrowRight","reasoning":"Move toward the exit."}} |
| </tool_call> |
| ``` |
|
|
| The `reasoning` field contains a short action note. Longer thinking, when enabled, |
| is returned separately. Mouse coordinates use a normalized 0–1000 range and |
| action durations are in seconds. |
|
|
| ## Run the model |
|
|
| Install vLLM and start an OpenAI-compatible server: |
|
|
| ```bash |
| pip install vllm==0.29.0 requests |
| |
| vllm serve game2player/Game2Player-9B \ |
| --served-model-name game2player-9b \ |
| --dtype bfloat16 \ |
| --max-model-len 65536 \ |
| --max-num-seqs 8 \ |
| --max-num-batched-tokens 4096 \ |
| --limit-mm-per-prompt '{"image":7,"video":0}' \ |
| --mm-processor-kwargs '{"max_pixels":1048576}' \ |
| --reasoning-parser qwen3 |
| ``` |
|
|
| Save a screenshot as `screen.png` and send a request. Replace the example game |
| rules and task with those of the game you want to play. |
|
|
| ```python |
| import base64 |
| from pathlib import Path |
| import requests |
| |
| system_prompt = """You are an expert game agent. Choose the next action from the screenshot. |
| |
| # Game Rules |
| Move the character through the maze to reach the exit. Walls block movement. |
| |
| # Role and Controls |
| ArrowUp, ArrowDown, ArrowLeft and ArrowRight move the character. |
| |
| # Task Instruction |
| Reach the exit. |
| |
| # Output Format |
| Return exactly one action inside <tool_call> tags. |
| Use press_key with a key from the controls above, or wait with a duration in seconds. |
| Include reasoning with one short sentence explaining the action. |
| Think in the thinking channel, then emit only the tool call as the final answer. |
| Example: |
| <tool_call>{"name":"computer_use","arguments":{"action":"press_key","key":"ArrowRight","reasoning":"Move toward the exit."}}</tool_call> |
| """ |
| |
| encoded = base64.b64encode(Path("screen.png").read_bytes()).decode("ascii") |
| response = requests.post( |
| "http://localhost:8000/v1/chat/completions", |
| json={ |
| "model": "game2player-9b", |
| "messages": [ |
| {"role": "system", "content": system_prompt}, |
| {"role": "user", "content": [ |
| {"type": "text", "text": "# Current step\nCurrent game screen:\n"}, |
| {"type": "image_url", "image_url": { |
| "url": "data:image/png;base64," + encoded |
| }}, |
| {"type": "text", "text": "\nReturn the next action."}, |
| ]}, |
| ], |
| "temperature": 1.0, |
| "max_tokens": 512, |
| "chat_template_kwargs": {"enable_thinking": False}, |
| }, |
| timeout=180, |
| ) |
| response.raise_for_status() |
| print(response.json()["choices"][0]["message"]["content"]) |
| ``` |
|
|
| To enable thinking, set `enable_thinking` to `True` and increase `max_tokens` and |
| the request timeout. Our thinking evaluations allow up to 32,768 output tokens. |
| Keep previous action notes in the history and omit previous thinking chains. |
|
|
| ## Evaluation |
|
|
| We evaluate Game2Player on PlayerBench, which contains 50 held-out games and |
| 287 tasks, and GameWorld, which contains 34 games and 170 tasks. Results are |
| available on the [project leaderboard](https://game2player.github.io/#leaderboard). |
|
|
| ## License |
|
|
| The model weights are released under [Apache-2.0](LICENSE). |
| Qwen3.5 is also licensed under Apache-2.0. Game titles, assets and screenshots |
| belong to their respective owners. The model license does not grant rights to |
| third-party game content. |
|
|
