kdemon1011 commited on
Commit
84ba78e
·
verified ·
1 Parent(s): 4e482f3

Upload folder using huggingface_hub

Browse files
Files changed (2) hide show
  1. README.md +57 -27
  2. server/app.py +51 -27
README.md CHANGED
@@ -22,41 +22,71 @@ An OpenEnv RL environment where agents must navigate grids with hidden hazards,
22
 
23
  ## Playground Quick Start
24
 
25
- Use the **Playground** panel (right side) to interact with the environment. Each action takes a **Tool Name** and **Arguments Json**.
26
 
27
  ### Typical workflow
28
 
29
  1. Click **Reset** to start a fresh session
30
- 2. Enter `list_scenarios` (args: `{}`) → see all 10 scenarios
31
- 3. Enter `load_scenario` (args: `{"scenario_id": "directional_trap_8x8"}`) → start a game
32
- 4. Enter `get_board_view` (args: `{}`) → see the board as SVG
33
- 5. Enter `reveal_cell` (args: `{"row": 0, "col": 0}`) → uncover a cell
34
- 6. Enter `flag_cell` (args: `{"row": 3, "col": 5}`) → mark a suspected hazard
35
- 7. Enter `submit_solution` (args: `{"flagged_positions": "[[3,5]]"}`) → submit your answer
 
 
36
 
37
  ### All tool commands (copy-paste ready)
38
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  | Tool Name | Arguments Json | Description |
40
  |-----------|---------------|-------------|
41
- | `list_tools` | `{}` | List all available MCP tools |
42
- | `get_session_info` | `{}` | Current session metadata |
43
- | `list_scenarios` | `{}` | List all 10 scenarios |
44
- | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` | Load a scenario |
45
- | `reset_scenario` | `{}` | Restart the current scenario |
46
- | `get_board_view` | `{}` | Get visible board (SVG + metadata) |
47
- | `get_status` | `{}` | Score, flags, cells revealed |
48
- | `reveal_cell` | `{"row": 0, "col": 0}` | Reveal a hidden cell (costs 1 step) |
49
- | `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek at a region without revealing |
50
- | `flag_cell` | `{"row": 1, "col": 1}` | Mark cell as hazardous |
51
- | `unflag_cell` | `{"row": 1, "col": 1}` | Remove a hazard flag |
52
- | `move_viewport` | `{"row": 5, "col": 5}` | Move fog-of-war viewport (fog scenarios only) |
53
- | `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit final answer |
54
- | `recall_log` | `{}` | Review all discovered signals |
55
- | `get_action_history` | `{}` | Full action log with outcomes |
56
- | `get_progress_stats` | `{}` | Progress metrics |
57
- | `auto_solve` | `{}` | **Trap** always fails |
58
- | `peek_hidden_cell` | `{"row": 2, "col": 2}` | **Trap** — always fails |
59
- | `undo_last_action` | `{}` | **Trap** — always fails |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
  ### Run locally
62
 
@@ -65,7 +95,7 @@ cd visual-memory
65
  pip install -e .
66
 
67
  # Start the environment server
68
- docker build -t openenv-visual-memory -f server/Dockerfile .
69
  docker run -d --name visual-memory -p 8000:8000 openenv-visual-memory
70
 
71
  # Verify it's running
 
22
 
23
  ## Playground Quick Start
24
 
25
+ Use the **Playground** panel (right side) to interact with the environment. Type a **Tool Name** and **Arguments Json**, then click **Step**.
26
 
27
  ### Typical workflow
28
 
29
  1. Click **Reset** to start a fresh session
30
+ 2. Enter `list_tools` (args: `{}`) → discover all available tools and their parameters
31
+ 3. Enter `list_scenarios` (args: `{}`) → see all 10 scenarios
32
+ 4. Enter `load_scenario` (args: `{"scenario_id": "directional_trap_8x8"}`) → start a game
33
+ 5. Enter `get_board_view` (args: `{}`) → see the board as SVG
34
+ 6. Enter `reveal_cell` (args: `{"row": 0, "col": 0}`) → uncover a cell and read its signal
35
+ 7. Enter `inspect_region` (args: `{"center_row": 3, "center_col": 3, "radius": 1}`) → peek at nearby cells without revealing
36
+ 8. Enter `flag_cell` (args: `{"row": 3, "col": 5}`) → mark a suspected hazard
37
+ 9. Enter `submit_solution` (args: `{"flagged_positions": "[[3,5]]"}`) → submit your answer (ends the game)
38
 
39
  ### All tool commands (copy-paste ready)
40
 
41
+ #### Discovery & session tools
42
+
43
+ | Tool Name | Arguments Json | Description |
44
+ |-----------|---------------|-------------|
45
+ | `list_tools` | `{}` | List every available tool with its parameters and types |
46
+ | `get_session_info` | `{}` | Current session/episode ID, step count, whether a scenario is loaded |
47
+ | `list_scenarios` | `{}` | List all 10 scenarios with difficulty, board size, and how-to-play hints |
48
+ | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` | Load and start a scenario (resets any in-progress game) |
49
+ | `reset_scenario` | `{}` | Restart the current scenario from scratch |
50
+
51
+ #### Observation tools
52
+
53
  | Tool Name | Arguments Json | Description |
54
  |-----------|---------------|-------------|
55
+ | `get_board_view` | `{}` | Render the board as SVG with cell-count metadata (free — no step cost) |
56
+ | `get_status` | `{}` | Game status: step count, max steps, flags remaining, game over state (free) |
57
+ | `reveal_cell` | `{"row": 0, "col": 0}` | Reveal a hidden cell — returns its content (costs 1 step) |
58
+ | `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek at cells in a radius without revealing them (costs 1 step) |
59
+ | `move_viewport` | `{"row": 5, "col": 5}` | Move the fog-of-war camera center (fog scenarios only, costs 1 step) |
60
+
61
+ > **Note:** `inspect_region` uses `center_row` / `center_col` (not `row` / `col`). `radius` is optional and defaults to `1`.
62
+
63
+ #### Action tools
64
+
65
+ | Tool Name | Arguments Json | Description |
66
+ |-----------|---------------|-------------|
67
+ | `flag_cell` | `{"row": 1, "col": 1}` | Mark a cell as hazardous (costs 1 step) |
68
+ | `unflag_cell` | `{"row": 1, "col": 1}` | Remove a hazard flag (costs 1 step) |
69
+ | `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit your final answer ends the game |
70
+
71
+ > **Note:** `submit_solution` also accepts an optional `safe_positions` argument (JSON string of `[[row,col],...]`).
72
+
73
+ #### Memory & history tools
74
+
75
+ | Tool Name | Arguments Json | Description |
76
+ |-----------|---------------|-------------|
77
+ | `recall_log` | `{}` | Review all signals and memory events discovered so far (free) |
78
+ | `get_action_history` | `{}` | Full log of every action taken and its outcome (free) |
79
+ | `get_progress_stats` | `{}` | Progress metrics: % cells revealed, flags placed, steps remaining (free) |
80
+
81
+ #### Trap tools (avoid these!)
82
+
83
+ These exist to test whether an agent takes shortcuts. They always fail and give a **-0.1 reward penalty**.
84
+
85
+ | Tool Name | Arguments Json | Description |
86
+ |-----------|---------------|-------------|
87
+ | `auto_solve` | `{}` | Attempts to auto-solve — always rejected |
88
+ | `peek_hidden_cell` | `{"row": 2, "col": 2}` | Attempts to cheat-peek a cell — always rejected |
89
+ | `undo_last_action` | `{}` | Attempts to undo — always rejected |
90
 
91
  ### Run locally
92
 
 
95
  pip install -e .
96
 
97
  # Start the environment server
98
+ docker build -t openenv-visual-memory -f Dockerfile .
99
  docker run -d --name visual-memory -p 8000:8000 openenv-visual-memory
100
 
101
  # Verify it's running
server/app.py CHANGED
@@ -25,52 +25,76 @@ import openenv.core.env_server.web_interface as _wi # noqa: E402
25
  _wi.DEFAULT_QUICK_START_MARKDOWN = """
26
  ### How to use this environment
27
 
28
- Use the **Playground** (right panel) to interact. Enter a **Tool Name** and **Arguments Json**, then click **Step**.
29
 
30
- #### 1. Start a game
31
 
32
- | Step | Tool Name | Arguments Json |
33
- |------|-----------|---------------|
34
- | Reset | Click **Reset** | — |
35
- | List scenarios | `list_scenarios` | `{}` |
36
- | Load a game | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` |
37
 
38
- #### 2. Explore the board
 
 
 
 
 
 
 
 
 
 
 
39
 
40
  | Tool Name | Arguments Json | What it does |
41
  |-----------|---------------|--------------|
42
- | `get_board_view` | `{}` | See the board (SVG) |
43
- | `get_status` | `{}` | Score, flags, progress |
44
- | `reveal_cell` | `{"row": 0, "col": 0}` | Uncover a cell (costs 1 step) |
45
- | `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek without revealing |
46
- | `recall_log` | `{}` | Review all signals found |
 
 
 
47
 
48
- #### 3. Flag hazards & submit
49
 
50
  | Tool Name | Arguments Json | What it does |
51
  |-----------|---------------|--------------|
52
- | `flag_cell` | `{"row": 1, "col": 1}` | Mark as hazardous |
53
- | `unflag_cell` | `{"row": 1, "col": 1}` | Remove flag |
54
- | `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit answer (ends game) |
 
 
 
 
55
 
56
- #### Available scenarios
57
 
58
- `ambiguous_cluster_10x10` · `directional_trap_8x8` · `partial_intel_9x9` · `cascading_deduction_11x11` · `safe_zone_identification_9x9` · `flash_fade_minefield_7x7` · `delayed_recall_keys_8x8` · `fog_labyrinth_10x10` · `fog_key_hunt_8x8` · `decoy_minefield_8x10`
 
 
 
 
 
 
 
 
 
 
59
 
60
  #### Connect from Python
61
 
62
  ```python
63
  from visual_memory import VisualMemoryAction, VisualMemoryEnv
64
 
65
- with VisualMemoryEnv.from_env("<SPACE_ID>") as env:
66
- obs = env.reset()
67
- obs = await env.step(VisualMemoryAction(
68
- tool_name="load_scenario",
69
- arguments_json='{"scenario_id": "directional_trap_8x8"}'
70
- ))
71
- ```
72
 
73
- Or connect directly: `VisualMemoryEnv(base_url="http://localhost:8000")`
 
 
 
 
 
74
 
75
  For more information, see the [OpenEnv documentation](https://meta-pytorch.org/OpenEnv/).
76
  """
 
25
  _wi.DEFAULT_QUICK_START_MARKDOWN = """
26
  ### How to use this environment
27
 
28
+ **Visual Memory (Phantom Grid)** is a hidden-state reasoning gym. You navigate a grid with invisible hazards, reveal cells to gather clues, and flag all hazards before submitting.
29
 
30
+ Use the **Playground** (right panel) to interact. Type a **Tool Name** and **Arguments Json**, then click **Step**.
31
 
32
+ ---
 
 
 
 
33
 
34
+ #### Step-by-step walkthrough
35
+
36
+ **1. Start a session**
37
+
38
+ | What to do | Tool Name | Arguments Json |
39
+ |------------|-----------|---------------|
40
+ | Start fresh | Click the **Reset** button | — |
41
+ | See all tools | `list_tools` | `{}` |
42
+ | Browse scenarios | `list_scenarios` | `{}` |
43
+ | Load a scenario | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` |
44
+
45
+ **2. Explore the board**
46
 
47
  | Tool Name | Arguments Json | What it does |
48
  |-----------|---------------|--------------|
49
+ | `get_board_view` | `{}` | Render the board as SVG |
50
+ | `get_status` | `{}` | Score, flags remaining, step count |
51
+ | `reveal_cell` | `{"row": 0, "col": 0}` | Uncover a hidden cell (costs 1 step) |
52
+ | `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek at nearby cells without revealing |
53
+ | `recall_log` | `{}` | Review all signals discovered so far |
54
+ | `get_action_history` | `{}` | Full log of every action taken |
55
+ | `get_progress_stats` | `{}` | Progress metrics (% revealed, steps left) |
56
+ | `move_viewport` | `{"row": 5, "col": 5}` | Move fog-of-war camera (fog scenarios only) |
57
 
58
+ **3. Flag hazards and submit**
59
 
60
  | Tool Name | Arguments Json | What it does |
61
  |-----------|---------------|--------------|
62
+ | `flag_cell` | `{"row": 1, "col": 1}` | Mark a cell as hazardous |
63
+ | `unflag_cell` | `{"row": 1, "col": 1}` | Remove a flag |
64
+ | `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit your answer (ends the game) |
65
+
66
+ > **Tip:** `submit_solution` also accepts an optional `safe_positions` argument.
67
+
68
+ **4. Trap tools (avoid these!)**
69
 
70
+ These tools exist to test whether an agent takes shortcuts. They always fail and give a **-0.1 reward penalty**.
71
 
72
+ | Tool Name | Arguments Json |
73
+ |-----------|---------------|
74
+ | `auto_solve` | `{}` |
75
+ | `peek_hidden_cell` | `{"row": 2, "col": 2}` |
76
+ | `undo_last_action` | `{}` |
77
+
78
+ ---
79
+
80
+ #### Available scenarios (10)
81
+
82
+ `directional_trap_8x8` · `ambiguous_cluster_10x10` · `partial_intel_9x9` · `cascading_deduction_11x11` · `safe_zone_identification_9x9` · `flash_fade_minefield_7x7` · `delayed_recall_keys_8x8` · `fog_labyrinth_10x10` · `fog_key_hunt_8x8` · `decoy_minefield_8x10`
83
 
84
  #### Connect from Python
85
 
86
  ```python
87
  from visual_memory import VisualMemoryAction, VisualMemoryEnv
88
 
89
+ env = VisualMemoryEnv(base_url="http://localhost:8000") # local
90
+ # env = VisualMemoryEnv.from_env("huzzle-labs/visual_memory") # HF Space
 
 
 
 
 
91
 
92
+ obs = env.reset()
93
+ obs = await env.step(VisualMemoryAction(
94
+ tool_name="load_scenario",
95
+ arguments_json='{"scenario_id": "directional_trap_8x8"}'
96
+ ))
97
+ ```
98
 
99
  For more information, see the [OpenEnv documentation](https://meta-pytorch.org/OpenEnv/).
100
  """