Spaces:
Sleeping
Sleeping
Upload folder using huggingface_hub
Browse files- README.md +57 -27
- server/app.py +51 -27
README.md
CHANGED
|
@@ -22,41 +22,71 @@ An OpenEnv RL environment where agents must navigate grids with hidden hazards,
|
|
| 22 |
|
| 23 |
## Playground Quick Start
|
| 24 |
|
| 25 |
-
Use the **Playground** panel (right side) to interact with the environment.
|
| 26 |
|
| 27 |
### Typical workflow
|
| 28 |
|
| 29 |
1. Click **Reset** to start a fresh session
|
| 30 |
-
2. Enter `
|
| 31 |
-
3. Enter `
|
| 32 |
-
4. Enter `
|
| 33 |
-
5. Enter `
|
| 34 |
-
6. Enter `
|
| 35 |
-
7. Enter `
|
|
|
|
|
|
|
| 36 |
|
| 37 |
### All tool commands (copy-paste ready)
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
| Tool Name | Arguments Json | Description |
|
| 40 |
|-----------|---------------|-------------|
|
| 41 |
-
| `
|
| 42 |
-
| `
|
| 43 |
-
| `
|
| 44 |
-
| `
|
| 45 |
-
| `
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
|
| 52 |
-
|
|
| 53 |
-
| `
|
| 54 |
-
| `
|
| 55 |
-
| `
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
### Run locally
|
| 62 |
|
|
@@ -65,7 +95,7 @@ cd visual-memory
|
|
| 65 |
pip install -e .
|
| 66 |
|
| 67 |
# Start the environment server
|
| 68 |
-
docker build -t openenv-visual-memory -f
|
| 69 |
docker run -d --name visual-memory -p 8000:8000 openenv-visual-memory
|
| 70 |
|
| 71 |
# Verify it's running
|
|
|
|
| 22 |
|
| 23 |
## Playground Quick Start
|
| 24 |
|
| 25 |
+
Use the **Playground** panel (right side) to interact with the environment. Type a **Tool Name** and **Arguments Json**, then click **Step**.
|
| 26 |
|
| 27 |
### Typical workflow
|
| 28 |
|
| 29 |
1. Click **Reset** to start a fresh session
|
| 30 |
+
2. Enter `list_tools` (args: `{}`) → discover all available tools and their parameters
|
| 31 |
+
3. Enter `list_scenarios` (args: `{}`) → see all 10 scenarios
|
| 32 |
+
4. Enter `load_scenario` (args: `{"scenario_id": "directional_trap_8x8"}`) → start a game
|
| 33 |
+
5. Enter `get_board_view` (args: `{}`) → see the board as SVG
|
| 34 |
+
6. Enter `reveal_cell` (args: `{"row": 0, "col": 0}`) → uncover a cell and read its signal
|
| 35 |
+
7. Enter `inspect_region` (args: `{"center_row": 3, "center_col": 3, "radius": 1}`) → peek at nearby cells without revealing
|
| 36 |
+
8. Enter `flag_cell` (args: `{"row": 3, "col": 5}`) → mark a suspected hazard
|
| 37 |
+
9. Enter `submit_solution` (args: `{"flagged_positions": "[[3,5]]"}`) → submit your answer (ends the game)
|
| 38 |
|
| 39 |
### All tool commands (copy-paste ready)
|
| 40 |
|
| 41 |
+
#### Discovery & session tools
|
| 42 |
+
|
| 43 |
+
| Tool Name | Arguments Json | Description |
|
| 44 |
+
|-----------|---------------|-------------|
|
| 45 |
+
| `list_tools` | `{}` | List every available tool with its parameters and types |
|
| 46 |
+
| `get_session_info` | `{}` | Current session/episode ID, step count, whether a scenario is loaded |
|
| 47 |
+
| `list_scenarios` | `{}` | List all 10 scenarios with difficulty, board size, and how-to-play hints |
|
| 48 |
+
| `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` | Load and start a scenario (resets any in-progress game) |
|
| 49 |
+
| `reset_scenario` | `{}` | Restart the current scenario from scratch |
|
| 50 |
+
|
| 51 |
+
#### Observation tools
|
| 52 |
+
|
| 53 |
| Tool Name | Arguments Json | Description |
|
| 54 |
|-----------|---------------|-------------|
|
| 55 |
+
| `get_board_view` | `{}` | Render the board as SVG with cell-count metadata (free — no step cost) |
|
| 56 |
+
| `get_status` | `{}` | Game status: step count, max steps, flags remaining, game over state (free) |
|
| 57 |
+
| `reveal_cell` | `{"row": 0, "col": 0}` | Reveal a hidden cell — returns its content (costs 1 step) |
|
| 58 |
+
| `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek at cells in a radius without revealing them (costs 1 step) |
|
| 59 |
+
| `move_viewport` | `{"row": 5, "col": 5}` | Move the fog-of-war camera center (fog scenarios only, costs 1 step) |
|
| 60 |
+
|
| 61 |
+
> **Note:** `inspect_region` uses `center_row` / `center_col` (not `row` / `col`). `radius` is optional and defaults to `1`.
|
| 62 |
+
|
| 63 |
+
#### Action tools
|
| 64 |
+
|
| 65 |
+
| Tool Name | Arguments Json | Description |
|
| 66 |
+
|-----------|---------------|-------------|
|
| 67 |
+
| `flag_cell` | `{"row": 1, "col": 1}` | Mark a cell as hazardous (costs 1 step) |
|
| 68 |
+
| `unflag_cell` | `{"row": 1, "col": 1}` | Remove a hazard flag (costs 1 step) |
|
| 69 |
+
| `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit your final answer — ends the game |
|
| 70 |
+
|
| 71 |
+
> **Note:** `submit_solution` also accepts an optional `safe_positions` argument (JSON string of `[[row,col],...]`).
|
| 72 |
+
|
| 73 |
+
#### Memory & history tools
|
| 74 |
+
|
| 75 |
+
| Tool Name | Arguments Json | Description |
|
| 76 |
+
|-----------|---------------|-------------|
|
| 77 |
+
| `recall_log` | `{}` | Review all signals and memory events discovered so far (free) |
|
| 78 |
+
| `get_action_history` | `{}` | Full log of every action taken and its outcome (free) |
|
| 79 |
+
| `get_progress_stats` | `{}` | Progress metrics: % cells revealed, flags placed, steps remaining (free) |
|
| 80 |
+
|
| 81 |
+
#### Trap tools (avoid these!)
|
| 82 |
+
|
| 83 |
+
These exist to test whether an agent takes shortcuts. They always fail and give a **-0.1 reward penalty**.
|
| 84 |
+
|
| 85 |
+
| Tool Name | Arguments Json | Description |
|
| 86 |
+
|-----------|---------------|-------------|
|
| 87 |
+
| `auto_solve` | `{}` | Attempts to auto-solve — always rejected |
|
| 88 |
+
| `peek_hidden_cell` | `{"row": 2, "col": 2}` | Attempts to cheat-peek a cell — always rejected |
|
| 89 |
+
| `undo_last_action` | `{}` | Attempts to undo — always rejected |
|
| 90 |
|
| 91 |
### Run locally
|
| 92 |
|
|
|
|
| 95 |
pip install -e .
|
| 96 |
|
| 97 |
# Start the environment server
|
| 98 |
+
docker build -t openenv-visual-memory -f Dockerfile .
|
| 99 |
docker run -d --name visual-memory -p 8000:8000 openenv-visual-memory
|
| 100 |
|
| 101 |
# Verify it's running
|
server/app.py
CHANGED
|
@@ -25,52 +25,76 @@ import openenv.core.env_server.web_interface as _wi # noqa: E402
|
|
| 25 |
_wi.DEFAULT_QUICK_START_MARKDOWN = """
|
| 26 |
### How to use this environment
|
| 27 |
|
| 28 |
-
|
| 29 |
|
| 30 |
-
|
| 31 |
|
| 32 |
-
|
| 33 |
-
|------|-----------|---------------|
|
| 34 |
-
| Reset | Click **Reset** | — |
|
| 35 |
-
| List scenarios | `list_scenarios` | `{}` |
|
| 36 |
-
| Load a game | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` |
|
| 37 |
|
| 38 |
-
####
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
|
| 40 |
| Tool Name | Arguments Json | What it does |
|
| 41 |
|-----------|---------------|--------------|
|
| 42 |
-
| `get_board_view` | `{}` |
|
| 43 |
-
| `get_status` | `{}` | Score, flags,
|
| 44 |
-
| `reveal_cell` | `{"row": 0, "col": 0}` | Uncover a cell (costs 1 step) |
|
| 45 |
-
| `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek without revealing |
|
| 46 |
-
| `recall_log` | `{}` | Review all signals
|
|
|
|
|
|
|
|
|
|
| 47 |
|
| 48 |
-
|
| 49 |
|
| 50 |
| Tool Name | Arguments Json | What it does |
|
| 51 |
|-----------|---------------|--------------|
|
| 52 |
-
| `flag_cell` | `{"row": 1, "col": 1}` | Mark as hazardous |
|
| 53 |
-
| `unflag_cell` | `{"row": 1, "col": 1}` | Remove flag |
|
| 54 |
-
| `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit answer (ends game) |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
-
|
| 57 |
|
| 58 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
#### Connect from Python
|
| 61 |
|
| 62 |
```python
|
| 63 |
from visual_memory import VisualMemoryAction, VisualMemoryEnv
|
| 64 |
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
obs = await env.step(VisualMemoryAction(
|
| 68 |
-
tool_name="load_scenario",
|
| 69 |
-
arguments_json='{"scenario_id": "directional_trap_8x8"}'
|
| 70 |
-
))
|
| 71 |
-
```
|
| 72 |
|
| 73 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
|
| 75 |
For more information, see the [OpenEnv documentation](https://meta-pytorch.org/OpenEnv/).
|
| 76 |
"""
|
|
|
|
| 25 |
_wi.DEFAULT_QUICK_START_MARKDOWN = """
|
| 26 |
### How to use this environment
|
| 27 |
|
| 28 |
+
**Visual Memory (Phantom Grid)** is a hidden-state reasoning gym. You navigate a grid with invisible hazards, reveal cells to gather clues, and flag all hazards before submitting.
|
| 29 |
|
| 30 |
+
Use the **Playground** (right panel) to interact. Type a **Tool Name** and **Arguments Json**, then click **Step**.
|
| 31 |
|
| 32 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
+
#### Step-by-step walkthrough
|
| 35 |
+
|
| 36 |
+
**1. Start a session**
|
| 37 |
+
|
| 38 |
+
| What to do | Tool Name | Arguments Json |
|
| 39 |
+
|------------|-----------|---------------|
|
| 40 |
+
| Start fresh | Click the **Reset** button | — |
|
| 41 |
+
| See all tools | `list_tools` | `{}` |
|
| 42 |
+
| Browse scenarios | `list_scenarios` | `{}` |
|
| 43 |
+
| Load a scenario | `load_scenario` | `{"scenario_id": "directional_trap_8x8"}` |
|
| 44 |
+
|
| 45 |
+
**2. Explore the board**
|
| 46 |
|
| 47 |
| Tool Name | Arguments Json | What it does |
|
| 48 |
|-----------|---------------|--------------|
|
| 49 |
+
| `get_board_view` | `{}` | Render the board as SVG |
|
| 50 |
+
| `get_status` | `{}` | Score, flags remaining, step count |
|
| 51 |
+
| `reveal_cell` | `{"row": 0, "col": 0}` | Uncover a hidden cell (costs 1 step) |
|
| 52 |
+
| `inspect_region` | `{"center_row": 3, "center_col": 3, "radius": 1}` | Peek at nearby cells without revealing |
|
| 53 |
+
| `recall_log` | `{}` | Review all signals discovered so far |
|
| 54 |
+
| `get_action_history` | `{}` | Full log of every action taken |
|
| 55 |
+
| `get_progress_stats` | `{}` | Progress metrics (% revealed, steps left) |
|
| 56 |
+
| `move_viewport` | `{"row": 5, "col": 5}` | Move fog-of-war camera (fog scenarios only) |
|
| 57 |
|
| 58 |
+
**3. Flag hazards and submit**
|
| 59 |
|
| 60 |
| Tool Name | Arguments Json | What it does |
|
| 61 |
|-----------|---------------|--------------|
|
| 62 |
+
| `flag_cell` | `{"row": 1, "col": 1}` | Mark a cell as hazardous |
|
| 63 |
+
| `unflag_cell` | `{"row": 1, "col": 1}` | Remove a flag |
|
| 64 |
+
| `submit_solution` | `{"flagged_positions": "[[0,1],[2,3]]"}` | Submit your answer (ends the game) |
|
| 65 |
+
|
| 66 |
+
> **Tip:** `submit_solution` also accepts an optional `safe_positions` argument.
|
| 67 |
+
|
| 68 |
+
**4. Trap tools (avoid these!)**
|
| 69 |
|
| 70 |
+
These tools exist to test whether an agent takes shortcuts. They always fail and give a **-0.1 reward penalty**.
|
| 71 |
|
| 72 |
+
| Tool Name | Arguments Json |
|
| 73 |
+
|-----------|---------------|
|
| 74 |
+
| `auto_solve` | `{}` |
|
| 75 |
+
| `peek_hidden_cell` | `{"row": 2, "col": 2}` |
|
| 76 |
+
| `undo_last_action` | `{}` |
|
| 77 |
+
|
| 78 |
+
---
|
| 79 |
+
|
| 80 |
+
#### Available scenarios (10)
|
| 81 |
+
|
| 82 |
+
`directional_trap_8x8` · `ambiguous_cluster_10x10` · `partial_intel_9x9` · `cascading_deduction_11x11` · `safe_zone_identification_9x9` · `flash_fade_minefield_7x7` · `delayed_recall_keys_8x8` · `fog_labyrinth_10x10` · `fog_key_hunt_8x8` · `decoy_minefield_8x10`
|
| 83 |
|
| 84 |
#### Connect from Python
|
| 85 |
|
| 86 |
```python
|
| 87 |
from visual_memory import VisualMemoryAction, VisualMemoryEnv
|
| 88 |
|
| 89 |
+
env = VisualMemoryEnv(base_url="http://localhost:8000") # local
|
| 90 |
+
# env = VisualMemoryEnv.from_env("huzzle-labs/visual_memory") # HF Space
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
|
| 92 |
+
obs = env.reset()
|
| 93 |
+
obs = await env.step(VisualMemoryAction(
|
| 94 |
+
tool_name="load_scenario",
|
| 95 |
+
arguments_json='{"scenario_id": "directional_trap_8x8"}'
|
| 96 |
+
))
|
| 97 |
+
```
|
| 98 |
|
| 99 |
For more information, see the [OpenEnv documentation](https://meta-pytorch.org/OpenEnv/).
|
| 100 |
"""
|