File size: 4,124 Bytes
adc02fa | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | # Simulator Backends
DoVLA-CIL uses a small simulator interface so the same CIL pipeline can run on a toy symbolic
backend today and real physics backends later.
## Interface
Every backend implements:
```python
seed(seed)
reset_task(task, scene=None)
serialize_state()
restore_state(state_blob)
render_observation()
get_symbolic_state()
execute_action_chunk(action)
close()
```
The critical requirement is exact reset. `serialize_state()` and `restore_state()` must restore the
same simulator state for every candidate action in a group.
## Registry
```python
from dovla_cil.sim.registry import list_backends, create_backend
print(list_backends()) # toy, maniskill, genesis
sim = create_backend("toy")
```
The registry lists optional backends without importing their heavy packages. Missing optional
dependencies raise helpful errors only when instantiated.
Shared config:
```yaml
sim:
backend: toy
seed: 0
params: {}
```
## Toy Backend
`toy` is a deterministic symbolic 2D tabletop backend. It supports:
- object positions
- robot end-effector and gripper state
- `move_to`, `grasp`, `release`, `push`, `place_at`, `open`, `close`
- exact pickle state serialization
- symbolic relations such as `inside`, `near`, `next_to`, `left_of`, `right_of`, `behind`,
`in_front_of`, `lifted`, `opened`, `closed`, and `grasped`
- simplified mass/friction effects for low-friction, heavy-object, and sticky-drawer stress tests
- out-of-workspace instability markers for irreversible-failure smoke cases
The toy backend is for smoke tests and local development only. It is not a physics benchmark and
does not justify real robot claims.
## ManiSkill3 Backend and Lattice Engine
`maniskill` lives in `dovla_cil/sim/maniskill_backend.py`. Importing the module does not require
ManiSkill3. Instantiating the backend checks for the package and raises an install hint if missing.
Config fields:
- `env_id`
- `obs_mode`
- `control_mode`
- `render_mode`
- `num_envs`
- `sim_backend`
Implementation checklist:
1. Install ManiSkill3 in the runtime environment.
2. Map each `TaskSpec.family` and predicate set to a ManiSkill environment and reset options.
3. Translate `SceneSpec` object poses, seeds, camera pose, and task metadata into environment reset.
4. Implement exact simulator and RNG serialization.
5. Translate `ActionChunk` values into the configured control mode.
6. Render observations and store large images by reference when needed.
7. Build `get_symbolic_state()` from object poses, articulation states, robot state, and contacts.
8. Return `RolloutResult` with before/after symbolic states, contacts, trajectory metadata, and
simulator diagnostics.
The generic `SimulatorBackend` wrapper remains a placeholder for arbitrary `TaskSpec` mapping.
The research path in `dovla_cil/generation/maniskill_lattice.py` is concrete: it restores official
ManiSkill HDF5 environment states, executes same-state action lattices with GPU PhysX, verifies
restore error, stores measured before/after states, and supports six explicit task profiles. RGB is
rendered later from those persisted states by `maniskill_render.py`, avoiding CUDA/Vulkan device
ordinal coupling on shared or MIG GPUs.
## Genesis Skeleton
`genesis` follows the same optional-dependency pattern in `dovla_cil/sim/genesis_backend.py`.
Config fields:
- `scene_backend`
- `render_mode`
- `num_envs`
- `dt`
- `substeps`
- `params`
Implementation checklist:
1. Install Genesis in the runtime environment.
2. Map task objects to bodies, articulations, materials, and robot assets.
3. Apply `SceneSpec` object poses, camera pose, lighting seed, physics seed, and metadata.
4. Replace placeholder pickle state with exact Genesis world, robot, and RNG serialization.
5. Translate `ActionChunk` values into robot controls or scripted skills.
6. Render observations and expose symbolic state for reward/effect extraction.
7. Return contacts, trajectory, before/after state, and simulator diagnostics.
The Genesis wrapper exists so large-scale code paths can discover the backend name and fail cleanly
until a real adapter is implemented.
|