docs/simulator_backends.md

Simulator Backends

DoVLA-CIL uses a small simulator interface so the same CIL pipeline can run on a toy symbolic backend today and real physics backends later.

Interface

Every backend implements:

seed(seed)
reset_task(task, scene=None)
serialize_state()
restore_state(state_blob)
render_observation()
get_symbolic_state()
execute_action_chunk(action)
close()

The critical requirement is exact reset. serialize_state() and restore_state() must restore the same simulator state for every candidate action in a group.

Registry

from dovla_cil.sim.registry import list_backends, create_backend

print(list_backends())       # toy, maniskill, genesis
sim = create_backend("toy")

The registry lists optional backends without importing their heavy packages. Missing optional dependencies raise helpful errors only when instantiated.

Shared config:

sim:
  backend: toy
  seed: 0
  params: {}

Toy Backend

toy is a deterministic symbolic 2D tabletop backend. It supports:

• object positions
• robot end-effector and gripper state
• move_to, grasp, release, push, place_at, open, close
• exact pickle state serialization
• symbolic relations such as inside, near, next_to, left_of, right_of, behind, in_front_of, lifted, opened, closed, and grasped
• simplified mass/friction effects for low-friction, heavy-object, and sticky-drawer stress tests
• out-of-workspace instability markers for irreversible-failure smoke cases

The toy backend is for smoke tests and local development only. It is not a physics benchmark and does not justify real robot claims.

ManiSkill3 Backend and Lattice Engine

maniskill lives in dovla_cil/sim/maniskill_backend.py. Importing the module does not require ManiSkill3. Instantiating the backend checks for the package and raises an install hint if missing.

Config fields:

• env_id
• obs_mode
• control_mode
• render_mode
• num_envs
• sim_backend

Implementation checklist:

1. Install ManiSkill3 in the runtime environment.
2. Map each TaskSpec.family and predicate set to a ManiSkill environment and reset options.
3. Translate SceneSpec object poses, seeds, camera pose, and task metadata into environment reset.
4. Implement exact simulator and RNG serialization.
5. Translate ActionChunk values into the configured control mode.
6. Render observations and store large images by reference when needed.
7. Build get_symbolic_state() from object poses, articulation states, robot state, and contacts.
8. Return RolloutResult with before/after symbolic states, contacts, trajectory metadata, and simulator diagnostics.

The generic SimulatorBackend wrapper remains a placeholder for arbitrary TaskSpec mapping. The research path in dovla_cil/generation/maniskill_lattice.py is concrete: it restores official ManiSkill HDF5 environment states, executes same-state action lattices with GPU PhysX, verifies restore error, stores measured before/after states, and supports six explicit task profiles. RGB is rendered later from those persisted states by maniskill_render.py, avoiding CUDA/Vulkan device ordinal coupling on shared or MIG GPUs.

Genesis Skeleton

genesis follows the same optional-dependency pattern in dovla_cil/sim/genesis_backend.py.

Config fields:

• scene_backend
• render_mode
• num_envs
• dt
• substeps
• params

Implementation checklist:

1. Install Genesis in the runtime environment.
2. Map task objects to bodies, articulations, materials, and robot assets.
3. Apply SceneSpec object poses, camera pose, lighting seed, physics seed, and metadata.
4. Replace placeholder pickle state with exact Genesis world, robot, and RNG serialization.
5. Translate ActionChunk values into robot controls or scripted skills.
6. Render observations and expose symbolic state for reward/effect extraction.
7. Return contacts, trajectory, before/after state, and simulator diagnostics.

The Genesis wrapper exists so large-scale code paths can discover the backend name and fail cleanly until a real adapter is implemented.