language: en
tags:
- pytorch
- autonomous-driving
- occupancy-prediction
- lidar
LO-World
Pretrained checkpoints for joint future 3D occupancy and LiDAR prediction in a shared BEV latent space, with a GenieDrive transformer for six-camera video generation.
Checkpoints
| File | Model |
|---|---|
bev_world_6f.pth |
6-frame nuScenes World Model |
bev_world_12f_autoregressive.pth |
12-frame autoregressive World Model |
bev_world_12f_layout_autoregressive.pth |
12-frame autoregressive World Model with BEVLayout conditioning |
bev_world_6f_stc.pth |
6-frame World Model fine-tuned with STCOcc inputs |
bev_world_6f_waymo.pth |
6-frame Waymo World Model |
occ_lidar_vae.pth |
Joint occupancy-LiDAR tokenizer |
geniedrive_448x800.safetensors |
GenieDrive transformer for 448 × 800 six-camera video generation; requires the Wan base model (see Video generation) |
Run the following from the LO-World code repository with huggingface_hub installed in your Conda environment:
hf download JimmyLiaoo/LO-World \
bev_world_6f.pth bev_world_12f_autoregressive.pth \
bev_world_12f_layout_autoregressive.pth bev_world_6f_stc.pth \
bev_world_6f_waymo.pth occ_lidar_vae.pth \
geniedrive_448x800.safetensors SHA256SUMS --local-dir ckpts
(cd ckpts && sha256sum -c SHA256SUMS)
Weights are saved to ckpts/. Use the matching configuration and data preparation instructions in the code repository. World Model checkpoints include tokenizer parameters.
Video generation
The released GenieDrive transformer generates six-camera videos at 448 × 800. Install the loworld-genie environment and prepare scene RGB/semantic conditions using the LO-World code repository.
Download the transformer and its required Wan base model from the repository root:
hf download JimmyLiaoo/LO-World geniedrive_448x800.safetensors --local-dir ckpts
hf download alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control \
--local-dir ckpts/geniedrive/Wan2.1-Fun-V1.1-1.3B-Control
The transformer is saved to ckpts/geniedrive_448x800.safetensors, matching the default inference path. Generate videos for a prepared scene:
conda run --no-capture-output -n loworld-genie python \
tools/genie/inference/generate_scene.py \
--scene-dir work_dirs/visualization/scene_0001
Output: six camera MP4s and a surround-view GIF under work_dirs/visualization/scene_0001/video/.
Data downloads
Download nuScenes train/validation annotation infos to data/nus_infos/:
hf download JimmyLiaoo/LO-World --include "nus_infos/*.pkl" --local-dir data
Download and extract 2 Hz BEVLayout arrays to data/bev_layout/ for Layout-conditioned models. Extraction requires tar with Zstandard support:
hf download JimmyLiaoo/LO-World \
data/bev_layout/2hz/train.tar.zst data/bev_layout/2hz/val.tar.zst --local-dir .
tar --zstd -xf data/bev_layout/2hz/train.tar.zst -C data/bev_layout
tar --zstd -xf data/bev_layout/2hz/val.tar.zst -C data/bev_layout
Optional 12 Hz condition packages provide one 81-frame clip per scene for 700 training and 150 validation scenes, sampled at uniform physical times:
hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_*_81f/**" --local-dir .
Outputs: data/bev_layout/12hz_train700_81f/ and data/bev_layout/12hz_val150_81f/. See the training and validation package cards for usage.
Original nuScenes sensor data and Occ3D labels are required separately. STCOcc models also require STCOcc predictions encoded with the released tokenizer.
BEVLayout data is derived from nuScenes and is for non-commercial research under the nuScenes terms. See the data card for details.