README.md

---
license: apache-2.0
library_name: pytorch
tags:
  - robotics
  - world action model
  - imitation-learning
  - vision-language-action
  - robocasa
  - lda
---

# LDA Robocasa Model

This repository provides the LDA Robocasa checkpoint and the auxiliary files required for inference with the LDA codebase.

- GitHub: [https://github.com/jiangranlv/LDA-1B](https://github.com/jiangranlv/LDA-1B)
- Project Page: [https://pku-epic.github.io/LDA/](https://pku-epic.github.io/LDA/)

## Files

The Hugging Face model repository contains:

```text
LDA-robocasa.pt
config.yaml
dataset_statistics.json
```

- `LDA-robocasa.pt`: PyTorch checkpoint weights.
- `config.yaml`: model configuration used to rebuild the LDA framework.
- `dataset_statistics.json`: dataset normalization statistics used to un-normalize predicted actions during inference.

## Required Local Directory Layout

The current LDA loader expects the `.pt` checkpoint to be placed inside a subdirectory, usually named `checkpoints`, while `config.yaml` and `dataset_statistics.json` must stay in the parent run directory.

After downloading the files, organize them locally as:

```text
LDA-robocasa/
|-- config.yaml
|-- dataset_statistics.json
`-- checkpoints/
    `-- LDA-robocasa.pt
```

The checkpoint path passed to LDA should be:

```text
LDA-robocasa/checkpoints/LDA-robocasa.pt
```

## Why This Layout Is Needed

`baseframework.from_pretrained()` loads the checkpoint path and infers the run directory from it:

```python
checkpoint_pt = Path(pretrained_checkpoint)
run_dir = checkpoint_pt.parents[1]
```

For example, if the checkpoint path is:

```text
LDA-robocasa/checkpoints/LDA-robocasa.pt
```

then the inferred run directory is:

```text
LDA-robocasa
```

The loader then expects to find:

```text
LDA-robocasa/config.yaml
LDA-robocasa/dataset_statistics.json
```

If `LDA-robocasa.pt` is placed directly next to `config.yaml` and `dataset_statistics.json`, the loader will infer the wrong parent directory and fail to find the required files.

## Download And Prepare

You can download the model repository with `huggingface_hub`:

```python
from pathlib import Path
import shutil

from huggingface_hub import snapshot_download

repo_dir = Path(snapshot_download(repo_id="YOUR_ORG_OR_USERNAME/LDA-robocasa"))

ckpt_dir = repo_dir / "checkpoints"
ckpt_dir.mkdir(exist_ok=True)

src_ckpt = repo_dir / "LDA-robocasa.pt"
dst_ckpt = ckpt_dir / "LDA-robocasa.pt"

if src_ckpt.exists() and not dst_ckpt.exists():
    shutil.move(str(src_ckpt), str(dst_ckpt))

print("Checkpoint path:", dst_ckpt)
```

Replace `YOUR_ORG_OR_USERNAME/LDA-robocasa` with the actual Hugging Face repository ID.

## Load The Model

```python
from lda.model.framework.base_framework import baseframework

ckpt_path = "LDA-robocasa/checkpoints/LDA-robocasa.pt"

model = baseframework.from_pretrained(ckpt_path)
model = model.to("cuda").eval()
```

## Start The Policy Server

From the LDA repository root, run:

```bash
python deployment/model_server/server_policy.py \
  --ckpt_path LDA-robocasa/checkpoints/LDA-robocasa.pt \
  --port 10093 \
  --use_bf16
```

## Run RoboCasa Evaluation

In a separate terminal with the RoboCasa environment activated, run:

```bash
export PYTHONPATH=$(pwd):${PYTHONPATH}

python examples/Robocasa_tabletop/eval_files/simulation_env.py \
  --args.env_name ${env_name} \
  --args.port 10093 \
  --args.n_episodes 50 \
  --args.n_envs 1 \
  --args.max_episode_steps 720 \
  --args.n_action_steps 12 \
  --args.video_out_path ${video_out_path} \
  --args.pretrained_path LDA-robocasa/checkpoints/LDA-robocasa.pt
```

You can also use the batch evaluation script:

```bash
bash examples/Robocasa_tabletop/eval_files/batch_eval_args.sh
```

Make sure the checkpoint path used by the script points to:

```text
LDA-robocasa/checkpoints/LDA-robocasa.pt
```

## Required Files Checklist

Before running inference, confirm that the following files exist:

```text
LDA-robocasa/config.yaml
LDA-robocasa/dataset_statistics.json
LDA-robocasa/checkpoints/LDA-robocasa.pt
```

The checkpoint file must:

- exist locally
- use the `.pt` suffix
- be placed one directory below the run directory

The config and statistics files must:

- be named exactly `config.yaml` and `dataset_statistics.json`
- be located in the inferred run directory
- correspond to the same training run as the checkpoint

## Troubleshooting

### Missing `config.yaml`

If you see an error similar to:

```text
Missing `config.yaml`
```

check that your local directory is organized as:

```text
LDA-robocasa/
|-- config.yaml
|-- dataset_statistics.json
`-- checkpoints/
    `-- LDA-robocasa.pt
```

and that you pass:

```text
LDA-robocasa/checkpoints/LDA-robocasa.pt
```

instead of:

```text
LDA-robocasa/LDA-robocasa.pt
```

### Missing `dataset_statistics.json`

If you see an error similar to:

```text
Missing `dataset_statistics.json`
```

make sure `dataset_statistics.json` is in the same directory as `config.yaml`, not inside the `checkpoints` directory.

### Invalid Checkpoint Suffix

The loader asserts that the checkpoint suffix is `.pt`. Make sure the checkpoint file is named:

```text
LDA-robocasa.pt
```

## Notes

`dataset_statistics.json` is required for action un-normalization. Removing or replacing it can cause predicted actions to be scaled incorrectly.

`config.yaml` is required because the LDA framework is rebuilt from the saved configuration before loading the checkpoint weights.