File size: 2,527 Bytes
3cd1076 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | # Waymo End-to-End Dataset Layout (Fast-dDrive)
Fast-dDrive trains and evaluates on the Waymo End-to-End Driving (WOD-E2E)
dataset. We do **not** redistribute Waymo data — follow the official process and
preprocess it into the JSON layout below.
## 1. Obtain the raw data
1. Apply for access to Waymo Open Dataset: https://waymo.com/open/
2. Download the WOD-E2E `train` and `val` tfrecord shards.
3. Optional: pre-compute the GT pickle for evaluation (see below).
## 2. Convert to Fast-dDrive JSON
Each training / val sample is a JSON object roughly of the form:
```json
{
"id": "scene_uuid",
"image": [
"waymo_train/scene_uuid/frame_0.jpg",
"waymo_train/scene_uuid/frame_1.jpg",
"..."
],
"conversations": [
{"from": "human", "value": "<image>...<image>Drive task prompt..."},
{"from": "gpt", "value": "{\"critical_objects\": {...}, \"future_meta_behavior\": {...}, \"trajectory\": \"[[+003.30,-000.01], ...]\", \"explanation\": \"...\"}"}
]
}
```
Notes:
- The model output is a JSON-shaped string; trajectory waypoints use the
`+XXX.XX,-XXX.XX` zero-padded format (parsed by `eval/batch_inference.py`).
A reference preprocessing script will be added under `fast_ddrive/data/`. For
now, point the training scripts at your own conversion via env vars:
```bash
export DATASET_JSON=/path/to/waymo_train.json
export IMAGE_FOLDER=/path/to/image/root # JSON image paths are relative to this
export EVAL_JSON=/path/to/waymo_val.json
export IMAGE_ROOT=/path/to/image/root # same root used for eval
```
## 3. Official Waymo ADE / RFS evaluation
`eval/evaluate_waymo_metrics.py` computes ADE@3s, ADE@5s and Rater Feedback
Score (RFS). It requires `tensorflow` + `waymo-open-dataset-tf-2-12-0`, which
conflict with the inference stack — install in a separate conda env (we call
ours `autovla`):
```bash
conda create -n autovla python=3.10
conda activate autovla
pip install tensorflow==2.12.0 waymo-open-dataset-tf-2-12-0 numpy tqdm
python eval/evaluate_waymo_metrics.py \
--pred_json /path/to/predictions.json \
--gt_tfrecords "/path/to/val*.tfrecord*" \
--output_dir results/
```
For faster eval on compute nodes, cache the GT dict once:
```bash
python eval/evaluate_waymo_metrics.py \
--pred_json /path/to/predictions.json \
--save_gt_dict_pkl gt_dict_val.pkl \
--gt_tfrecords "/path/to/val*.tfrecord*"
# subsequent runs:
python eval/evaluate_waymo_metrics.py \
--pred_json /path/to/predictions.json \
--gt_dict_pkl gt_dict_val.pkl
```
|