RAP-WAM: RoboDojo base checkpoint

Paper · Project page · Public training code

This repository contains the retained RoboDojo base policy at training step 742500, exported for inference without changing the saved model tensor values or dtypes. RAP-WAM learns joint action and future-observation prediction through predictive rehearsal and action perturbations, starting from the video-pretrained Wan2.2-TI2V-5B backbone.

This is a single base checkpoint, not the task-conditioned collection of retained specialists used for the complete research system. Specialist checkpoints, agent-guided post-training and task-policy selection are not included. The paper's aggregate results must not be attributed to this bundle without evaluating it under the corresponding protocol.

Supported interface

Item	Value
Benchmark / embodiment	RoboDojo / ARX X5 dual arm (arx_x5)
Actions	14 absolute joint/gripper targets; left arm and gripper, then right arm and gripper
Observations	Head RGB plus left/right wrist RGB, and the matching robot state
Image layout	Head resized to 320 × 256 above two 160 × 128 wrist views; final height × width 384 × 320
Language	Original ordered table of 35 training instructions; UMT5 encoding for unseen instructions
Prediction horizon	16 action steps
Denoising at evaluation	30 steps
Replanning	Execute 8 actions per replan
Smoothing	Savitzky–Golay, window 9, polynomial order 2, over the full predicted horizon
Boundary blending	4 actions, using the previous pre-blend smoothed continuation

The 35 instructions identify training language contexts, not 35 reported evaluation units. Language lookup is case-insensitive and trims surrounding whitespace. Unseen instructions use the original UMT5 path instead of being assigned to an arbitrary known task. Episode reset clears observation and blending state while preserving the historical continuous policy random-number stream.

The architecture section of inference_config.json retains the constructor's sampling default. Evaluation uses the 30-step value in its protocol section. A compatible deployment wrapper must implement the complete observation, normalization, replanning, smoothing and blending protocol, not just load model.pt.

Download and integrity

The six bundle files total 10,313,050,444 bytes. Use a directory on shared storage when working on a cluster. With huggingface_hub installed:

export RAP_WAM_WORK_DIR=/absolute/shared/path/rap_wam_eval
export HF_HOME="$RAP_WAM_WORK_DIR/cache/huggingface"
export HF_HUB_CACHE="$HF_HOME/hub"
export TMPDIR="$RAP_WAM_WORK_DIR/tmp"
mkdir -p "$HF_HOME" "$HF_HUB_CACHE" "$TMPDIR"
python - <<'PYTHON'
import os
from pathlib import Path
from huggingface_hub import HfApi, snapshot_download

repo = "hyppyh010403/RAP-WAM"
# Resolve once, print the exact revision, and reuse it for subsequent runs.
revision = HfApi().model_info(repo).sha
output = Path(os.environ["RAP_WAM_WORK_DIR"]) / "s742500"
snapshot_download(
    repo_id=repo,
    revision=revision,
    local_dir=output,
    allow_patterns=[
        "model.pt", "task_context.pt", "task_strings.json",
        "inference_config.json", "model_tensors.json", "SHA256SUMS",
    ],
)
print("Repository revision:", revision)
print("Bundle:", output)
PYTHON
(cd "$RAP_WAM_WORK_DIR/s742500" && sha256sum --check SHA256SUMS)

For a reproducible run, record the printed 40-character repository revision alongside the adapter revision and evaluation settings. model.pt and task_context.pt are PyTorch tensor artifacts rather than a Transformers from_pretrained model. Use the compatible RAP-WAM inference loader; do not unpickle untrusted checkpoint files or use an unrelated public training constructor as a drop-in replacement.

The original Wan assets are external. Obtain the complete Wan2.2-TI2V-5B asset directory, including backbone configuration/shards, Wan2.2_VAE.pth, UMT5 encoder and tokenizer, and set WAN_CHECKPOINT_DIR accordingly. The bundle preserves the fine-tuned transformer and policy tensors; external assets are still needed by the historical constructor and observation/language encoders.

Bundle contents

model.pt contains 886 tensors, including 30 normalization tensors. model_tensors.json records tensor identities, shapes, dtypes and checksums. task_context.pt and task_strings.json preserve the original language table and order. Optimizer, scheduler, demonstrations and training caches are omitted.

File	Bytes	SHA-256
model.pt	10,019,266,099	48359d224b3ad8f41e29416a723a36ba199e56ada25c97273e1f0b0319cc9435
task_context.pt	293,605,893	845c145ddad653c8566f5dbfb909aaf9a074a5f698c6c3aaf23b51423cc5d569
task_strings.json	2,918	32e9ffa07ec3635151699faf0cf8b6ab2b9c87e31b207dc5c257f2fe89d9ff3b
inference_config.json	1,841	ef3d4398bb82440d1c15780b19ed50855e72e69473626c3abe9da1985d1baef5
model_tensors.json	173,279	d5fac96f8fb176669306e1725c896b2efbea2faac920f1f0c6afdce9d5353120
SHA256SUMS	414	ce9f1a58d30f63bb31b306de2fa15a13e67240f922e65f98082b2911ac8de341

The canonical model-tensor fingerprint is c3d500da9fcec1e83aa4414f71f68b329bddb0a5553fd0addfaf28dcf76f37ea. Public provenance is recorded in RELEASE_MANIFEST.json.

Validation and scope

The inference export was checked against the retained historical checkpoint for complete tensor equality, including normalization. Four real GPU-forward comparisons produced identical outputs between the new and historical wrappers sharing the same strictly loaded policy, with matched random-number state. Plain-image, encoded-image and batch-size-2 WebSocket debug paths were also exercised. These checks validate the integration and wrapper behavior, not independently trained models or aggregate benchmark performance.

A RoboDojo stack_bowls integration smoke test at seed 0, layout 0 completed successfully with native score 100. This is one episode and is not an estimate of task-level or benchmark-wide success rate. No benchmark aggregate is claimed for this release.

The public training repository provides a separate portable base-model training workflow. Its package/configuration and training recipe are not an exact continuation interface for this historical checkpoint. This bundle does not include optimizer state and is intended for inference, not bit-exact training resumption. The corresponding XPolicyLab adapter is being prepared separately; this card does not claim that an adapter PR has been merged.

License and attribution

The model-weight release uses Apache-2.0, matching the Wan foundation model's weight license. See LICENSE and NOTICE for the full text and attribution. RAP-WAM project source code is separately licensed under MIT in its source repository; that source-code license is not used to relicense the Wan foundation weights. External assets and datasets retain their respective terms.

Citation

@misc{he2026rapwam,
  title={RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations},
  author={Yuanpeng He and Ceyao Zhang and Fangjing Li and Kefei Zhu and Lijian Li and Yueheng Li and Tianjia He and Wenpin Jiao and Zhi Jin and Yaodong Yang and Yuanpei Chen},
  year={2026},
  eprint={2610.16443},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2610.16443}
}