Rename RTC-AC model to AC-Stream and update model card

README.md

@@ -1,3 +1,192 @@
 ---
 license: apache-2.0
+library_name: pytorch
+tags:
+  - robotics
+  - robot-learning
+  - world-model
+  - action-generation
+  - libero
+  - pytorch
 ---
+
+# StreamWAM
+
+StreamWAM provides synchronous and asynchronous world-action inference for robot manipulation. This repository contains two LIBERO checkpoints in their original FastWAM checkpoint format. StreamWAM loads them directly at runtime; no offline checkpoint conversion or duplicate model copy is required.
+
+## Models
+
+| Directory | Inference mode | Description |
+| --- | --- | --- |
+| `joint-cd/` | Joint CD | Direct synchronous one-step consistency inference that jointly predicts world and action tokens. |
+| `ac-stream/` | AC-Stream | D0/D8 asynchronous inference that overlaps model prediction with action execution. The same checkpoint supports eager and accelerated backends. |
+
+Each directory contains:
+
+```text
+model.pt
+dataset_stats.json
+```
+
+The checkpoints use a Wan2.2 TI2V 5B backbone. The Wan2.2 model assets are not included in this repository and must be obtained separately.
+
+## Inference geometry
+
+Both checkpoints target the following LIBERO setup:
+
+- two RGB cameras, concatenated horizontally;
+- per-camera resolution: 224 x 224;
+- model image input: 224 x 448;
+- action dimension: 7;
+- proprioception dimension: 8;
+- action horizon: 32;
+- video context: 9 frames at inference time;
+- BF16 model execution.
+
+Joint CD uses one consistency step and replans every 16 executed actions.
+
+AC-Stream uses the H32/s16/d8 schedule:
+
+- D0 starts without a clean action prefix;
+- D8 conditions on the first 8 clean actions of the current horizon;
+- background D8 inference is launched while actions are executing;
+- the controller advances by a stride of 16 actions;
+- deadline misses block at the boundary instead of installing a stale prediction.
+
+## Requirements
+
+Inference requires:
+
+1. the StreamWAM source code;
+2. Wan2.2 TI2V 5B model assets;
+3. a LIBERO source checkout and simulator dependencies;
+4. the checkpoint and matching `dataset_stats.json` from this repository.
+
+The accelerated AC-Stream backend was validated with:
+
+```text
+Python 3.10.20
+PyTorch 2.7.1+cu128
+Triton 3.3.1
+CUDA 12.8 runtime
+NVIDIA H100 80GB HBM3
+BF16
+```
+
+It uses static full-graph `torch.compile`, Inductor CUDA Graph Trees, prompt cross-attention K/V caching, attention-mask and schedule caching, and D0/D8 prewarming. Compilation or prewarming failure is reported instead of silently falling back to eager execution.
+
+## Download
+
+Install the Hugging Face CLI, authenticate if required, and download the repository:
+
+```bash
+hf download SJTU-DENG-Lab/StreamWAM \
+  --local-dir checkpoints/streamwam
+```
+
+The resulting layout is:
+
+```text
+checkpoints/streamwam/
+├── joint-cd/
+│   ├── model.pt
+│   └── dataset_stats.json
+└── ac-stream/
+    ├── model.pt
+    └── dataset_stats.json
+```
+
+## LIBERO evaluation
+
+Set the external asset paths first:
+
+```bash
+export BACKBONE_PATH=/path/to/Wan2.2-TI2V-5B
+export LIBERO_HOME_PATH=/path/to/LIBERO
+```
+
+### Joint CD
+
+The following evaluates all 40 LIBERO tasks once with four persistent workers:
+
+```bash
+python examples/libero/multigpu_rollout.py \
+  --gpus 0,1,2,3 \
+  --suites libero_spatial,libero_object,libero_goal,libero_10 \
+  --num-trials 1 \
+  --config examples/libero/configs/recipes/streamwam_libero_joint_cd_wan22_5b.yaml \
+  --checkpoint-format fastwam \
+  --checkpoint checkpoints/streamwam/joint-cd/model.pt \
+  --backbone-path "$BACKBONE_PATH" \
+  --stats-path checkpoints/streamwam/joint-cd/dataset_stats.json \
+  --libero-home "$LIBERO_HOME_PATH" \
+  --num-steps-wait 30 \
+  --replan-steps 16 \
+  --num-inference-steps 1 \
+  --sampling-method consistency \
+  --fixed-seed \
+  --mujoco-gl egl \
+  --save-video
+```
+
+### AC-Stream eager
+
+```bash
+python examples/libero/multigpu_rollout.py \
+  --gpus 0,1,2,3 \
+  --suites libero_spatial,libero_object,libero_goal,libero_10 \
+  --num-trials 1 \
+  --config examples/libero/configs/recipes/streamwam_libero_rtc_ac_wan22_5b.yaml \
+  --checkpoint-format fastwam \
+  --checkpoint checkpoints/streamwam/ac-stream/model.pt \
+  --backbone-path "$BACKBONE_PATH" \
+  --stats-path checkpoints/streamwam/ac-stream/dataset_stats.json \
+  --libero-home "$LIBERO_HOME_PATH" \
+  --num-steps-wait 30 \
+  --replan-steps 16 \
+  --num-inference-steps 1 \
+  --sampling-method rtc_ac \
+  --fixed-seed \
+  --mujoco-gl egl \
+  --save-video
+```
+
+### AC-Stream accelerated
+
+Use the same checkpoint and append `--rtc-ac-accelerated`:
+
+```bash
+python examples/libero/multigpu_rollout.py \
+  --gpus 0,1,2,3 \
+  --suites libero_spatial,libero_object,libero_goal,libero_10 \
+  --num-trials 1 \
+  --config examples/libero/configs/recipes/streamwam_libero_rtc_ac_wan22_5b.yaml \
+  --checkpoint-format fastwam \
+  --checkpoint checkpoints/streamwam/ac-stream/model.pt \
+  --backbone-path "$BACKBONE_PATH" \
+  --stats-path checkpoints/streamwam/ac-stream/dataset_stats.json \
+  --libero-home "$LIBERO_HOME_PATH" \
+  --num-steps-wait 30 \
+  --replan-steps 16 \
+  --num-inference-steps 1 \
+  --sampling-method rtc_ac \
+  --rtc-ac-accelerated \
+  --fixed-seed \
+  --mujoco-gl egl \
+  --save-video
+```
+
+For the validated accelerated runtime, the final report should show one Dynamo graph, zero recompiles, zero Inductor CUDA Graph skips, and successful D0/D8 prewarming. Raw inference averages include startup effects; use the reported steady-state D8 latency for backend comparisons.
+
+## Checkpoint loading
+
+Use `--checkpoint-format fastwam` for both models. StreamWAM validates the video expert, action expert, and proprioception encoder key sets and tensor shapes before copying weights. A partial or incompatible checkpoint fails before mutating the model.
+
+The matching statistics file is required for LIBERO state normalization and action denormalization. Do not mix statistics between `joint-cd` and `ac-stream`.
+
+## Scope and limitations
+
+- These checkpoints are provided for LIBERO evaluation.
+- Wan2.2 backbone assets and LIBERO are external dependencies and are not redistributed here.
+- AC-Stream accelerated performance depends on GPU architecture, PyTorch/Triton versions, compilation cache state, and simulator workload.
+- The accelerated backend changes execution strategy but uses the same AC-Stream checkpoint and inference semantics.