File size: 2,484 Bytes
64b9802 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | ---
language:
- en
tags:
- futureworlds
- world-model
- robotics
- video-prediction
- reinforcement-learning
---
# FutureWorlds — RoboCasa
Local release candidate, not yet published. Model weights are verified by CPU SHA-256; end-to-end inference with the portable RGB pipeline is not yet validated.
## Checkpoint variants
| Folder | Method | Updates | Use |
|---|---|---:|---|
| `models/main` | MemSPO (R0) | 400 | Paper main table |
| `models/sft` | SFT | 50,000 | Initialization/control |
| `models/memspo200` | MemSPO (R0) | 200 | Matched ablation |
| `models/grpo200` | GRPO (R0) | 200 | Matched ablation |
| `models/ordinary200` | Ordinary global beam8, top4 (R0) | 200 | Proposal-strategy ablation |
All post-training checkpoints start from their paired SFT checkpoint. `main` and `memspo200` must not be interchanged.
## Main-table evaluation
128 trajectories, 32 predicted frames, 2 initial observations, per-frame Beam4, anchor1 + recent6, no true-future refresh. Recorded split: held-out test; runtime: GPU FP32.
| PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---:|---:|---:|
| 18.4719 | 0.765948 | 0.183338 |
These are recorded paper results; staging weights does not constitute a fresh reproduction.
## Required components
- Dataset-specific `codec/` and `preprocessing/` files, paired by the manifest.
- Trained text adapters in each checkpoint and frozen T5-base at revision `a9723ea7f1b39c1eae772870f3b547bf6ef7e6c1`; T5 is referenced, not duplicated.
- 52 action tokens per predicted frame. RoboCasa uses four ordered 13-dimensional control blocks; do not flatten time in a different order.
- Original FP32 safetensors; no post-hoc quantization or checkpoint merging.
## Loading and scope
Use `futureworlds.checkpoints.load_world_model(bundle_root, "robocasa", variant="main")` for the predictor only. This does not decode RGB, prepare actions, or download T5 automatically. A complete RGB demo and clean-environment GPU equivalence run remain pending. These models are custom action-conditioned predictors, not generic `pipeline("text-to-video")` models.
## Provenance and limitations
Core architecture and tokenizer derive from RLVR-World/iVideoGPT. Upstream license texts are included under `licenses/`; the license for original additions is not yet selected. The raw datasets and third-party baseline weights are not included. Learned-world feedback examples are qualitative generated trajectories, not independently verified physical task-success results.
|