Simplify FoveaVLA-Dual model card

README.md

@@ -6,7 +6,6 @@ tags:
   - robotics
   - vision-language-action
   - foveation
-  - metric-depth
   - openpi
   - pi05
   - droid
@@ -15,90 +14,31 @@ tags:
 
 # FoveaVLA-Dual
 
-FoveaVLA-Dual is the dual-camera model from **FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs**. It builds on π₀.₅ and predicts robot actions together with the interaction region to observe at the next policy query. FoveaVLA-Dual adds a second external viewpoint to the same foveation mechanism.
+FoveaVLA-Dual is the dual-camera variant of **FoveaVLA: Foveation Actions Replace the Wrist Camera in VLAs**. Built on π₀.₅, it uses two exterior RGB-D cameras with full images, predicted ROI crops, and matching depth. **No wrist camera is required.**
 
-The model operates **without a wrist camera**. It receives a full RGB image and ROI crop from each exterior camera, plus all four matching metric-depth inputs. RGB and depth use a shared SigLIP encoder, with depth supplied as separate visual token streams.
+- **Inputs:** two exterior ZED stereo cameras in physical-left / physical-right order; robot state; language instruction.
+- **Outputs:** 16 joint-position actions with gripper commands and two ordered interaction boxes for t+8. Execute 8 actions before the next query.
+- **Training data:** DROID demonstrations with RoboInter interaction annotations.
+- **Config:** `foveavla_dual`.
 
-Code, data preparation, training, and deployment: [Stanford-TML/FoveaVLA](https://github.com/Stanford-TML/FoveaVLA).
-
-## Model and observation contract
-
-| Item | Value |
-|---|---|
-| Training/serving config | `foveavla_dual` |
-| Exterior cameras | two exterior ZED stereo cameras, ordered physical-left then physical-right |
-| RGB inputs | 4 streams: left full + ROI, right full + ROI |
-| Depth inputs | 4 streams, aligned with the corresponding RGB inputs |
-| Wrist camera | None |
-| Robot actions | 7 absolute joint positions + 1 gripper command |
-| Action prediction | 16 actions per query; execute 8 before observing again |
-| Foveation prediction | Two ordered interaction boxes for t+8 |
-| Model image resolution | 224 × 224 |
-| Training data | DROID demonstrations with RoboInter interaction annotations |
-| Checkpoint format | JAX/Orbax OCDBT |
-
-The first query uses the full image as the ROI context. Later queries crop the newly captured image using the preceding t+8 prediction while retaining the full-image context. Crops use the released direct-rendering recipe with context scale 1.5.
-
-Metric depth comes from the pinned Fast-FoundationStereo estimator on each ZED's own rectified stereo pair. Stereo input is 640 × 360; depth is left-imager-aligned optical-axis Z in meters, bounded to 0.2–3.0 m, with validity masks. Offline data preparation and online serving use the same conversion and crop geometry. The supported ZED SDK version is 5.1.2.
-
-## Repository layout
-
-The checkpoint is directly at the repository root:
+## Checkpoint
 
 ```text
-assets/
-  droid/norm_stats.json
+assets/droid/norm_stats.json
 params/
 _CHECKPOINT_METADATA
-README.md
 ```
 
-`params/` contains inference weights. `assets/droid/norm_stats.json` contains this checkpoint's state/action normalization statistics and is required for deployment. The full optimizer/training state is not included.
+The checkpoint is at the repository root. It includes inference weights and their normalization statistics; optimizer state is not included.
 
-## Download and serve
+## Use
 
-From the FoveaVLA code repository, prepare the server environment and download this model:
+After setting up the [FoveaVLA server environment](https://github.com/Stanford-TML/FoveaVLA/blob/main/docs/deployment.md), run:
 
 ```bash
-uv venv --python 3.11
-uv sync --locked
-scripts/setup_fast_foundation_depth.sh
 uv run --no-sync scripts/download_checkpoint.py --model=foveavla_dual
-
 XLA_PYTHON_CLIENT_MEM_FRACTION=0.70 uv run --no-sync scripts/serve.py \
   --model=foveavla_dual --checkpoint=checkpoints/foveavla_dual
 ```
 
-The downloader uses a pinned Hub revision. If repository access requires authentication, log in with `huggingface-cli login` first.
-
-For a direct Hub download:
-
-```python
-from huggingface_hub import snapshot_download
-
-checkpoint_dir = snapshot_download(
-    repo_id="bingaochen/FoveaVLA-Dual",
-    local_dir="checkpoints/foveavla_dual",
-    allow_patterns=["assets/**", "params/**", "_CHECKPOINT_METADATA"],
-)
-```
-
-Pass `checkpoint_dir` to `scripts/serve.py` with `--model=foveavla_dual`. Use the server entrypoint so that stereo depth is generated with the matching estimator. The checkpoint loads its own normalization statistics.
-
-Before connecting the robot, check the software inference chain:
-
-```bash
-uv run --no-sync scripts/smoke_inference.py --host=127.0.0.1 --port=8000
-```
-
-See the [deployment guide](https://github.com/Stanford-TML/FoveaVLA/blob/main/docs/deployment.md) for the separate Python 3.10 DROID client, camera serials and ordering, supported robot embodiments, and real-robot commands.
-
-## Intended use
-
-This checkpoint supports research on foveation and robot manipulation with the released DROID deployment stack. Performance depends on camera placement, calibration, robot configuration, and task conditions. Camera alignment and physical task performance require validation on the target setup.
-
-To train a new model, use `foveavla_dual` and the [data](https://github.com/Stanford-TML/FoveaVLA/blob/main/docs/data.md) and [training](https://github.com/Stanford-TML/FoveaVLA/blob/main/docs/training.md) guides. This inference archive cannot resume the original optimizer state.
-
-## Acknowledgements
-
-FoveaVLA builds on [OpenPI](https://github.com/Physical-Intelligence/openpi), [DROID](https://droid-dataset.github.io/droid/the-droid-dataset.html), [RoboInter](https://github.com/InternRobotics/RoboInter), and [Fast-FoundationStereo](https://github.com/NVlabs/Fast-FoundationStereo). See the code repository's license and Gemma notice; third-party models, datasets, and software retain their respective terms.
+See [Stanford-TML/FoveaVLA](https://github.com/Stanford-TML/FoveaVLA) for data preparation, training, and real-robot deployment. The deployment guide covers stereo-depth setup, GPU memory, camera calibration, and supported robot configurations.