README.md

metadata

datasets:
  - multicam
  - stereo4d
  - waymo
  - egoexo4d
  - dynamic_replica
  - spring
  - point_odyssey
  - re10k
  - dl3dv
license: apache-2.0
metrics:
  - psnr
  - ssim
  - lpips
pipeline_tag: image-to-3d

Model Card for Fast Spatial Memory Models

This repo is a public release of Fast Spatial Memory with Elastic Test-Time Training, as well as a self-retrained (non-official!) version of 4D-LRM.

Model Details

Model Description

• Developed by: [MIT-IBM Watson Lab]
• License: [Apache License 2.0]
• Task: 3D/4D Reconstruction from long observation sequences.

Model Sources

• Repository: https://github.com/Mars-tin/fast-spatial-mem
• Paper: https://arxiv.org/abs/2604.07350
• Homepage: https://fast-spatial-memory.github.io/

Sample Usage

You can download the pretrained weights from this repository using the hf_hub_download function from the huggingface_hub library:

import os
import shutil
from huggingface_hub import hf_hub_download

repo_id = "marstin/fast-spatial-mem"
local_path = "static/weights"
path_in_repo = "lvsm_checkpoints/fsm_4dlvsm_patch8_res256.pth"

# Download (cached under ~/.cache/huggingface/hub)
cached_path = hf_hub_download(
    repo_id=repo_id,
    filename=path_in_repo,
    repo_type="model"
)

# Copy to your desired local folder
os.makedirs(os.path.dirname(local_path), exist_ok=True)
target_path = os.path.join(local_path, os.path.basename(path_in_repo))
shutil.copy(cached_path, target_path)

Performance Documentations

Checkpoint	Steoro4D PSNR	Steoro4D LPIPS	Steoro4D SSIM	NVIDIA PSNR	NVIDIA LPIPS	NVIDIA SSIM	DL3DV PSNR	DL3DV LPIPS	DL3DV SSIM
lrm_checkpoints/fsm_4dlrm_patch8_res256.pth	27.54	0.163	0.841	20.17	0.337	0.567	21.89	0.314	0.692
lrm_checkpoints/fsm_4dlrm_patch8_multilen_res256.pth	24.88	0.245	0.786	23.27	0.275	0.700	20.71	0.365	0.652
lrm_checkpoints/fsm_4dlrm_patch8_static_res256.pth	28.51	0.110	0.865	16.45	0.386	0.407	23.59	0.206	0.766
lvsm_checkpoints/fsm_3dlvsm_patch8_res256.pth	32.16	0.043	0.931	23.10	0.117	0.713	24.64	0.118	0.787
lvsm_checkpoints/fsm_4dlvsm_patch8_multilen_res256.pth	31.24	0.072	0.925	23.90	0.105	0.747	24.54	0.135	0.772
lvsm_checkpoints/fsm_3dlvsm_patch8_res256.pth	N/A	N/A	N/A	N/A	N/A	N/A	27.01	0.084	0.859

Citation

Fast Spatial Memory with Elastic Test-Time Training

Ziqiao Ma*, Xueyang Yu*, Haoyu Zhen, Yuncong Yang, Joyce Chai, Chuang Gan

@article{ma2026fast,
  title={Fast Spatial Memory with Elastic Test-Time Training},
  author={Ma, Ziqiao and Yu, Xueyang and Zhen, Haoyu and Yang, Yuncong and Chai, Joyce and Gan, Chuang},
  journal={arXiv preprint arXiv:2604.07350},
  year={2026}
}

4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time

Ziqiao Ma, Xuweiyi Chen, Shoubin Yu, Sai Bi, Kai Zhang, Chen Ziwen, Sihan Xu, Jianing Yang, Zexiang Xu, Kalyan Sunkavalli, Mohit Bansal, Joyce Chai, Hao Tan

@inproceedings{ma20254dlrm,
  title={4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time},
  author={Ma, Ziqiao and Chen, Xuweiyi and Yu, Shoubin and Bi, Sai and Zhang, Kai and Ziwen, Chen and Xu, Sihan and Yang, Jianing and Xu, Zexiang and Sunkavalli, Kalyan and others},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
  year={2025}
}

Model Card Author

Martin Ziqiao Ma

Model Card Contact

marstin@umich.edu / ziqiaoma@ibm.com