Hugging Face's logo

marstin
/
fast-spatial-mem

Image-to-3D
Model card Files
xet
 Community
1
fast-spatial-mem
File size: 3,843 Bytes
6da7422
 
 
 
 
 
 
 
 
 
 
7a10f75
6da7422
 
 
 
7a10f75
58e87b0
7a10f75
58e87b0
 
7a10f75
58e87b0
 
 
 
 
 
 
7a10f75
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
58e87b0
7a10f75
 
 
 
 
 
58e87b0
7a10f75
 
 
 
 
58e87b0
 
 
 
 
 
 
 
 
b71af25
58e87b0
 
 
 
 
 
 
 
 
b3699df
58e87b0
 
b3699df
 
58e87b0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
---
datasets:
- multicam
- stereo4d
- waymo
- egoexo4d
- dynamic_replica
- spring
- point_odyssey
- re10k
- dl3dv
license: apache-2.0
metrics:
- psnr
- ssim
- lpips
pipeline_tag: image-to-3d
---

# Model Card for Fast Spatial Memory Models

This repo is a public release of [**Fast Spatial Memory with Elastic Test-Time Training**](https://fast-spatial-memory.github.io/), as well as a *self-retrained (non-official!)* version of [**4D-LRM**](https://4dlrm.github.io/).

## Model Details

### Model Description

- **Developed by:** [MIT-IBM Watson Lab]
- **License:** [Apache License 2.0]
- **Task:** 3D/4D Reconstruction from long observation sequences.

### Model Sources

- **Repository:** [https://github.com/Mars-tin/fast-spatial-mem](https://github.com/Mars-tin/fast-spatial-mem)
- **Paper:** [https://arxiv.org/abs/2604.07350](https://arxiv.org/abs/2604.07350)
- **Homepage:** [https://fast-spatial-memory.github.io/](https://fast-spatial-memory.github.io/)

## Sample Usage

You can download the pretrained weights from this repository using the `hf_hub_download` function from the `huggingface_hub` library:

```python
import os
import shutil
from huggingface_hub import hf_hub_download

repo_id = "marstin/fast-spatial-mem"
local_path = "static/weights"
path_in_repo = "lvsm_checkpoints/fsm_4dlvsm_patch8_res256.pth"

# Download (cached under ~/.cache/huggingface/hub)
cached_path = hf_hub_download(
    repo_id=repo_id,
    filename=path_in_repo,
    repo_type="model"
)

# Copy to your desired local folder
os.makedirs(os.path.dirname(local_path), exist_ok=True)
target_path = os.path.join(local_path, os.path.basename(path_in_repo))
shutil.copy(cached_path, target_path)
```

## Performance Documentations

| Checkpoint | Steoro4D PSNR | Steoro4D LPIPS | Steoro4D SSIM | NVIDIA PSNR | NVIDIA LPIPS | NVIDIA SSIM | DL3DV PSNR | DL3DV LPIPS | DL3DV SSIM |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| `lrm_checkpoints/fsm_4dlrm_patch8_res256.pth` | 27.54 | 0.163 | 0.841 | 20.17 | 0.337 | 0.567 | 21.89 | 0.314 | 0.692 |
| `lrm_checkpoints/fsm_4dlrm_patch8_multilen_res256.pth` | 24.88 | 0.245 | 0.786 | 23.27 | 0.275 | 0.700 | 20.71 | 0.365 | 0.652 |
| `lrm_checkpoints/fsm_4dlrm_patch8_static_res256.pth` | 28.51 | 0.110 | 0.865 | 16.45 | 0.386 | 0.407 | 23.59 | 0.206 | 0.766 |
| `lvsm_checkpoints/fsm_3dlvsm_patch8_res256.pth` | 32.16 | 0.043 | 0.931 | 23.10 | 0.117 | 0.713 | 24.64 | 0.118 | 0.787 |
| `lvsm_checkpoints/fsm_4dlvsm_patch8_multilen_res256.pth` | 31.24 | 0.072 | 0.925 | 23.90 | 0.105 | 0.747 | 24.54 | 0.135 | 0.772 |
| `lvsm_checkpoints/fsm_3dlvsm_patch8_res256.pth` | N/A | N/A | N/A | N/A | N/A | N/A | 27.01 | 0.084 | 0.859 |


## Citation

### Fast Spatial Memory with Elastic Test-Time Training
Ziqiao Ma*, Xueyang Yu*, Haoyu Zhen, Yuncong Yang, Joyce Chai, Chuang Gan 

```bibtex
@article{ma2026fast,
  title={Fast Spatial Memory with Elastic Test-Time Training},
  author={Ma, Ziqiao and Yu, Xueyang and Zhen, Haoyu and Yang, Yuncong and Chai, Joyce and Gan, Chuang},
  journal={arXiv preprint arXiv:2604.07350},
  year={2026}
}
```

### 4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time
Ziqiao Ma, Xuweiyi Chen, Shoubin Yu, Sai Bi, Kai Zhang, Chen Ziwen, Sihan Xu, Jianing Yang, Zexiang Xu, Kalyan Sunkavalli, Mohit Bansal, Joyce Chai, Hao Tan

```bibtex
@inproceedings{ma20254dlrm,
  title={4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time},
  author={Ma, Ziqiao and Chen, Xuweiyi and Yu, Shoubin and Bi, Sai and Zhang, Kai and Ziwen, Chen and Xu, Sihan and Yang, Jianing and Xu, Zexiang and Sunkavalli, Kalyan and others},
  booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
  year={2025}
}
```

## Model Card Author

Martin Ziqiao Ma

## Model Card Contact

marstin@umich.edu / ziqiaoma@ibm.com