Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: gpl-3.0
|
| 3 |
+
pipeline_tag: image-to-3d
|
| 4 |
+
tags:
|
| 5 |
+
- novel-view-synthesis
|
| 6 |
+
- neural-rendering
|
| 7 |
+
- nerf
|
| 8 |
+
- 3d
|
| 9 |
+
datasets:
|
| 10 |
+
- RealEstate10K
|
| 11 |
+
- DL3DV-10K
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
# CLiFT: Compressive Light-Field Tokens for Compute-Efficient and Adaptive Neural Rendering
|
| 15 |
+
|
| 16 |
+
Pretrained checkpoints for **CLiFT** ([arXiv](https://arxiv.org/abs/2507.08776), [project page](https://clift-nvs.github.io/), [code](https://github.com/eric-zqwang/CLiFT)).
|
| 17 |
+
|
| 18 |
+
CLiFT represents a scene as compressed light-field tokens (CLiFTs): an encoder tokenizes the input views, latent K-means selects centroid tokens, and a neural condenser aggregates each cluster into its centroid. A single trained model supports compute-adaptive rendering — fewer tokens for lower storage/compute, more tokens for higher quality.
|
| 19 |
+
|
| 20 |
+
## Checkpoints
|
| 21 |
+
|
| 22 |
+
| File | Description | Place at (in the [code repo](https://github.com/eric-zqwang/CLiFT)) |
|
| 23 |
+
|---|---|---|
|
| 24 |
+
| `re10k/first_stage.ckpt` | RealEstate10K encoder-decoder (LVSM-style, random token-drop) | `output/re10k_first_stage/training/last.ckpt` |
|
| 25 |
+
| `re10k/second_stage.ckpt` | RealEstate10K condenser (full CLiFT model) | `output/re10k_second_stage/training/last.ckpt` |
|
| 26 |
+
| `dl3dv/first_stage.ckpt` | DL3DV encoder-decoder, fine-tuned from the RE10K model | `output/dl3dv_first_stage/training/last.ckpt` |
|
| 27 |
+
| `dl3dv/second_stage.ckpt` | DL3DV condenser (full CLiFT model) | `output/dl3dv_second_stage/training/last.ckpt` |
|
| 28 |
+
|
| 29 |
+
The second-stage checkpoints are the full CLiFT models used for the paper's main results. The first-stage checkpoints are used by the random / K-means selection baselines, for K-means annotation, and as the initialization for condenser training.
|
| 30 |
+
|
| 31 |
+
## Usage
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
git clone https://github.com/eric-zqwang/CLiFT.git
|
| 35 |
+
cd CLiFT
|
| 36 |
+
# download the checkpoints to the paths above, prepare data (see docs/), then e.g.
|
| 37 |
+
bash script/eval/eval_clift.sh # RE10K
|
| 38 |
+
bash script/eval/eval_dl3dv.sh 6 # DL3DV, 6 context views
|
| 39 |
+
```
|
| 40 |
+
|
| 41 |
+
See the [test guide](https://github.com/eric-zqwang/CLiFT/blob/main/docs/test.md) for evaluation and the [training guide](https://github.com/eric-zqwang/CLiFT/blob/main/docs/training.md) for the two-stage training pipeline.
|
| 42 |
+
|
| 43 |
+
## Citation
|
| 44 |
+
|
| 45 |
+
```bibtex
|
| 46 |
+
@article{Wang2025CLiFT,
|
| 47 |
+
author = {Wang, Zhengqing and Wu, Yuefan and Chen, Jiacheng and Zhang, Fuyang and Furukawa, Yasutaka},
|
| 48 |
+
title = {CLiFT: Compressive Light-Field Tokens for Compute Efficient and Adaptive Neural Rendering},
|
| 49 |
+
journal = {arXiv preprint arXiv:2507.08776},
|
| 50 |
+
year = {2025},
|
| 51 |
+
}
|
| 52 |
+
```
|