Migrate LayeredFlow checkpoints from the original Drive release
Browse filesUpload seven unchanged RAFT and MultiRAFT checkpoints, with a model card, source manifest, and SHA-256 checksums.
- MultiRAFT_L.pth +3 -0
- MultiRAFT_S+L.pth +3 -0
- README.md +85 -0
- SHA256SUMS +7 -0
- checkpoint-manifest.json +44 -0
- first_layer_L.pth +3 -0
- first_layer_S+L.pth +3 -0
- first_layer_S.pth +3 -0
- last_layer_L.pth +3 -0
- raft-sintel.pth +3 -0
MultiRAFT_L.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9a8f7678e7124ac13449a99a740b99abbbad97906e542595a98883aa4d2d6986
|
| 3 |
+
size 34091437
|
MultiRAFT_S+L.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:028eee1795285115d9e67008879a9cee4de584256657c0448dfb27cee53497b7
|
| 3 |
+
size 34091437
|
README.md
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
tags:
|
| 3 |
+
- optical-flow
|
| 4 |
+
- multi-layer-optical-flow
|
| 5 |
+
- pytorch
|
| 6 |
+
- raft
|
| 7 |
+
- multiraft
|
| 8 |
+
- layeredflow
|
| 9 |
+
datasets:
|
| 10 |
+
- Hermera/LayeredFlow-Syn
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# LayeredFlow pre-trained checkpoints
|
| 14 |
+
|
| 15 |
+
Pre-trained RAFT and MultiRAFT checkpoints for [LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow](https://arxiv.org/abs/2409.05688) (ECCV 2024), by Hongyu Wen, Erich Liang, and Jia Deng.
|
| 16 |
+
|
| 17 |
+
These files are migrated from the checkpoint folder linked by the [official LayeredFlow repository](https://github.com/princeton-vl/LayeredFlow): [original Google Drive release](https://drive.google.com/drive/folders/1vB8wfeS1JpfTz-jKdiD_e_IM5Sn5Wztl). The original checkpoint filenames are preserved.
|
| 18 |
+
|
| 19 |
+
## Models
|
| 20 |
+
|
| 21 |
+
- **RAFT:** single-layer optical-flow models evaluated on the first-layer and last-layer benchmark subsets.
|
| 22 |
+
- **MultiRAFT:** the repository's multi-layer RAFT model, which predicts four optical-flow layers and is evaluated on the full benchmark.
|
| 23 |
+
|
| 24 |
+
The checkpoints use the original PyTorch `.pth` format and are loaded by the LayeredFlow evaluation scripts.
|
| 25 |
+
|
| 26 |
+
## Files
|
| 27 |
+
|
| 28 |
+
| Checkpoint | Use | Size (bytes) |
|
| 29 |
+
| --- | --- | ---: |
|
| 30 |
+
| `raft-sintel.pth` | RAFT Sintel checkpoint; also used for MultiRAFT initialization | 21,108,000 |
|
| 31 |
+
| `last_layer_L.pth` | RAFT last-layer evaluation | 21,103,087 |
|
| 32 |
+
| `first_layer_S+L.pth` | RAFT first-layer evaluation | 21,103,087 |
|
| 33 |
+
| `first_layer_S.pth` | RAFT first-layer evaluation | 21,103,087 |
|
| 34 |
+
| `first_layer_L.pth` | RAFT first-layer evaluation | 21,103,087 |
|
| 35 |
+
| `MultiRAFT_S+L.pth` | MultiRAFT multi-layer evaluation | 34,091,437 |
|
| 36 |
+
| `MultiRAFT_L.pth` | MultiRAFT multi-layer evaluation | 34,091,437 |
|
| 37 |
+
|
| 38 |
+
The seven checkpoints are unchanged copies of the original release. `checkpoint-manifest.json` records each source URL, byte size, and SHA-256 digest; `SHA256SUMS` contains the checksums.
|
| 39 |
+
|
| 40 |
+
## Download and evaluate
|
| 41 |
+
|
| 42 |
+
Clone the [LayeredFlow source repository](https://github.com/princeton-vl/LayeredFlow) and follow its installation and benchmark-data setup instructions. From the source repository root, download the checkpoints with the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli):
|
| 43 |
+
|
| 44 |
+
```bash
|
| 45 |
+
hf download Hermera/LayeredFlow --include 'first_layer_*.pth' 'last_layer_*.pth' 'raft-sintel.pth' --local-dir RAFT/checkpoints
|
| 46 |
+
hf download Hermera/LayeredFlow --include 'MultiRAFT_*.pth' 'raft-sintel.pth' --local-dir MultiRAFT/checkpoints
|
| 47 |
+
```
|
| 48 |
+
|
| 49 |
+
This places the files under `RAFT/checkpoints/` and `MultiRAFT/checkpoints/`, where the evaluation scripts expect them.
|
| 50 |
+
|
| 51 |
+
Run the following validation examples from the source repository root:
|
| 52 |
+
|
| 53 |
+
```bash
|
| 54 |
+
# First-layer optical flow
|
| 55 |
+
(cd RAFT && python3 evaluate_firstlayer.py --checkpoint first_layer_S+L.pth --dataset layeredflow)
|
| 56 |
+
|
| 57 |
+
# Last-layer optical flow
|
| 58 |
+
(cd RAFT && python3 evaluate_lastlayer.py --checkpoint last_layer_L.pth --dataset layeredflow)
|
| 59 |
+
|
| 60 |
+
# Multi-layer optical flow
|
| 61 |
+
(cd MultiRAFT && python3 evaluate.py --checkpoint MultiRAFT_S+L.pth --dataset layeredflow --mixed_precision)
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
The repository's evaluation scripts require a CUDA-capable environment. See the [benchmark website](https://layeredflow.cs.princeton.edu/) for test-set submission instructions.
|
| 65 |
+
|
| 66 |
+
## Related resources
|
| 67 |
+
|
| 68 |
+
- [Source code and benchmark documentation](https://github.com/princeton-vl/LayeredFlow)
|
| 69 |
+
- [LayeredFlow benchmark](https://layeredflow.cs.princeton.edu/)
|
| 70 |
+
- [LayeredFlow-Syn synthetic images and annotations](https://huggingface.co/datasets/Hermera/LayeredFlow-Syn)
|
| 71 |
+
|
| 72 |
+
## Attribution and license
|
| 73 |
+
|
| 74 |
+
The original LayeredFlow code is distributed under the [BSD 3-Clause license](https://github.com/princeton-vl/LayeredFlow/blob/main/LICENSE). This checkpoint migration does not introduce new license terms. Refer to the original release and its authors for checkpoint licensing.
|
| 75 |
+
|
| 76 |
+
## Citation
|
| 77 |
+
|
| 78 |
+
```bibtex
|
| 79 |
+
@article{wen2024layeredflow,
|
| 80 |
+
title={LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow},
|
| 81 |
+
author={Hongyu Wen and Erich Liang and Jia Deng},
|
| 82 |
+
journal={arXiv preprint arXiv:2409.05688},
|
| 83 |
+
year={2024},
|
| 84 |
+
}
|
| 85 |
+
```
|
SHA256SUMS
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
90630d2e7d488a0d3ccb5e8194524850c4c05c732ea4ff99799822c7fa5c5cbf raft-sintel.pth
|
| 2 |
+
b58868989acc6aa3564294cb99fbe174ce8fbc192de0bd1f0703e25ef34c413d last_layer_L.pth
|
| 3 |
+
d005921d6aeb97e138e8bdf2e1f4982a8c5c69cd9f112e1ce3861f7330f272c7 first_layer_S+L.pth
|
| 4 |
+
befe609962712f67a1198d8c008b86c3ba2112b5b81c08008d793241a1ba90d8 first_layer_S.pth
|
| 5 |
+
264d9694ea9a32b08e7b398a092e832722719db61a650d9dac5b09a2542e0859 first_layer_L.pth
|
| 6 |
+
028eee1795285115d9e67008879a9cee4de584256657c0448dfb27cee53497b7 MultiRAFT_S+L.pth
|
| 7 |
+
9a8f7678e7124ac13449a99a740b99abbbad97906e542595a98883aa4d2d6986 MultiRAFT_L.pth
|
checkpoint-manifest.json
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[
|
| 2 |
+
{
|
| 3 |
+
"filename": "raft-sintel.pth",
|
| 4 |
+
"size_bytes": 21108000,
|
| 5 |
+
"sha256": "90630d2e7d488a0d3ccb5e8194524850c4c05c732ea4ff99799822c7fa5c5cbf",
|
| 6 |
+
"source_url": "https://drive.google.com/file/d/1ytOmvkrBJirfiNFJTsrzXAAKLpxXfmzf/view"
|
| 7 |
+
},
|
| 8 |
+
{
|
| 9 |
+
"filename": "last_layer_L.pth",
|
| 10 |
+
"size_bytes": 21103087,
|
| 11 |
+
"sha256": "b58868989acc6aa3564294cb99fbe174ce8fbc192de0bd1f0703e25ef34c413d",
|
| 12 |
+
"source_url": "https://drive.google.com/file/d/1irQ0BKoVHMxaJQdU8WQjobYYSrJmfFg5/view"
|
| 13 |
+
},
|
| 14 |
+
{
|
| 15 |
+
"filename": "first_layer_S+L.pth",
|
| 16 |
+
"size_bytes": 21103087,
|
| 17 |
+
"sha256": "d005921d6aeb97e138e8bdf2e1f4982a8c5c69cd9f112e1ce3861f7330f272c7",
|
| 18 |
+
"source_url": "https://drive.google.com/file/d/1hV0dSQJXRvY3iPkYfS8_aN213ZOibvo-/view"
|
| 19 |
+
},
|
| 20 |
+
{
|
| 21 |
+
"filename": "first_layer_S.pth",
|
| 22 |
+
"size_bytes": 21103087,
|
| 23 |
+
"sha256": "befe609962712f67a1198d8c008b86c3ba2112b5b81c08008d793241a1ba90d8",
|
| 24 |
+
"source_url": "https://drive.google.com/file/d/1ds9EDwmYUq2x_ZHO5wZCL70f86yaUMjc/view"
|
| 25 |
+
},
|
| 26 |
+
{
|
| 27 |
+
"filename": "first_layer_L.pth",
|
| 28 |
+
"size_bytes": 21103087,
|
| 29 |
+
"sha256": "264d9694ea9a32b08e7b398a092e832722719db61a650d9dac5b09a2542e0859",
|
| 30 |
+
"source_url": "https://drive.google.com/file/d/1xHsX9y2gyMSr4uEhzcAyKOyvvGhv0vha/view"
|
| 31 |
+
},
|
| 32 |
+
{
|
| 33 |
+
"filename": "MultiRAFT_S+L.pth",
|
| 34 |
+
"size_bytes": 34091437,
|
| 35 |
+
"sha256": "028eee1795285115d9e67008879a9cee4de584256657c0448dfb27cee53497b7",
|
| 36 |
+
"source_url": "https://drive.google.com/file/d/1nAStMNhdBUq27d0d4d4DvS0PCqM517hT/view"
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"filename": "MultiRAFT_L.pth",
|
| 40 |
+
"size_bytes": 34091437,
|
| 41 |
+
"sha256": "9a8f7678e7124ac13449a99a740b99abbbad97906e542595a98883aa4d2d6986",
|
| 42 |
+
"source_url": "https://drive.google.com/file/d/1Rj3CKtLPVH3kZjIXnEfCCOqIo6cAB-y1/view"
|
| 43 |
+
}
|
| 44 |
+
]
|
first_layer_L.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:264d9694ea9a32b08e7b398a092e832722719db61a650d9dac5b09a2542e0859
|
| 3 |
+
size 21103087
|
first_layer_S+L.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d005921d6aeb97e138e8bdf2e1f4982a8c5c69cd9f112e1ce3861f7330f272c7
|
| 3 |
+
size 21103087
|
first_layer_S.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:befe609962712f67a1198d8c008b86c3ba2112b5b81c08008d793241a1ba90d8
|
| 3 |
+
size 21103087
|
last_layer_L.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b58868989acc6aa3564294cb99fbe174ce8fbc192de0bd1f0703e25ef34c413d
|
| 3 |
+
size 21103087
|
raft-sintel.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:90630d2e7d488a0d3ccb5e8194524850c4c05c732ea4ff99799822c7fa5c5cbf
|
| 3 |
+
size 21108000
|