Image-to-Image
Diffusers
Safetensors

Add pipeline tag, library name, and improve model card

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +24 -5
README.md CHANGED
@@ -1,19 +1,24 @@
1
  ---
2
- license: mit
3
  base_model:
4
  - stabilityai/stable-diffusion-2
 
 
 
5
  ---
 
6
  # StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
7
  This is the official repo for paper [StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation](https://www.arxiv.org/abs/2505.06668)
8
 
 
 
9
  ## Setup
10
- 0. Clone the [code repo](https://github.com/ziyiwhat/StableMotion).
11
  1. Create your environment from `requirements.txt`.
12
  2. Download [DIR-D](https://drive.google.com/file/d/1KR5DtekPJin3bmQPlTGP4wbM1zFR80ak/view?usp=sharing) and [RS-Real](https://huggingface.co/datasets/Yzl-code/RS-Diffusion/tree/main). Put them into `StableMotion_SIR` and `StableMotion_RSC` respectively.
13
 
14
  ## StableMotion for Stitched Image Rectangling (SIR)
15
  ### Inference
16
- 0. Download the checkpoints of `StableMotion_SIR`
17
  1. Run `cd StableMotion_SIR && sh sample.sh`. You may want to change this file to modify the inference configurations.
18
  2. Run `sh metrics.sh` to evaluate the results.
19
 
@@ -23,9 +28,23 @@ This is the official repo for paper [StableMotion: Repurposing Diffusion-Based I
23
 
24
  ## StableMotion for Rolling Shutter Correction (RSC)
25
  ### Inference
26
- 0. Download the checkpoints of `StableMotion_RSC`
27
  1. Run `cd StableMotion_RSC && sh sample.sh`. You may want to change this file to modify the inference configurations.
28
  2. Run `sh metrics.sh` to evaluate the results.
29
 
30
  ### Training
31
- Run `cd StableMotion_RSC && sh train.sh`. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
2
  base_model:
3
  - stabilityai/stable-diffusion-2
4
+ license: mit
5
+ library_name: diffusers
6
+ pipeline_tag: image-to-image
7
  ---
8
+
9
  # StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation
10
  This is the official repo for paper [StableMotion: Repurposing Diffusion-Based Image Priors for Motion Estimation](https://www.arxiv.org/abs/2505.06668)
11
 
12
+ Official Code Repository: [GitHub - ivowang/StableMotion](https://github.com/ivowang/StableMotion)
13
+
14
  ## Setup
15
+ 0. Clone the [code repo](https://github.com/ivowang/StableMotion).
16
  1. Create your environment from `requirements.txt`.
17
  2. Download [DIR-D](https://drive.google.com/file/d/1KR5DtekPJin3bmQPlTGP4wbM1zFR80ak/view?usp=sharing) and [RS-Real](https://huggingface.co/datasets/Yzl-code/RS-Diffusion/tree/main). Put them into `StableMotion_SIR` and `StableMotion_RSC` respectively.
18
 
19
  ## StableMotion for Stitched Image Rectangling (SIR)
20
  ### Inference
21
+ 0. Download the checkpoints of [StableMotion_SIR](https://huggingface.co/ziyiwhat/StableMotion/tree/main/StableMotion_SIR)
22
  1. Run `cd StableMotion_SIR && sh sample.sh`. You may want to change this file to modify the inference configurations.
23
  2. Run `sh metrics.sh` to evaluate the results.
24
 
 
28
 
29
  ## StableMotion for Rolling Shutter Correction (RSC)
30
  ### Inference
31
+ 0. Download the checkpoints of [StableMotion_RSC](https://huggingface.co/ziyiwhat/StableMotion/tree/main/StableMotion_RSC)
32
  1. Run `cd StableMotion_RSC && sh sample.sh`. You may want to change this file to modify the inference configurations.
33
  2. Run `sh metrics.sh` to evaluate the results.
34
 
35
  ### Training
36
+ Run `cd StableMotion_RSC && sh train.sh`. You may want to change this file to modify the training configurations. The default configuration requires approximately 40 GB of VRAM per card.
37
+
38
+ ## GPT Rule-Based Evaluation
39
+ Each task folder has a `gpt_eval` subfolder with the script used in the paper to score results with a vision LLM (GPT) on a fixed rubric. `StableMotion_SIR/gpt_eval/score_rectangle.py` scores Stitched Image Rectangling (SIR) `(input, output)` pairs, and `StableMotion_RSC/gpt_eval/score_rolling_shutter.py` scores Rolling Shutter Correction (RSC) `[input | Yang | Ours]` triptychs. Both call an OpenAI-Responses-compatible API and emit per-pair scores plus an aggregate `summary.json` (mean/std/95% CI). To run, copy `provider.example.json` to `provider.json` in the relevant folder, add your endpoint/key, then e.g. `cd StableMotion_SIR/gpt_eval && python score_rectangle.py <input_dir> <result_dir>`. See each folder's `README.md` for the full rubric, flags, and outputs.
40
+
41
+ ## Citation
42
+
43
+ ```bibtex
44
+ @article{wang2025stablemotion,
45
+ title={StableMotion: One-Step Motion Estimation with Diffusion Prior},
46
+ author={Wang, Ziyi and Li, Haipeng and Sui, Lin and Zhou, Tianhao and Jiang, Hai and Nie, Lang and Liu, Shuaicheng},
47
+ journal={arXiv preprint arXiv:2505.06668},
48
+ year={2025}
49
+ }
50
+ ```