WAS
/

EMA-VFI

@@ -1,123 +1,129 @@
----
-license: apache-2.0
-tags:
-  - video-frame-interpolation
-  - frame-interpolation
-  - video
-  - comfyui
-  - ema-vfi
----
-# EMA-VFI, released checkpoints as safetensors
-The four checkpoints released with **EMA-VFI**, converted from the original pickles to
-safetensors and nothing else. The weights are unchanged, tensor for tensor.
-Upstream: <https://github.com/MCG-NJU/EMA-VFI>. This is a mirror, not a new model. All credit
-for the network and the training belongs to its authors.
-## Why this exists
-The originals are distributed as bare `state_dict` **pickles** inside a Google Drive folder.
-That is two problems for anyone building on them. A pickle executes arbitrary code when it is
-loaded, and a Drive folder gives out per-file links that are not stable enough to script
-against, so tools end up telling people to fetch a file by hand.
-Safetensors cannot execute anything on load, and a Hub URL stays put.
-## The files
-| File | Size | Parameters | Quality | Multiplier above 2 |
-|---|--:|--:|---|---|
-| `ours_t.safetensors` | 250.5 MiB | 65.66M | best | **yes** |
-| `ours.safetensors` | 250.5 MiB | 65.66M | best | no |
-| `ours_small_t.safetensors` | 55.3 MiB | 14.49M | lower | **yes** |
-| `ours_small.safetensors` | 55.3 MiB | 14.49M | lower | no |
-Take **`ours_t`** unless you have a reason not to. The `_t` pair was trained to land anywhere
-between two frames, so it can do 3x, 4x and beyond. The other two only ever land halfway, which
-limits them to a multiplier of 2. The `small` pair is a narrower network: quicker, and visibly
-weaker on fast motion.
-## Provenance
-Every file was downloaded from the authors' own Drive folder, `16jUa3HkQ85Z5lb5gce1yoaWkP-rdCd0o`,
-linked from the upstream README. No third-party mirror was used.
-| Original | SHA-256 of the source pickle |
-|---|---|
-| `ours_t.pkl` | `c09e82d193e303d7b91f1e36bf535886b820cb56105574355211d60a6b557a99` |
-| `ours.pkl` | `f9085bb32d8b2eaa70eb1df08cc25bc7e973c50a74d74fb593ebe0d5e100a11f` |
-| `ours_small_t.pkl` | `ad845671f7250d408937718aca19eaf87aa1ca79215070f7641eea332b703233` |
-| `ours_small.pkl` | `ef0e455a7fe75c24d415bbcd83c29a9c0134ba853942bde0cd4dc4ed01e3f98f` |
-Each hash is repeated in its safetensors file's own metadata, under `source_sha256`, alongside
-the architecture the weights belong to. `conversion_report.json` in this repo records the whole
-run.
-## What was checked before publishing
-1. Loaded with `torch.load(weights_only=True)`, which refuses arbitrary pickle opcodes, so a
-   payload would have raised rather than run.
-2. Built the network at the matching size and required `load_state_dict(strict=True)`, which
-   proves the tensors really are this architecture and that none is missing or spare.
-3. Reloaded each converted file and required the same strict fit, then compared every tensor
-   against the original with `torch.equal`. All exact.
-## What the conversion changed
-Two things, both mechanical, and both what a loader had to do anyway:
-- **The `module.` prefix is stripped.** The originals were saved from a `DataParallel` wrapper,
-  so every key carried it.
-- **`attn_mask` and `HW` are dropped.** These are shifted-window buffers the backbone registers
-  during `forward` from whatever padded size it is given, so a saved pair belongs to the
-  resolution it was trained at and is wrong at any other. Upstream's own loader drops them too.
-  8 such keys in the full models, 4 in the small ones.
-Everything else is byte-identical, float32 throughout.
-## Use it in ComfyUI
-With [WAS Node Suite](https://github.com/WASasquatch/was-node-suite-comfyui) and its
-**EMA-VFI Frame Interpolation** node. Put the file here:
-```
-ComfyUI/models/EMA-VFI/ours_t.safetensors
-```
-Create the `EMA-VFI` directory if it is not there, and restart ComfyUI. The checkpoint list is
-read when the node builds its schema, so a file added while ComfyUI is running does not appear
-until it restarts.
-Keep the filename as it is. The node picks the architecture from the name, so a renamed file
-will not be recognised.
-## Use it in plain torch
-The network itself is not in this repo. Take it from upstream, or from
-`modules/vendor/ema_vfi` in WAS Node Suite, then:
-```python
-from safetensors.torch import load_file
-state = load_file("ours_t.safetensors")
-net.load_state_dict(state, strict=True)   # no prefix to strip, no keys to drop
-```
-## Licence
-Apache-2.0, as released by the authors: "This project is released under the Apache 2.0 license."
-The full text is in `LICENSE` beside these files.
-## Citation
-```bibtex
-@inproceedings{zhang2023extracting,
-  title={Extracting motion and appearance via inter-frame attention for efficient video frame interpolation},
-  author={Zhang, Guozhen and Zhu, Yuhan and Wang, Haonan and Chen, Youxin and Wu, Gangshan and Wang, Limin},
-  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
-  pages={5682--5692},
-  year={2023}
-}
-```

+---
+license: apache-2.0
+tags:
+  - video-frame-interpolation
+  - frame-interpolation
+  - video
+  - comfyui
+  - ema-vfi
+---
+# EMA-VFI, released checkpoints as safetensors
+The four checkpoints released with **EMA-VFI**, converted from the original pickles to
+safetensors and nothing else. The weights are unchanged, tensor for tensor.
+Upstream: <https://github.com/MCG-NJU/EMA-VFI>. This is a mirror, not a new model. All credit
+for the network and the training belongs to its authors.
+## Why this exists
+The originals are distributed as bare `state_dict` **pickles** inside a Google Drive folder.
+That is two problems for anyone building on them. A pickle executes arbitrary code when it is
+loaded, and a Drive folder gives out per-file links that are not stable enough to script
+against, so tools end up telling people to fetch a file by hand.
+Safetensors cannot execute anything on load, and a Hub URL stays put.
+## The files
+| File | Size | Parameters | Quality | Multiplier above 2 |
+|---|--:|--:|---|---|
+| `ours_t.safetensors` | 250.5 MiB | 65.66M | best | **yes** |
+| `ours.safetensors` | 250.5 MiB | 65.66M | best | no |
+| `ours_small_t.safetensors` | 55.3 MiB | 14.49M | lower | **yes** |
+| `ours_small.safetensors` | 55.3 MiB | 14.49M | lower | no |
+Take **`ours_t`** unless you have a reason not to. The `_t` pair was trained to land anywhere
+between two frames, so it can do 3x, 4x and beyond. The other two only ever land halfway, which
+limits them to a multiplier of 2. The `small` pair is a narrower network: quicker, and visibly
+weaker on fast motion.
+## Provenance
+Every file was downloaded from the authors' own Drive folder, `16jUa3HkQ85Z5lb5gce1yoaWkP-rdCd0o`,
+linked from the upstream README. No third-party mirror was used.
+| Original | SHA-256 of the source pickle |
+|---|---|
+| `ours_t.pkl` | `c09e82d193e303d7b91f1e36bf535886b820cb56105574355211d60a6b557a99` |
+| `ours.pkl` | `f9085bb32d8b2eaa70eb1df08cc25bc7e973c50a74d74fb593ebe0d5e100a11f` |
+| `ours_small_t.pkl` | `ad845671f7250d408937718aca19eaf87aa1ca79215070f7641eea332b703233` |
+| `ours_small.pkl` | `ef0e455a7fe75c24d415bbcd83c29a9c0134ba853942bde0cd4dc4ed01e3f98f` |
+Each hash is repeated in its safetensors file's own metadata, under `source_sha256`, alongside
+the architecture the weights belong to. `conversion_report.json` in this repo records the whole
+run.
+## What was checked before publishing
+1. Loaded with `torch.load(weights_only=True)`, which refuses arbitrary pickle opcodes, so a
+   payload would have raised rather than run.
+2. Built the network at the matching size and required `load_state_dict(strict=True)`, which
+   proves the tensors really are this architecture and that none is missing or spare.
+3. Reloaded each converted file and required the same strict fit, then compared every tensor
+   against the original with `torch.equal`. All exact.
+## What the conversion changed
+Two things, both mechanical, and both what a loader had to do anyway:
+- **The `module.` prefix is stripped.** The originals were saved from a `DataParallel` wrapper,
+  so every key carried it.
+- **`attn_mask` and `HW` are dropped.** These are shifted-window buffers the backbone registers
+  during `forward` from whatever padded size it is given, so a saved pair belongs to the
+  resolution it was trained at and is wrong at any other. Upstream's own loader drops them too.
+  8 such keys in the full models, 4 in the small ones.
+Everything else is byte-identical, float32 throughout.
+## Use it in ComfyUI
+With [WAS Node Suite](https://github.com/WASasquatch/was-node-suite-comfyui) and its
+**EMA-VFI Frame Interpolation** node, either way round.
+**Let it fetch.** Set `features.network: true` in the pack's `config.yaml`, pick a checkpoint in
+the node's menu and queue the graph. The file is downloaded once into `ComfyUI/models/EMA-VFI`
+and kept. Nothing is fetched at startup, only on the run that needs it.
+**Or place it yourself.** Download the file and put it here:
+```
+ComfyUI/models/EMA-VFI/ours_t.safetensors
+```
+Create the `EMA-VFI` directory if it is not there, and restart ComfyUI. The checkpoint list is
+read when the node builds its schema, so a file added while ComfyUI is running does not appear
+until it restarts.
+Keep the filename as it is. The node picks the architecture from the name, so a renamed file
+will not be recognised.
+## Use it in plain torch
+The network itself is not in this repo. Take it from upstream, or from
+`modules/vendor/ema_vfi` in WAS Node Suite, then:
+```python
+from safetensors.torch import load_file
+state = load_file("ours_t.safetensors")
+net.load_state_dict(state, strict=True)   # no prefix to strip, no keys to drop
+```
+## Licence
+Apache-2.0, as released by the authors: "This project is released under the Apache 2.0 license."
+The full text is in `LICENSE` beside these files.
+## Citation
+```bibtex
+@inproceedings{zhang2023extracting,
+  title={Extracting motion and appearance via inter-frame attention for efficient video frame interpolation},
+  author={Zhang, Guozhen and Zhu, Yuhan and Wang, Haonan and Chen, Youxin and Wu, Gangshan and Wang, Limin},
+  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
+  pages={5682--5692},
+  year={2023}
+}
+```