README.md

---

license: apache-2.0
tags:
  - video-frame-interpolation
  - frame-interpolation
  - video
  - comfyui
  - ema-vfi
---


# EMA-VFI, released checkpoints as safetensors

The four checkpoints released with **EMA-VFI**, converted from the original pickles to
safetensors and nothing else. The weights are unchanged, tensor for tensor.

Upstream: <https://github.com/MCG-NJU/EMA-VFI>. This is a mirror, not a new model. All credit
for the network and the training belongs to its authors.

## Why this exists

The originals are distributed as bare `state_dict` **pickles** inside a Google Drive folder.
That is two problems for anyone building on them. A pickle executes arbitrary code when it is
loaded, and a Drive folder gives out per-file links that are not stable enough to script
against, so tools end up telling people to fetch a file by hand.

Safetensors cannot execute anything on load, and a Hub URL stays put.

## The files

| File | Size | Parameters | Quality | Multiplier above 2 |
|---|--:|--:|---|---|
| `ours_t.safetensors` | 250.5 MiB | 65.66M | best | **yes** |
| `ours.safetensors` | 250.5 MiB | 65.66M | best | no |
| `ours_small_t.safetensors` | 55.3 MiB | 14.49M | lower | **yes** |
| `ours_small.safetensors` | 55.3 MiB | 14.49M | lower | no |

Take **`ours_t`** unless you have a reason not to. The `_t` pair was trained to land anywhere

between two frames, so it can do 3x, 4x and beyond. The other two only ever land halfway, which

limits them to a multiplier of 2. The `small` pair is a narrower network: quicker, and visibly

weaker on fast motion.



## Provenance



Every file was downloaded from the authors' own Drive folder, `16jUa3HkQ85Z5lb5gce1yoaWkP-rdCd0o`,

linked from the upstream README. No third-party mirror was used.



| Original | SHA-256 of the source pickle |

|---|---|

| `ours_t.pkl` | `c09e82d193e303d7b91f1e36bf535886b820cb56105574355211d60a6b557a99` |

| `ours.pkl` | `f9085bb32d8b2eaa70eb1df08cc25bc7e973c50a74d74fb593ebe0d5e100a11f` |

| `ours_small_t.pkl` | `ad845671f7250d408937718aca19eaf87aa1ca79215070f7641eea332b703233` |

| `ours_small.pkl` | `ef0e455a7fe75c24d415bbcd83c29a9c0134ba853942bde0cd4dc4ed01e3f98f` |



Each hash is repeated in its safetensors file's own metadata, under `source_sha256`, alongside

the architecture the weights belong to. `conversion_report.json` in this repo records the whole

run.



## What was checked before publishing



1. Loaded with `torch.load(weights_only=True)`, which refuses arbitrary pickle opcodes, so a

   payload would have raised rather than run.

2. Built the network at the matching size and required `load_state_dict(strict=True)`, which

   proves the tensors really are this architecture and that none is missing or spare.

3. Reloaded each converted file and required the same strict fit, then compared every tensor

   against the original with `torch.equal`. All exact.



## What the conversion changed



Two things, both mechanical, and both what a loader had to do anyway:



- **The `module.` prefix is stripped.** The originals were saved from a `DataParallel` wrapper,

  so every key carried it.

- **`attn_mask` and `HW` are dropped.** These are shifted-window buffers the backbone registers
  during `forward` from whatever padded size it is given, so a saved pair belongs to the
  resolution it was trained at and is wrong at any other. Upstream's own loader drops them too.
  8 such keys in the full models, 4 in the small ones.

Everything else is byte-identical, float32 throughout.

## Use it in ComfyUI

With [WAS Node Suite](https://github.com/WASasquatch/was-node-suite-comfyui) and its
**EMA-VFI Frame Interpolation** node, either way round.

**Let it fetch.** Set `features.network: true` in the pack's `config.yaml`, pick a checkpoint in
the node's menu and queue the graph. The file is downloaded once into `ComfyUI/models/EMA-VFI`
and kept. Nothing is fetched at startup, only on the run that needs it.

**Or place it yourself.** Download the file and put it here:

```

ComfyUI/models/EMA-VFI/ours_t.safetensors

```

Create the `EMA-VFI` directory if it is not there, and restart ComfyUI. The checkpoint list is
read when the node builds its schema, so a file added while ComfyUI is running does not appear
until it restarts.

Keep the filename as it is. The node picks the architecture from the name, so a renamed file
will not be recognised.

## Use it in plain torch

The network itself is not in this repo. Take it from upstream, or from
`modules/vendor/ema_vfi` in WAS Node Suite, then:

```python

from safetensors.torch import load_file



state = load_file("ours_t.safetensors")

net.load_state_dict(state, strict=True)   # no prefix to strip, no keys to drop

```

## Licence

Apache-2.0, as released by the authors: "This project is released under the Apache 2.0 license."
The full text is in `LICENSE` beside these files.

## Citation

```bibtex

@inproceedings{zhang2023extracting,

  title={Extracting motion and appearance via inter-frame attention for efficient video frame interpolation},

  author={Zhang, Guozhen and Zhu, Yuhan and Wang, Haonan and Chen, Youxin and Wu, Gangshan and Wang, Limin},

  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},

  pages={5682--5692},

  year={2023}

}

```