File size: 3,125 Bytes
11c2def
ed519a7
 
 
 
11c2def
 
 
ed519a7
 
 
11c2def
 
ed519a7
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c6789f1
ed519a7
 
c6789f1
4bc4c77
 
 
 
 
 
 
ed519a7
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
---
title: VAE Rhythm
emoji: 🎼
colorFrom: yellow
colorTo: indigo
sdk: gradio
sdk_version: 6.32.0
app_file: app.py
python_version: "3.12"
startup_duration_timeout: 1h
short_description: Detect AI-generated video via VAE decoding traces
---

# VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection

Demo for the paper **VAE Rhythm: Temporal Decoding Traces for AI-Generated Video
Detection** ([arXiv 2611.21114](https://arxiv.org/abs/2611.21114), Show Lab Β· NUS).
It ships the paper's two detectors:

1. **Phase scan** (training-free) β€” re-encode a clip with the paper's protocol
   (x264 CRF 23, GOP 32, short side capped at 720 px), reconstruct K=24 successive
   W=13-frame windows with a frozen causal Wan2.1 video VAE, and read the
   periodicity of the reconstruction-error residual. Generated videos are decoded
   in frame groups, so their error profile repeats with the group period; real
   videos give a nearly flat one.
2. **CaDet** (trained detector, [Jack97/CaDet](https://huggingface.co/Jack97/CaDet)) β€”
   two parameter-efficient paths on a frozen DINOv3 ViT-L/16 read 8 frames per clip
   and output a sum of z-scores standardised on real validation videos.

## Interpretation

- **Phase scan**: a high z(Ppk) (above β‰ˆ 2.35, the 95th percentile of real videos) is
  evidence that a clip was decoded by a causal video VAE. **A low value is *not*
  evidence that a video is real** β€” low motion, frame-rate conversion, upscaling and
  re-encoding all weaken the rhythm.
- **CaDet**: the score is a z-score sum (β‰ˆ 0 for typical real videos, larger = more
  likely generated), **not a probability**. Operating points shift across real-video
  sources; heavily re-encoded, very short or low-resolution clips are harder.
- Neither score alone is a definitive real/fake verdict.

## Examples

Each example is a **VAE pair** built with the paper's own recipe: a stock clip from the
CC0 dataset [linoyts/repo-to-space-example-videos](https://huggingface.co/datasets/linoyts/repo-to-space-example-videos),
protocol-encoded, next to a copy of the same frames through one Wan2.1 VAE round trip
re-encoded with the same chain. The pairs showcase the *contrast* a VAE round trip
leaves in the traces. Measured on this Space: the cat pair scans Ppk 0.477 vs 0.091
(C8 0.82 vs 0.11) and the rope pair 0.311 vs 0.135 (low motion β€” see the warning in
the app), while the 50 fps veggie pair's scan contrast is weak (frame-rate conversion
weakens the rhythm, as the interpretation rules note). The CaDet score rises for the
round-trip twin in all three pairs (βˆ’0.84β†’βˆ’0.20, 2.62β†’3.10, 3.30β†’5.69). Clip names
are neutral on purpose β€” no ground-truth authenticity claim is made about the stock
footage itself.

## References

- Paper: Sun, Li, Mei, Shou. *VAE Rhythm: Temporal Decoding Traces for AI-Generated
  Video Detection.* arXiv 2611.21114 (2026).
- Code: [showlab/vae-rhythm](https://github.com/showlab/vae-rhythm) (MIT).
- Detector weights: [Jack97/CaDet](https://huggingface.co/Jack97/CaDet).
- Project page: [showlab.github.io/vae-rhythm](https://showlab.github.io/vae-rhythm/).