Abstract
Generative novel view synthesis (NVS) employs a diffusion model to render a static or dynamic scene from a target camera view. Explicit warp methods complete a reprojection of the source with video diffusion models, and implicit methods encode the camera parameters or 3D cues. Both add inference overhead, and the generated layout often drifts from the ground truth. We present WaveView, which injects the warped source into the phase of the initial noise of a pretrained generator. The Dual-Tree Complex Wavelet Packet Transform (DT-CWPT) allows local and anisotropic phase injection. The generator is thus constrained less in disoccluded regions and along the epipolar line. WaveView keeps the architecture and adds negligible overhead. On the DL3DV benchmark, it outperforms all baselines in every overlap bin. In the hardest bin, it gains 0.84 dB PSNR on the covisible region over MetaView, which shares its generator, while running 2.1 times faster. For dynamic NVS, fine-tuning AnyView with the same noise raises its zero-shot PSNR on AnyViewBench from 11.58 to 12.35 dB.
Community
๐ WaveView: The Novel View Is in the Noise
Generative novel view synthesis usually gives the target camera to the diffusion model as conditioning: through extra control layers, camera positional encodings, or a reprojected depth image the model must complete. WaveView asks a different question: what if the novel view were already in the initial noise?
How it works
- Warp the source into the target camera with depth.
- Decompose the warp and a Gaussian noise into wavelet bands: each band takes its phase from the warp and its magnitude from the noise, so the result is still a valid Gaussian-like initial noise.
- A depth error moves a pixel only along its epipolar line, so phase is kept across that line and released along it as depth uncertainty grows; disoccluded regions stay pure noise.
- The generator's architecture and conditioning are unchanged.
Results
- ๐ On DL3DV, WaveView outperforms warp-guided and positional-encoding methods in every overlap bin; on the hardest bin it gains +0.84 dB PSNR over MetaView (same generator) on the region seen in the source.
- โก 17.9 s per view vs. 38.0 s for MetaView and 64.1 s for ViewCrafter (about 2.1ร faster than MetaView), because nothing is added to the network.
- ๐ฅ The same noise transfers to the video generator AnyView, raising its AnyViewBench zero-shot PSNR from 11.58 โ 12.35 dB.
Get this paper in your agent:
hf papers read 2611.23994 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper