Bernini-R-S2V

Speech-driven video on Bernini-R , T2V, I2V, and V2V with lip-sync.

This model adds single-speaker audio support to Bernini-R, so you can drive video with speech in text-to-video, image-to-video, and video-to-video setups. It is not state-of-the-art audio-to-video, but it removes the need for post-processing or extra models just to add speech to Wan videos. For basic talking-head work, or longer videos built from short sequences, it is a handy all-in-one option on top of Bernini's motion and editing strengths.

ComfyUI detects these as WAN22_S2V / 'WanModel_S2V' (audio keys trigger S2V model type).

Usage

  1. Download the checkpoints.
  2. Place in:
    ComfyUI/models/diffusion_models/
    
  3. Add wav2vec2 to:
    ComfyUI/models/audio_encoders/
    
  4. Install ComfyUI-WanBerniniS2V custom node:
    (Create a folder in 'ComfyUI/custom_nodes/'  named 'ComfyUI-WanBerniniS2V', then save the Python files in that folder.)
    
  5. Restart ComfyUI.
  6. Search for the Bernini S2V Conditioning node or use the example workfllow

Audio tips

Setting Recommendation
Channels Mono, wav2vec2 downmixes stereo internally
Sample rate 44.1 khz or 48 khz (resampled to 16 kHz)
Content Clear speech, less background music = better sync
Length in my testing, max 9 to 15 seconds
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for rzgar/Bernini-R-S2V

Finetuned
(5)
this model

Collection including rzgar/Bernini-R-S2V