Spaces:

wavespeed
/

wan2.2-s2v

Running

App Files Files Community

wan2.2-s2v / index.html

chengzeyi

Add UTM attribution to outbound wavespeed.ai links

efe4cbd 1 day ago

Raw

History Blame Contribute Delete

11 kB

	<!doctype html>
	<html lang="en">
	<head>
	<meta charset="utf-8" />
	<meta name="viewport" content="width=device-width, initial-scale=1" />
	<title>Wan2.2-S2V — audio-driven video generation</title>
	<meta name="description" content="Reference for Wan2.2-S2V, the speech-to-video checkpoint from Alibaba's Tongyi Lab that animates a reference portrait from an audio track." />
	<link rel="canonical" href="https://wavespeed.ai/models/wavespeed-ai/wan-2.2/speech-to-video?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" />

	<meta property="og:type" content="website" />
	<meta property="og:title" content="Wan2.2-S2V — audio-driven video generation" />
	<meta property="og:description" content="Reference for Wan2.2-S2V, the speech-to-video checkpoint from Alibaba's Tongyi Lab that animates a reference portrait from an audio track." />
	<meta property="og:url" content="https://wavespeed.ai/models/wavespeed-ai/wan-2.2/speech-to-video" />
	<meta name="twitter:card" content="summary_large_image" />

	<link rel="stylesheet" href="style.css" />
	</head>
	<body>
	<header class="site-header">
	<div class="wrap">
	<a class="brand" href="https://wavespeed.ai?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">
	<span class="brand-mark" aria-hidden="true"></span>
	<span>WaveSpeed AI</span>
	</a>
	<nav class="header-nav">
	<a class="jump opt" href="#overview">Overview</a>
	<a class="jump opt" href="#inputs">Inputs</a>
	<a class="jump" href="#run">Run it</a>
	<a class="jump" href="#resources">Resources</a>
	<a href="https://wavespeed.ai/models/wavespeed-ai/wan-2.2/speech-to-video?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">wavespeed.ai ↗</a>
	</nav>
	</div>
	</header>

	<main class="wrap">
	<div class="hero">
	<p class="eyebrow">Alibaba · Tongyi Lab</p>
	<h1>Wan2.2-S2V</h1>
	<p class="lede">The speech-to-video member of the Wan 2.2 family. Given one reference image and an audio track, it generates a video whose lip movement, head motion and gesture follow the audio.</p>
	<ul class="meta">
	<li><b>Developer</b> Alibaba Tongyi Lab</li>
	<li><b>Task</b> audio + image → video</li>
	<li><b>Params</b> 14B</li>
	<li><b>License</b> Apache-2.0</li>
	</ul>
	</div>

	<section id="overview">
	<h2>Overview</h2>
	<p class="section-note">S2V extends the Wan 2.2 video backbone with audio conditioning. Rather than driving only the mouth region, the audio signal conditions the whole frame, so posture and gesture move with the speech as well.</p>
	<div class="grid">
	<div class="card">
	<h3>Audio-conditioned motion</h3>
	<p>Lip shape, head pose and upper-body movement are generated jointly from the audio track instead of being composited from a separate talking-head module.</p>
	</div>
	<div class="card">
	<h3>Identity from one image</h3>
	<p>A single reference frame supplies appearance and identity; no per-subject fine-tuning or reference video is required.</p>
	</div>
	<div class="card">
	<h3>Speech and song</h3>
	<p>The checkpoint is trained on both spoken and sung audio, so it is not restricted to conversational delivery.</p>
	</div>
	<div class="card">
	<h3>Long-form generation</h3>
	<p>Clips are produced in chunks with motion carried across boundaries, which keeps identity stable past the length of a single window.</p>
	</div>
	</div>
	</section>
	<section id="inputs">
	<h2>Inputs</h2>
	<div class="table-scroll">
	<table>
	<thead><tr><th>Field</th><th>Type</th><th>Notes</th></tr></thead>
	<tbody>
	<tr><td><code>image</code></td><td>URL</td><td>Reference frame. A clear, front-facing subject works best.</td></tr>
	<tr><td><code>audio</code></td><td>URL</td><td>Driving audio track. Determines the clip length.</td></tr>
	<tr><td><code>prompt</code></td><td>string</td><td>Optional description of scene, framing and camera behaviour.</td></tr>
	<tr><td><code>seed</code></td><td>int</td><td>Optional. Fix it to make a run reproducible.</td></tr>
	</tbody>
	</table>
	</div>
	</section>
	<section id="run">
	<h2>Run it</h2>
	<p class="section-note">Upload local files first with <code>POST /media/upload/binary</code> and pass the returned URLs, or reference any publicly reachable URL directly.</p>
	<div class="code">
	<div class="code-tabs" role="tablist">
	<button type="button" role="tab" aria-selected="true" data-panel="run-0">cURL</button>
	<button type="button" role="tab" aria-selected="false" data-panel="run-1">Python</button>
	<button type="button" role="tab" aria-selected="false" data-panel="run-2">JavaScript</button>
	</div>
	<pre id="run-0" role="tabpanel"><code># 1. submit the job
	curl -X POST "https://api.wavespeed.ai/api/v3/wavespeed-ai/wan-2.2/speech-to-video" \
	-H "Authorization: Bearer $WAVESPEED_API_KEY" \
	-H "Content-Type: application/json" \
	-d '{
	"image": "https://example.com/portrait.jpg",
	"audio": "https://example.com/speech.mp3",
	"prompt": "A woman speaking to camera in a sunlit studio, medium shot",
	"enable_sync_mode": false
	}'

	# -> {"code": 200, "data": {"id": "<request-id>", "status": "created", ...}}

	# 2. poll until status is "completed"
	curl "https://api.wavespeed.ai/api/v3/predictions/<request-id>/result" \
	-H "Authorization: Bearer $WAVESPEED_API_KEY"

	# -> {"code": 200, "data": {"status": "completed", "outputs": ["https://..."]}}</code></pre>
	<pre id="run-1" role="tabpanel" hidden><code>import os, time, requests

	API = "https://api.wavespeed.ai/api/v3"
	KEY = os.environ["WAVESPEED_API_KEY"]
	HEADERS = {"Authorization": f"Bearer {KEY}"}

	# submit
	res = requests.post(
	f"{API}/wavespeed-ai/wan-2.2/speech-to-video",
	headers={**HEADERS, "Content-Type": "application/json"},
	json={
	"image": "https://example.com/portrait.jpg",
	"audio": "https://example.com/speech.mp3",
	"prompt": "A woman speaking to camera in a sunlit studio, medium shot",
	"enable_sync_mode": false
	},
	timeout=30,
	)
	res.raise_for_status()
	request_id = res.json()["data"]["id"]

	# poll
	while True:
	data = requests.get(
	f"{API}/predictions/{request_id}/result",
	headers=HEADERS,
	timeout=30,
	).json()["data"]

	if data["status"] == "completed":
	print(data["outputs"][0])
	break
	if data["status"] == "failed":
	raise RuntimeError(data.get("error", "generation failed"))
	time.sleep(1.5)</code></pre>
	<pre id="run-2" role="tabpanel" hidden><code>const API = "https://api.wavespeed.ai/api/v3";
	const KEY = process.env.WAVESPEED_API_KEY;
	const headers = { Authorization: `Bearer ${KEY}` };

	// submit
	const submit = await fetch(`${API}/wavespeed-ai/wan-2.2/speech-to-video`, {
	method: "POST",
	headers: { ...headers, "Content-Type": "application/json" },
	body: JSON.stringify({
	"image": "https://example.com/portrait.jpg",
	"audio": "https://example.com/speech.mp3",
	"prompt": "A woman speaking to camera in a sunlit studio, medium shot",
	"enable_sync_mode": false
	}),
	});
	const { data: { id } } = await submit.json();

	// poll
	for (;;) {
	const res = await fetch(`${API}/predictions/${id}/result`, { headers });
	const { data } = await res.json();

	if (data.status === "completed") {
	console.log(data.outputs[0]);
	break;
	}
	if (data.status === "failed") throw new Error(data.error ?? "generation failed");
	await new Promise((r) => setTimeout(r, 1500));
	}</code></pre>
	</div>
	<div class="callout"><p>Requests are asynchronous: <code>POST</code> returns a request id, then you poll <code>/predictions/<id>/result</code> until <code>status</code> is <code>completed</code>. Set <code>enable_sync_mode: true</code> to have the call block and return outputs directly.</p><p>API keys are created in the <a href="https://wavespeed.ai/dashboard?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">WaveSpeed dashboard</a>.</p></div>
	<div class="btn-row">
	<a class="btn" href="https://wavespeed.ai/models/wavespeed-ai/wan-2.2/speech-to-video?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">Open on WaveSpeed</a>
	<a class="btn secondary" href="https://wavespeed.ai/docs?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">API reference</a>
	</div>
	</section>
	<section id="resources">
	<h2>Resources</h2>
	<ul class="links">
	<li><a href="https://huggingface.co/Wan-AI/Wan2.2-S2V-14B" target="_blank" rel="noopener"><span>Wan2.2-S2V-14B weights</span><span class="host">huggingface.co</span></a></li>
	<li><a href="https://github.com/Wan-Video/Wan2.2" target="_blank" rel="noopener"><span>Wan2.2 on GitHub</span><span class="host">github.com</span></a></li>
	<li><a href="https://wavespeed.ai/collections/wan-2-2?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener"><span>Wan 2.2 collection</span><span class="host">wavespeed.ai</span></a></li>
	<li><a href="https://wavespeed.ai/docs?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener"><span>WaveSpeed API docs</span><span class="host">wavespeed.ai</span></a></li>
	</ul>
	</section>
	</main>

	<footer class="site-footer">
	<div class="wrap">
	<p>This page is a model reference maintained by WaveSpeed AI. The model itself is developed and released by its respective authors; trademarks belong to them. WaveSpeed AI provides hosted inference for it.</p>
	<div class="footer-links">
	<a href="https://wavespeed.ai?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">WaveSpeed AI</a>
	<a href="https://wavespeed.ai/docs?utm_source=huggingface&utm_medium=space&utm_campaign=wan2_2_s2v" target="_blank" rel="noopener">Docs</a>
	<a href="https://huggingface.co/wavespeed" target="_blank" rel="noopener">Hugging Face</a>
	</div>
	</div>
	</footer>

	<script src="tabs.js"></script>
	</body>
	</html>