Video_to_video_WAN / API_GUIDE.md
kulkas2pintu's picture
Switch apply-step from Wan-Animate to Wan2.1-VACE masked inpaint: regenerates ONLY the clothing region, copies face/hair/skin/motion/background from the source at original sharpness. UI: video + prompt -> video. VACE on 'large' (49 frames @6fps, ~265 quota incl remote FireRed) fits a FREE account. Boot probes reference_images + pack size. Removed dead wan_engine helpers; stopped purging the VACE model cache.
b95bfc8 verified
|
Raw
History Blame Contribute Delete
8.01 kB

A newer version of the Gradio SDK is available: 6.26.0

Upgrade

API Guide β€” Change Clothes in a Video

Space: kulkas2pintu/Video_to_video_WAN Host: https://kulkas2pintu-video-to-video-wan.hf.space

Changes a person's clothing in a video from a text prompt:

  1. trims the clip,
  2. finds the frame where the person shows the most of their full body,
  3. edits that frame's outfit with FireRed using your prompt,
  4. re-inserts that re-clothed person into the video with Wan 2.2 Animate, keeping the original motion and scene.

ZeroGPU quota β€” read this first

This Space runs on ZeroGPU. Quota is not wall-clock seconds β€” it depends on the GPU size a call uses:

xlarge :  quota = declared_duration Γ— 2      (full GPU)
large  :  quota = declared_duration Γ— 1.5    (half GPU, 1.5Γ— the wall time)

The FireRed step runs on a separate Space and the Wan step runs here. Both are billed to your account, out of the same daily budget.

Account Daily quota = real GPU seconds
Unauthenticated 120 60
Free 300 150
PRO 2400 1200

Admission is checked against the declared duration, so the tier you pick decides whether your call is even accepted:

tier FireRed (remote) Wan (here) Total Free (300)?
free ~19 s 255 s (large) ~274 s βœ… fits
pro ~19 s 500 s (xlarge) ~519 s ❌ rejected

A rejected call costs nothing (no worker is spawned), so an over-quota attempt is harmless β€” it just fails fast.


Endpoints

Endpoint Purpose GPU cost
/quota_probe Reports what a run will reserve. 0 (not a GPU call)
/change_clothes Run the full pipeline. see table above

/quota_probe used to be a real GPU call reserving 580 quota β€” which a free account could never pass. It is now a plain informational endpoint costing nothing, and it returns a descriptive string, not "ok".


/change_clothes

Inputs (positional order)

# Name Type Required Default Notes
1 video video file yes β€” Must be {"video": handle_file(path_or_url), "subtitles": None} β€” a bare handle_file() fails with a VideoData validation error.
2 prompt str yes β€” The new outfit, e.g. "a red hoodie and black jeans".
3 resolution_choice "Low Res" | "Medium Res" no "Low Res" Low Res = 640Γ—368, Medium Res = 832Γ—480. Forced to Low Res when tier="free".
4 tier "free" | "pro" no "free" Quota/quality profile β€” see below.

tier was added last, so existing 3-argument callers keep working (they get "free").

What each tier produces

tier clip out frames segment steps resolution
free 10.0 s @ 8 fps 80 81 5 Low Res (forced)
pro 9.5 s @ 8 fps 76 77 8 your choice

There are no seed / guidance_scale parameters. FireRed's settings are fixed internally (4 steps, CFG 1.0, random seed).

Outputs (in order)

# Name Type Notes
1 result video The finished video (filepath).
2 edited_reference_frame_firered image The re-clothed still FireRed produced.
3 status str (markdown) Status, timings, and quota reserved.

Python (gradio_client)

pip install gradio_client

Use submit(), not predict() β€” this is a long job

A run takes several minutes (CPU pose extraction, then the GPU stages, plus any queue wait). predict() blocks the whole time and is easy to time out.

import time
from gradio_client import Client, handle_file

client = Client("kulkas2pintu/Video_to_video_WAN", hf_token="hf_...")

# Optional: see what a run will reserve (costs nothing, returns a description)
print(client.predict("free", api_name="/quota_probe"))

job = client.submit(
    {"video": handle_file("my_clip.mp4"), "subtitles": None},   # 1 video
    "a red hoodie and black jeans",    # 2 prompt
    "Low Res",                         # 3 resolution_choice
    "free",                            # 4 tier
    api_name="/change_clothes",
)

while not job.done():
    print("status:", job.status().code)
    time.sleep(10)

video_path, ref_image, status = job.result()
print(status)
print("result video:", video_path)

video_path is a temp file the client downloaded β€” copy it somewhere permanent if you want to keep it. handle_file also accepts a URL.


JavaScript (@gradio/client)

npm i @gradio/client
import { Client } from "@gradio/client";

const app = await Client.connect("kulkas2pintu/Video_to_video_WAN", {
  hf_token: "hf_...",
});

const videoBlob = new Blob([await (await fetch("my_clip.mp4")).arrayBuffer()]);

const out = await app.predict("/change_clothes", [
  videoBlob,                        // 1 video
  "a red hoodie and black jeans",   // 2 prompt
  "Low Res",                        // 3 resolution_choice
  "free",                           // 4 tier
]);

console.log(out.data); // [result video, reference image, status]

Raw HTTP (curl)

HOST=https://kulkas2pintu-video-to-video-wan.hf.space
TOKEN=hf_...

# (0) upload the video, get its server path
FILE=$(curl -s -H "Authorization: Bearer $TOKEN" \
  -F "files=@my_clip.mp4" "$HOST/gradio_api/upload" \
  | python -c "import sys,json;print(json.load(sys.stdin)[0])")

# (1) start the job -> returns an event id   (NOTE: 4 data items)
EVENT=$(curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d "{\"data\":[{\"video\":{\"path\":\"$FILE\"},\"subtitles\":null},\"a red hoodie and black jeans\",\"Low Res\",\"free\"]}" \
  "$HOST/gradio_api/call/change_clothes" \
  | python -c "import sys,json;print(json.load(sys.stdin)['event_id'])")

# (2) stream the result
curl -s -N -H "Authorization: Bearer $TOKEN" "$HOST/gradio_api/call/change_clothes/$EVENT"

Authentication

  • The Space is public, but ZeroGPU is quota-metered per account.
  • Pass an hf_token; GPU time is billed against that token's account.
  • With tier="free", a free account fits one 10-second generation per day (~274 of 300 quota). A second run that day will be rejected.
  • With tier="pro" you need a PRO account (~519 quota per run of 2400/day).

Errors you may hit

Error Meaning What to do
You have exceeded your ... quota (Xs requested vs Ys left) Not enough quota for the tier you chose. Use tier="free", or wait for the daily reset. Nothing was charged.
'GPU task aborted' The GPU call exceeded its reserved time (or OOM'd). Retry; report the generate: timing from the logs.
AcceleratorError: uncorrectable ECC error The assigned ZeroGPU card is faulty hardware. Restart the Space to re-schedule onto a healthy node. Not a code bug.
FREE tier needs ...segment_frame_length... The installed diffusers build can't do the cheap free path. Use tier="pro", or pin diffusers==0.39.0.
HTTP 500 / 503 on the page Container busy/restarting β€” normal for a heavy ZeroGPU Space. Retry in a moment.

Notes & limits

  • Output is 8 fps. Frame counts are capped so generation fits exactly one pipeline segment inside the ZeroGPU per-call time limit.
  • One person. The pipeline picks the most full-body person frame.
  • Cold start additionally downloads/loads model weights (several minutes).
  • Identity vs. realism. Wan Animate regenerates the person from the reference, so the face can drift slightly and the background is re-rendered.
  • Queue holds up to 20 jobs; GPU runs are serialized.