Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="UTF-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>VEFX-Code Β· Reference code for VEFX-Bench</title> | |
| <style> | |
| :root { --fg:#1a1a1a; --muted:#666; --bg:#fff; --card:#f7f7f9; --accent:#5b6cff; --border:#e5e7eb; } | |
| * { box-sizing:border-box; } | |
| body { font-family:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Helvetica,Arial,sans-serif; | |
| max-width:880px; margin:0 auto; padding:2.5em 1.25em; color:var(--fg); background:var(--bg); line-height:1.6; } | |
| h1 { font-size:2.2em; margin:0 0 .2em; } | |
| h2 { margin-top:2em; padding-bottom:.3em; border-bottom:1px solid var(--border); } | |
| .tagline { color:var(--muted); font-size:1.1em; margin-bottom:1.5em; } | |
| .pill-row { display:flex; flex-wrap:wrap; gap:.6em; margin:1.5em 0; } | |
| .pill { display:inline-flex; align-items:center; gap:.4em; padding:.5em 1em; background:var(--card); | |
| border:1px solid var(--border); border-radius:999px; text-decoration:none; color:var(--fg); | |
| font-size:.95em; transition:all .15s; } | |
| .pill:hover { border-color:var(--accent); color:var(--accent); } | |
| table { width:100%; border-collapse:collapse; margin:1em 0; font-size:.92em; } | |
| th, td { padding:.5em .7em; border-bottom:1px solid var(--border); text-align:left; } | |
| th { background:var(--card); font-weight:600; } | |
| td.num, th.num { text-align:center; } | |
| code { background:var(--card); padding:.15em .4em; border-radius:4px; font-size:.9em; } | |
| pre { background:var(--card); padding:1em; border-radius:6px; overflow-x:auto; } | |
| pre code { background:none; padding:0; } | |
| .grid { display:grid; grid-template-columns:1fr 1fr; gap:1em; margin:1em 0; } | |
| .card { background:var(--card); border:1px solid var(--border); border-radius:8px; padding:1em; } | |
| .card h3 { margin:0 0 .3em; font-size:1.05em; } | |
| .card p { margin:0; color:var(--muted); font-size:.9em; } | |
| footer { margin-top:3em; padding-top:1.5em; border-top:1px solid var(--border); color:var(--muted); font-size:.9em; } | |
| a { color:var(--accent); } | |
| </style> | |
| </head> | |
| <body> | |
| <h1>π¬ VEFX-Code</h1> | |
| <p class="tagline">Reference code & inference utilities for the <strong>VEFX-Bench</strong> benchmark β a comprehensive benchmark for evaluating text-driven video editing and visual effects.</p> | |
| <div class="pill-row"> | |
| <a class="pill" href="https://huggingface.co/spaces/VEFX-Reward/VEFX-Code/tree/main" target="_blank">π Browse Files</a> | |
| <a class="pill" href="https://huggingface.co/spaces/VEFX-Reward/VEFX-Code/blob/main/README.md" target="_blank">π Full README</a> | |
| <a class="pill" href="https://huggingface.co/datasets/VEFX-Reward/VEFX-Bench" target="_blank">π¦ VEFX-Bench Dataset</a> | |
| <a class="pill" href="https://huggingface.co/VEFX-Reward/VEFX-Reward-4B" target="_blank">π€ VEFX-Reward-4B Model</a> | |
| </div> | |
| <h2>π What's in VEFX-Bench</h2> | |
| <p>5,049 annotated examples spanning 9 categories & 32 subcategories, evaluated by <strong>VEFX-Reward</strong> β a VLM-based reward model that scores edits across three dimensions on a 1β4 scale:</p> | |
| <div class="grid"> | |
| <div class="card"><h3>Instructional Following (IF)</h3><p>Does the edit accurately reflect the editing instruction?</p></div> | |
| <div class="card"><h3>Render Quality (RQ)</h3><p>Visual clarity, temporal consistency, physical plausibility.</p></div> | |
| <div class="card"><h3>Edit Exclusivity (EE)</h3><p>Were only the intended regions modified, without side-effects?</p></div> | |
| </div> | |
| <h2>π Model Leaderboard</h2> | |
| <p>VEFX-Reward scores on 1β4 scale. Ranked by <strong>GeoAgg</strong> (Ξ±=2 for IF, Ξ²=1 for RQ, Ξ³=1 for EE). Higher is better.</p> | |
| <table> | |
| <thead><tr><th class="num">Rank</th><th>Model</th><th>Type</th><th class="num">IF β</th><th class="num">RQ β</th><th class="num">EE β</th><th class="num">GeoAgg β</th></tr></thead> | |
| <tbody> | |
| <tr><td class="num">π₯</td><td><strong>Kling o3 Omni</strong></td><td>Commercial</td><td class="num">3.033</td><td class="num"><strong>3.588</strong></td><td class="num">3.043</td><td class="num"><strong>3.057</strong></td></tr> | |
| <tr><td class="num">π₯</td><td><strong>Kling o1</strong></td><td>Commercial</td><td class="num"><strong>3.040</strong></td><td class="num">3.534</td><td class="num">2.976</td><td class="num">2.985</td></tr> | |
| <tr><td class="num">π₯</td><td><strong>Runway Gen-4.5</strong></td><td>Commercial</td><td class="num">2.817</td><td class="num">3.319</td><td class="num">2.923</td><td class="num">2.912</td></tr> | |
| <tr><td class="num">4</td><td>Seedance 2.0</td><td>Commercial</td><td class="num">2.811</td><td class="num">3.421</td><td class="num">3.088</td><td class="num">2.766</td></tr> | |
| <tr><td class="num">5</td><td>Grok Imagine</td><td>Commercial</td><td class="num">2.606</td><td class="num">3.346</td><td class="num"><strong>3.376</strong></td><td class="num">2.723</td></tr> | |
| <tr><td class="num">6</td><td>Luma Ray 3</td><td>Commercial</td><td class="num">2.702</td><td class="num">3.403</td><td class="num">2.705</td><td class="num">2.717</td></tr> | |
| <tr><td class="num">7</td><td>UniVideo</td><td>Open-source</td><td class="num">2.294</td><td class="num">3.266</td><td class="num">3.091</td><td class="num">2.516</td></tr> | |
| <tr><td class="num">8</td><td>Wan 2.6</td><td>Commercial</td><td class="num">2.012</td><td class="num">3.317</td><td class="num">2.446</td><td class="num">2.146</td></tr> | |
| <tr><td class="num">9</td><td>Luma Ray 2</td><td>Commercial</td><td class="num">2.038</td><td class="num">2.532</td><td class="num">1.363</td><td class="num">1.804</td></tr> | |
| <tr><td class="num">10</td><td>VACE</td><td>Open-source</td><td class="num">2.027</td><td class="num">3.172</td><td class="num">1.180</td><td class="num">1.775</td></tr> | |
| </tbody> | |
| </table> | |
| <h2>π₯ Sample Videos</h2> | |
| <p>An example pair from <code>examples/sample_videos/</code> β original input on the left, edited output on the right.</p> | |
| <div class="grid"> | |
| <div><p style="margin:0 0 .3em;font-size:.9em;color:var(--muted);"><strong>Original</strong></p> | |
| <video controls style="width:100%;border-radius:6px;" src="examples/sample_videos/original.mp4"></video></div> | |
| <div><p style="margin:0 0 .3em;font-size:.9em;color:var(--muted);"><strong>Edited</strong></p> | |
| <video controls style="width:100%;border-radius:6px;" src="examples/sample_videos/edited.mp4"></video></div> | |
| </div> | |
| <h2>π Quick Start</h2> | |
| <pre><code>pip install -r requirements.txt | |
| python examples/quick_start.py \ | |
| --original examples/sample_videos/original.mp4 \ | |
| --edited examples/sample_videos/edited.mp4 \ | |
| --instruction "Change the color of the trailer to bright yellow"</code></pre> | |
| <p>See <a href="https://huggingface.co/spaces/VEFX-Reward/VEFX-Code/tree/main/examples" target="_blank">examples/</a> for batch & multi-GPU scoring scripts.</p> | |
| <footer> | |
| Released under Apache-2.0. Code lives in the <a href="https://huggingface.co/spaces/VEFX-Reward/VEFX-Code/tree/main" target="_blank">Files tab</a>. | |
| </footer> | |
| </body> | |
| </html> | |