Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <meta name="description" content="Official StreamAV-Bench leaderboard for streaming audio-video generation."> | |
| <meta name="theme-color" content="#f5f2ea"> | |
| <meta property="og:title" content="StreamAV-Bench Leaderboard"> | |
| <meta property="og:description" content="Metric-wise evaluation of streaming audio-video generation systems."> | |
| <meta property="og:type" content="website"> | |
| <title>StreamAV-Bench Leaderboard</title> | |
| <link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 64 64'%3E%3Crect width='64' height='64' rx='16' fill='%2318231f'/%3E%3Cpath d='M12 35h8l4-14 7 28 7-37 5 23h9' fill='none' stroke='%23fffef9' stroke-width='5' stroke-linecap='round' stroke-linejoin='round'/%3E%3C/svg%3E"> | |
| <link rel="preconnect" href="https://fonts.googleapis.com"> | |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> | |
| <link href="https://fonts.googleapis.com/css2?family=DM+Sans:wght@400;500;600;700&family=DM+Serif+Display&display=swap" rel="stylesheet"> | |
| <link rel="stylesheet" href="styles.css"> | |
| <script src="app.js" defer></script> | |
| </head> | |
| <body> | |
| <a class="skip-link" href="#leaderboard">Skip to leaderboard</a> | |
| <header class="site-header"> | |
| <div class="shell header-inner"> | |
| <a class="brand" href="https://liukqchoco.github.io/StreamAVBench/" target="_blank" rel="noopener noreferrer"> | |
| <svg class="brand-mark" viewBox="0 0 42 42" role="img" aria-label="StreamAV-Bench"> | |
| <rect x="1" y="1" width="40" height="40" rx="11" fill="currentColor"/> | |
| <path d="M9 23h4l2.6-7 4.2 14L24 11l3.4 12H33" fill="none" stroke="#fffdf7" stroke-width="2.8" stroke-linecap="round" stroke-linejoin="round"/> | |
| <circle cx="33" cy="23" r="2.2" fill="#ef9b77"/> | |
| </svg> | |
| <span> | |
| <strong>StreamAV-Bench</strong> | |
| <small>Leaderboard</small> | |
| </span> | |
| </a> | |
| <nav aria-label="Primary navigation"> | |
| <a href="#leaderboard">Results</a> | |
| <a href="#metrics">Metrics</a> | |
| </nav> | |
| </div> | |
| </header> | |
| <main> | |
| <section class="hero"> | |
| <div class="hero-grid" aria-hidden="true"></div> | |
| <div class="hero-orb hero-orb-one" aria-hidden="true"></div> | |
| <div class="hero-orb hero-orb-two" aria-hidden="true"></div> | |
| <div class="shell hero-inner"> | |
| <div class="hero-copy"> | |
| <h1>StreamAV-Bench <em>Leaderboard</em></h1> | |
| <p class="hero-tagline" id="hero-tagline">Streaming systems, evaluated while they run.</p> | |
| <p class="hero-description" id="hero-description"> | |
| Official metric-wise results for streaming audio-video generation across progressive and interactive evaluation tracks. | |
| </p> | |
| <div class="hero-actions"> | |
| <a class="button button-primary" href="#leaderboard"> | |
| Explore results <span aria-hidden="true">↓</span> | |
| </a> | |
| </div> | |
| <nav class="hero-resources" aria-label="Project resources"> | |
| <a id="paper-link" class="hero-resource is-disabled" aria-disabled="true"> | |
| <span class="hero-resource-icon">▤</span> | |
| <span><strong>Paper</strong><small data-link-status>Coming soon</small></span> | |
| </a> | |
| <a id="project-link" class="hero-resource" href="https://liukqchoco.github.io/StreamAVBench/" target="_blank" rel="noopener noreferrer"> | |
| <span class="hero-resource-icon">↗</span> | |
| <span><strong>Project</strong><small data-link-status>Project page</small></span> | |
| </a> | |
| <a id="code-link" class="hero-resource" href="https://github.com/liukqchoco/StreamAVBench-Code" target="_blank" rel="noopener noreferrer"> | |
| <span class="hero-resource-icon"></></span> | |
| <span><strong>Code</strong><small data-link-status>Code repository</small></span> | |
| </a> | |
| <a id="dataset-link" class="hero-resource is-disabled" aria-disabled="true"> | |
| <span class="hero-resource-icon">≋</span> | |
| <span><strong>Dataset</strong><small data-link-status>Coming soon</small></span> | |
| </a> | |
| </nav> | |
| </div> | |
| <div class="stream-card"> | |
| <div class="stream-card-header"> | |
| <span>Evaluation signal</span> | |
| <i><b></b> 180-second rollout</i> | |
| </div> | |
| <div class="signal-group signal-progressive"> | |
| <div class="signal-label"> | |
| <strong>Progressive</strong> | |
| <small>one global prompt</small> | |
| </div> | |
| <div class="signal-track"> | |
| <span style="--level: 82%"></span> | |
| <span style="--level: 76%"></span> | |
| <span style="--level: 70%"></span> | |
| <span style="--level: 66%"></span> | |
| <span style="--level: 59%"></span> | |
| <span style="--level: 54%"></span> | |
| </div> | |
| <div class="signal-times"><span>0</span><span>60</span><span>120</span><span>180s</span></div> | |
| </div> | |
| <div class="signal-divider"><span>committed history</span></div> | |
| <div class="signal-group signal-interactive"> | |
| <div class="signal-label"> | |
| <strong>Interactive</strong> | |
| <small>five runtime updates</small> | |
| </div> | |
| <div class="update-track"> | |
| <span>P0</span><i></i><span>U1</span><i></i><span>U2</span><i></i><span>U3</span><i></i><span>U4</span><i></i><span>U5</span> | |
| </div> | |
| <div class="response-line"> | |
| <b style="--delay: 18%"></b> | |
| <b style="--delay: 42%"></b> | |
| <b style="--delay: 31%"></b> | |
| <b style="--delay: 54%"></b> | |
| <b style="--delay: 38%"></b> | |
| </div> | |
| </div> | |
| <p class="stream-card-note"><span>◆</span> Scores expose change over time—not only the final output.</p> | |
| </div> | |
| </div> | |
| </section> | |
| <section class="stat-band" aria-label="Benchmark statistics"> | |
| <div class="shell stats"> | |
| <div><strong id="stat-systems">13</strong><span>evaluated systems</span></div> | |
| <div><strong id="stat-metrics">32</strong><span>core metrics</span></div> | |
| <div><strong>2</strong><span>streaming tracks</span></div> | |
| <div><strong id="stat-scenarios">320</strong><span>verified scenarios</span></div> | |
| <div><strong id="stat-duration">180s</strong><span>continuous generation</span></div> | |
| </div> | |
| </section> | |
| <section class="section leaderboard-section" id="leaderboard"> | |
| <div class="shell"> | |
| <div class="section-heading results-heading"> | |
| <p class="kicker">Official results</p> | |
| <h2>Compare systems by metric.</h2> | |
| <p class="section-intro">Choose a metric below to rank systems. Metrics use different scales and are therefore reported separately.</p> | |
| </div> | |
| <div class="explorer"> | |
| <div class="track-switcher"> | |
| <div class="track-tabs" id="track-tabs" role="tablist" aria-label="Evaluation views"></div> | |
| <p id="track-description">Quality, alignment, continuity, and efficiency metrics applied across both tracks.</p> | |
| </div> | |
| <div class="category-bar"> | |
| <span>View</span> | |
| <div class="category-tabs" id="category-tabs" aria-label="Metric categories"></div> | |
| </div> | |
| <div class="table-panel"> | |
| <div class="table-panel-header"> | |
| <div> | |
| <p class="table-kicker" id="table-kicker">Metric-wise ranking</p> | |
| <h3 id="category-title">Quality & Alignment</h3> | |
| <p id="category-description">Perceptual quality and semantic or temporal correspondence between audio and video.</p> | |
| </div> | |
| <p class="ranking-label" id="ranking-label">Ranked by <strong>Visual Aesthetics</strong><span>Higher is better</span></p> | |
| </div> | |
| <div class="toolbar"> | |
| <label class="search-control"> | |
| <span class="sr-only">Search systems</span> | |
| <svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="6.5"></circle><path d="m16 16 4 4"></path></svg> | |
| <input id="model-search" type="search" placeholder="Search systems..." autocomplete="off"> | |
| </label> | |
| <label class="metric-select-control"> | |
| <span>Rank by</span> | |
| <select id="metric-select" aria-label="Select the metric used to rank systems"></select> | |
| </label> | |
| <label> | |
| <span>Paradigm</span> | |
| <select id="paradigm-filter"> | |
| <option value="all">All paradigms</option> | |
| <option value="Native joint">Native joint</option> | |
| <option value="Cascaded">Cascaded</option> | |
| </select> | |
| </label> | |
| <label> | |
| <span>Source</span> | |
| <select id="access-filter"> | |
| <option value="all">All sources</option> | |
| <option value="Open">Open-source</option> | |
| <option value="Web">Closed-source</option> | |
| </select> | |
| </label> | |
| <button class="tool-button" id="reset-filters" type="button" title="Reset filters"> | |
| <span aria-hidden="true">↺</span> Reset | |
| </button> | |
| <button class="tool-button download-button" id="download-csv" type="button"> | |
| <span aria-hidden="true">↓</span> CSV | |
| </button> | |
| </div> | |
| <div class="data-error" id="data-error" role="alert" hidden> | |
| <strong>Leaderboard unavailable</strong> | |
| <p>The leaderboard data could not be loaded.</p> | |
| </div> | |
| <div class="table-scroll" tabindex="0" aria-label="Scrollable leaderboard table"> | |
| <table id="leaderboard-table"> | |
| <caption class="sr-only">StreamAV-Bench system results</caption> | |
| </table> | |
| </div> | |
| <div class="table-footer"> | |
| <p id="table-status" aria-live="polite">13 of 13 systems</p> | |
| <div class="legend" aria-label="Cell highlighting legend"> | |
| <span><i class="legend-best"></i> Best published value</span> | |
| <span><i class="legend-second"></i> Second distinct value</span> | |
| <span><b>—</b> Unavailable</span> | |
| </div> | |
| </div> | |
| </div> | |
| <div class="evidence-note"> | |
| <span aria-hidden="true">i</span> | |
| <div id="context-copy"> | |
| <p><strong>Reading the ranks.</strong> Click any metric header to rank by that metric. Published three-decimal ties use competition ranking (for example, 1, 1, 3). PVRL and PARL are conditional on successful target achievement and should be read with PVUAR and PAUAR.</p> | |
| <p>HDF is evaluated only when its required source state was established. NBC, FPS, and TTFC are unavailable for PixVerse R1, HappyOyster, and Odyssey-2 because their web interfaces do not expose the required traces.</p> | |
| </div> | |
| </div> | |
| </div> | |
| </div> | |
| </section> | |
| <section class="section metrics-section" id="metrics"> | |
| <div class="shell"> | |
| <div class="section-heading split-heading"> | |
| <div> | |
| <p class="kicker">Metric guide</p> | |
| <h2>Know what each number measures.</h2> | |
| </div> | |
| <p class="section-intro">The cards update with the selected leaderboard category. Choose a card to rank the table by that metric.</p> | |
| </div> | |
| <div class="metric-guide" id="metric-guide"></div> | |
| </div> | |
| </section> | |
| <section class="section protocol-section"> | |
| <div class="shell"> | |
| <div class="section-heading"> | |
| <p class="kicker">How to read StreamAV-Bench</p> | |
| <h2>Performance is a profile, not a single number.</h2> | |
| </div> | |
| <div class="protocol-grid"> | |
| <article> | |
| <span>01</span> | |
| <h3>Choose the evaluation track</h3> | |
| <p>Use Progressive for fixed-prompt long-horizon behavior and Interactive for runtime response, retention, and reuse.</p> | |
| </article> | |
| <article> | |
| <span>02</span> | |
| <h3>Read score and drift together</h3> | |
| <p>High quality or fulfillment can coexist with temporal drift. Read absolute scores alongside their drift counterparts.</p> | |
| </article> | |
| <article> | |
| <span>03</span> | |
| <h3>Read success rate with latency</h3> | |
| <p>Low conditional latency does not guarantee reliable updates. Read response latency together with achievement rate.</p> | |
| </article> | |
| </div> | |
| </div> | |
| </section> | |
| </main> | |
| <footer> | |
| <div class="shell footer-inner"> | |
| <div class="footer-brand"> | |
| <svg class="brand-mark" viewBox="0 0 42 42" aria-hidden="true"> | |
| <rect x="1" y="1" width="40" height="40" rx="11" fill="currentColor"/> | |
| <path d="M9 23h4l2.6-7 4.2 14L24 11l3.4 12H33" fill="none" stroke="#fffdf7" stroke-width="2.8" stroke-linecap="round" stroke-linejoin="round"/> | |
| <circle cx="33" cy="23" r="2.2" fill="#ef9b77"/> | |
| </svg> | |
| <span><strong>StreamAV-Bench</strong><small>Persistent · responsive · synchronized</small></span> | |
| </div> | |
| <p>Official benchmark results. Scores are reproduced from the final paper tables.</p> | |
| <a href="#leaderboard">Back to results ↑</a> | |
| </div> | |
| </footer> | |
| <noscript> | |
| <div class="noscript">JavaScript is required to render and sort the leaderboard.</div> | |
| </noscript> | |
| </body> | |
| </html> | |