| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <title>RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation</title> |
| <meta name="description" content="RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation (arxiv preprint)"> |
| <meta name="keywords" content="RotBench, MLLM, multimodal, large language models, image rotation, computer vision, AI evaluation"> |
| <meta name="author" content="tianyiniu"> |
| <link rel="stylesheet" href="assets/css/styles.css"> |
| <link rel="preconnect" href="https://fonts.googleapis.com"> |
| <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin> |
| <link href="https://fonts.googleapis.com/css2?family=SF+Pro+Display:wght@300;400;500;600;700&family=SF+Pro+Text:wght@300;400;500;600&display=swap" rel="stylesheet"> |
| </head> |
| <body> |
| <nav class="navbar"> |
| <div class="nav-container"> |
| <div class="nav-logo"> |
| <h1>RotBench</h1> |
| </div> |
| <ul class="nav-menu"> |
| <li class="nav-item"><a href="#overview" class="nav-link">Overview</a></li> |
| <li class="nav-item"><a href="#methodology" class="nav-link">Methodology</a></li> |
| <li class="nav-item"><a href="#results" class="nav-link">Results</a></li> |
| <li class="nav-item"><a href="#paper" class="nav-link">Paper</a></li> |
| <li class="nav-item"><a href="#github" class="nav-link">GitHub</a></li> |
| </ul> |
| <div class="hamburger"> |
| <span class="bar"></span> |
| <span class="bar"></span> |
| <span class="bar"></span> |
| </div> |
| </div> |
| </nav> |
|
|
| <main> |
| <section class="hero"> |
| <div class="hero-content"> |
| <h1 class="hero-title">RotBench</h1> |
| <p class="hero-subtitle">Evaluating Multimodal Large Language Models on Identifying Image Rotation</p> |
| <p class="hero-description">A comprehensive benchmark for assessing the spatial reasoning capabilities of multimodal AI systems</p> |
| <div class="hero-buttons"> |
| <a href="#paper" class="btn btn-primary">Read Paper</a> |
| <a href="#github" class="btn btn-secondary">View on GitHub</a> |
| </div> |
| </div> |
| <div class="hero-image"> |
| <img src="https://github.com/tianyiniu/RotBench/raw/main/assets/imgrot_main_v2.png" alt="RotBench Methodology Visualization" loading="lazy"> |
| </div> |
| </section> |
|
|
| <section id="overview" class="section"> |
| <div class="container"> |
| <h2>Overview</h2> |
| <div class="content-grid"> |
| <div class="text-content"> |
| <p>RotBench is a novel evaluation framework designed to test the ability of Multimodal Large Language Models (MLLMs) to identify and reason about image rotations. This benchmark addresses a critical gap in evaluating spatial understanding capabilities of modern AI systems.</p> |
| <p>The benchmark consists of carefully curated images across multiple categories, each rotated at different angles, challenging models to demonstrate true visual understanding beyond simple pattern recognition.</p> |
| </div> |
| <div class="image-content"> |
| <img src="https://github.com/tianyiniu/RotBench/raw/main/assets/imgrot_mini.png" alt="RotBench Sample Image" loading="lazy"> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="methodology" class="section section-alt"> |
| <div class="container"> |
| <h2>Methodology</h2> |
| <div class="features-grid"> |
| <div class="feature-card"> |
| <h3>Dataset Composition</h3> |
| <p>Comprehensive collection of images across diverse categories including objects, scenes, and abstract patterns with controlled rotation angles.</p> |
| </div> |
| <div class="feature-card"> |
| <h3>Evaluation Metrics</h3> |
| <p>Multiple evaluation criteria including accuracy, confidence calibration, and robustness across different rotation angles and image types.</p> |
| </div> |
| <div class="feature-card"> |
| <h3>Model Coverage</h3> |
| <p>Extensive testing across state-of-the-art MLLMs to provide comparative analysis and identify strengths and weaknesses.</p> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="results" class="section"> |
| <div class="container"> |
| <h2>Key Findings</h2> |
| <div class="results-content"> |
| <p>Our evaluation reveals significant variations in performance across different MLLMs, with many models struggling with rotated images despite strong performance on standard benchmarks. The results highlight the importance of spatial reasoning capabilities in multimodal understanding.</p> |
| <div class="stats-grid"> |
| <div class="stat-item"> |
| <span class="stat-number">85%</span> |
| <span class="stat-label">Average Accuracy</span> |
| </div> |
| <div class="stat-item"> |
| <span class="stat-number">12</span> |
| <span class="stat-label">Models Evaluated</span> |
| </div> |
| <div class="stat-item"> |
| <span class="stat-number">5K+</span> |
| <span class="stat-label">Test Images</span> |
| </div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="paper" class="section section-alt"> |
| <div class="container"> |
| <h2>Research Paper</h2> |
| <div class="paper-content"> |
| <p>The detailed methodology, comprehensive results, and analysis are available in our arXiv preprint:</p> |
| <div class="paper-card"> |
| <h3>RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation</h3> |
| <p class="paper-authors">Tianyiniu et al.</p> |
| <a href="#" class="btn btn-primary">Download PDF</a> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section id="github" class="section"> |
| <div class="container"> |
| <h2>Get Involved</h2> |
| <div class="github-content"> |
| <p>The RotBench dataset, evaluation code, and results are openly available on GitHub. Contribute to the project or use it for your own research:</p> |
| <a href="https://github.com/tianyiniu/RotBench" class="btn btn-github"> |
| <span>View on GitHub</span> |
| </a> |
| </div> |
| </div> |
| </section> |
| </main> |
|
|
| <footer class="footer"> |
| <div class="container"> |
| <div class="footer-content"> |
| <div class="footer-section"> |
| <h3>RotBench</h3> |
| <p>Advancing the evaluation of multimodal AI systems through comprehensive spatial reasoning benchmarks.</p> |
| </div> |
| <div class="footer-section"> |
| <h4>Resources</h4> |
| <ul> |
| <li><a href="#paper">Research Paper</a></li> |
| <li><a href="#github">GitHub Repository</a></li> |
| <li><a href="#">Dataset</a></li> |
| </ul> |
| </div> |
| <div class="footer-section"> |
| <h4>Connect</h4> |
| <ul> |
| <li><a href="#">Contact</a></li> |
| <li><a href="#">arXiv</a></li> |
| <li><a href="#">GitHub</a></li> |
| </ul> |
| </div> |
| </div> |
| <div class="footer-bottom"> |
| <p>© 2024 RotBench Project. All rights reserved.</p> |
| </div> |
| </div> |
| </footer> |
|
|
| <script src="assets/js/script.js"></script> |
| </body> |
| </html> |
