File size: 8,666 Bytes
f9b8369 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 | <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation</title>
<meta name="description" content="RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation (arxiv preprint)">
<meta name="keywords" content="RotBench, MLLM, multimodal, large language models, image rotation, computer vision, AI evaluation">
<meta name="author" content="tianyiniu">
<link rel="stylesheet" href="assets/css/styles.css">
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=SF+Pro+Display:wght@300;400;500;600;700&family=SF+Pro+Text:wght@300;400;500;600&display=swap" rel="stylesheet">
</head>
<body>
<nav class="navbar">
<div class="nav-container">
<div class="nav-logo">
<h1>RotBench</h1>
</div>
<ul class="nav-menu">
<li class="nav-item"><a href="#overview" class="nav-link">Overview</a></li>
<li class="nav-item"><a href="#methodology" class="nav-link">Methodology</a></li>
<li class="nav-item"><a href="#results" class="nav-link">Results</a></li>
<li class="nav-item"><a href="#paper" class="nav-link">Paper</a></li>
<li class="nav-item"><a href="#github" class="nav-link">GitHub</a></li>
</ul>
<div class="hamburger">
<span class="bar"></span>
<span class="bar"></span>
<span class="bar"></span>
</div>
</div>
</nav>
<main>
<section class="hero">
<div class="hero-content">
<h1 class="hero-title">RotBench</h1>
<p class="hero-subtitle">Evaluating Multimodal Large Language Models on Identifying Image Rotation</p>
<p class="hero-description">A comprehensive benchmark for assessing the spatial reasoning capabilities of multimodal AI systems</p>
<div class="hero-buttons">
<a href="#paper" class="btn btn-primary">Read Paper</a>
<a href="#github" class="btn btn-secondary">View on GitHub</a>
</div>
</div>
<div class="hero-image">
<img src="https://github.com/tianyiniu/RotBench/raw/main/assets/imgrot_main_v2.png" alt="RotBench Methodology Visualization" loading="lazy">
</div>
</section>
<section id="overview" class="section">
<div class="container">
<h2>Overview</h2>
<div class="content-grid">
<div class="text-content">
<p>RotBench is a novel evaluation framework designed to test the ability of Multimodal Large Language Models (MLLMs) to identify and reason about image rotations. This benchmark addresses a critical gap in evaluating spatial understanding capabilities of modern AI systems.</p>
<p>The benchmark consists of carefully curated images across multiple categories, each rotated at different angles, challenging models to demonstrate true visual understanding beyond simple pattern recognition.</p>
</div>
<div class="image-content">
<img src="https://github.com/tianyiniu/RotBench/raw/main/assets/imgrot_mini.png" alt="RotBench Sample Image" loading="lazy">
</div>
</div>
</div>
</section>
<section id="methodology" class="section section-alt">
<div class="container">
<h2>Methodology</h2>
<div class="features-grid">
<div class="feature-card">
<h3>Dataset Composition</h3>
<p>Comprehensive collection of images across diverse categories including objects, scenes, and abstract patterns with controlled rotation angles.</p>
</div>
<div class="feature-card">
<h3>Evaluation Metrics</h3>
<p>Multiple evaluation criteria including accuracy, confidence calibration, and robustness across different rotation angles and image types.</p>
</div>
<div class="feature-card">
<h3>Model Coverage</h3>
<p>Extensive testing across state-of-the-art MLLMs to provide comparative analysis and identify strengths and weaknesses.</p>
</div>
</div>
</div>
</section>
<section id="results" class="section">
<div class="container">
<h2>Key Findings</h2>
<div class="results-content">
<p>Our evaluation reveals significant variations in performance across different MLLMs, with many models struggling with rotated images despite strong performance on standard benchmarks. The results highlight the importance of spatial reasoning capabilities in multimodal understanding.</p>
<div class="stats-grid">
<div class="stat-item">
<span class="stat-number">85%</span>
<span class="stat-label">Average Accuracy</span>
</div>
<div class="stat-item">
<span class="stat-number">12</span>
<span class="stat-label">Models Evaluated</span>
</div>
<div class="stat-item">
<span class="stat-number">5K+</span>
<span class="stat-label">Test Images</span>
</div>
</div>
</div>
</div>
</section>
<section id="paper" class="section section-alt">
<div class="container">
<h2>Research Paper</h2>
<div class="paper-content">
<p>The detailed methodology, comprehensive results, and analysis are available in our arXiv preprint:</p>
<div class="paper-card">
<h3>RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotation</h3>
<p class="paper-authors">Tianyiniu et al.</p>
<a href="#" class="btn btn-primary">Download PDF</a>
</div>
</div>
</div>
</section>
<section id="github" class="section">
<div class="container">
<h2>Get Involved</h2>
<div class="github-content">
<p>The RotBench dataset, evaluation code, and results are openly available on GitHub. Contribute to the project or use it for your own research:</p>
<a href="https://github.com/tianyiniu/RotBench" class="btn btn-github">
<span>View on GitHub</span>
</a>
</div>
</div>
</section>
</main>
<footer class="footer">
<div class="container">
<div class="footer-content">
<div class="footer-section">
<h3>RotBench</h3>
<p>Advancing the evaluation of multimodal AI systems through comprehensive spatial reasoning benchmarks.</p>
</div>
<div class="footer-section">
<h4>Resources</h4>
<ul>
<li><a href="#paper">Research Paper</a></li>
<li><a href="#github">GitHub Repository</a></li>
<li><a href="#">Dataset</a></li>
</ul>
</div>
<div class="footer-section">
<h4>Connect</h4>
<ul>
<li><a href="#">Contact</a></li>
<li><a href="#">arXiv</a></li>
<li><a href="#">GitHub</a></li>
</ul>
</div>
</div>
<div class="footer-bottom">
<p>© 2024 RotBench Project. All rights reserved.</p>
</div>
</div>
</footer>
<script src="assets/js/script.js"></script>
</body>
</html> |