File size: 13,222 Bytes
ce65fef fcb188b ce65fef 3c9e74a ce65fef fcb188b ce65fef 3c9e74a ce65fef 3c9e74a ce65fef 3c9e74a ce65fef fcb188b 261f212 fcb188b ce65fef fcb188b ce65fef fcb188b ce65fef fcb188b ce65fef fcb188b 3c9e74a fcb188b ce65fef fcb188b 261f212 fcb188b ce65fef f2a5900 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 | <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>LLM Tool-Calling Models: Cost & Context Analysis</title>
<link rel="stylesheet" href="style.css">
<script src="https://cdn.jsdelivr.net/npm/chart.js@4.4.0/dist/chart.umd.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/chartjs-plugin-annotation@3.0.1/dist/chartjs-plugin-annotation.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/papaparse@5.4.1/papaparse.min.js"></script>
</head>
<body>
<header>
<h1>Agentic LLM Price Comparison</h1>
<p class="subtitle">Cost-effectiveness analysis of 218 tool-calling models from OpenRouter</p>
<p class="date">Data snapshot: November 8, 2025</p>
<nav class="header-nav">
<a href="output-input-analysis.html">🔢 Output/Input Price Analysis →</a>
</nav>
</header>
<main>
<section class="intro">
<h2>Overview</h2>
<p>This analysis examines LLM models with tool-calling capabilities - essential for agentic AI systems, MCP (Model Context Protocol) implementations, and multi-tool orchestration. Models are evaluated across two key dimensions:</p>
<ul>
<li><strong>Cost per token</strong> - Economic efficiency for production use</li>
<li><strong>Context window size</strong> - Capability for complex, long-form tasks</li>
</ul>
<div class="attribution">
<h3>Methodology</h3>
<p><strong>Quadrant Classification:</strong> Models are categorized into four quadrants based on median values calculated from the dataset:</p>
<ul>
<li><strong>Cost Division:</strong> Median average cost (mean of input + output pricing) = <span id="median-cost-display">calculated from data</span></li>
<li><strong>Context Division:</strong> Median context window size = <span id="median-context-display">calculated from data</span></li>
</ul>
<p>This creates four categories: Low Cost/High Context, High Cost/High Context, Low Cost/Low Context, and High Cost/Low Context.</p>
<h3>Attribution & Data Notes</h3>
<ul>
<li><strong>Data Collection:</strong> <a href="https://danielrosehill.com" target="_blank">Daniel Rosehill</a></li>
<li><strong>Collection Date:</strong> November 8, 2025</li>
<li><strong>Source:</strong> Prices derived from API calls to <a href="https://openrouter.ai/" target="_blank">OpenRouter</a></li>
<li><strong>Pricing Variability:</strong> OpenRouter pricing can fluctuate slightly according to the end inference provider. The same models may have slightly different pricing even at the time of capture.</li>
<li><strong>Data Exclusions:</strong> Free models (cost = $0) were excluded from this analysis</li>
</ul>
</div>
</section>
<section class="stats-grid">
<div class="stat-card">
<div class="stat-number" id="total-models">218</div>
<div class="stat-label">Models Analyzed</div>
</div>
<div class="stat-card">
<div class="stat-number" id="vendors">-</div>
<div class="stat-label">Vendors</div>
</div>
<div class="stat-card">
<div class="stat-number" id="median-input">-</div>
<div class="stat-label">Median Input ($/M)</div>
</div>
<div class="stat-card">
<div class="stat-number" id="median-output">-</div>
<div class="stat-label">Median Output ($/M)</div>
</div>
<div class="stat-card">
<div class="stat-number" id="median-multiple">-</div>
<div class="stat-label">Median Out/In Multiple</div>
</div>
<div class="stat-card">
<div class="stat-number" id="context-range">-</div>
<div class="stat-label">Context Range</div>
</div>
</section>
<div class="tabs">
<button class="tab-button active" data-tab="models">All Models</button>
<button class="tab-button" data-tab="quadrants">Quadrant Analysis</button>
<button class="tab-button" data-tab="analysis">Visual Analysis</button>
</div>
<div class="tab-content active" id="tab-models">
<section class="data-notes">
<h3>Data Notes</h3>
<ul>
<li><strong>Cost Calculation:</strong> Input and output pricing shown separately (per million tokens)</li>
<li><strong>Data Exclusions:</strong> Free models (cost = $0) were excluded from this analysis</li>
<li><strong>Color Coding:</strong> Cells are color-coded by tier - see legend below for ranges</li>
</ul>
</section>
<section class="data-table-section">
<h2>All Models - Comprehensive View</h2>
<p>Complete dataset with color-coded cost and context tiers. Click column headers to sort.</p>
<div class="tier-legends">
<div class="legend-group">
<h4>Cost Tiers ($/M tokens)</h4>
<div class="legend-items">
<div class="legend-item"><span class="tier-indicator cost-very-low"></span> Very Low (< $0.10)</div>
<div class="legend-item"><span class="tier-indicator cost-low"></span> Low ($0.10 - $0.50)</div>
<div class="legend-item"><span class="tier-indicator cost-medium"></span> Medium ($0.50 - $2.00)</div>
<div class="legend-item"><span class="tier-indicator cost-high"></span> High ($2.00 - $5.00)</div>
<div class="legend-item"><span class="tier-indicator cost-very-high"></span> Very High (> $5.00)</div>
</div>
</div>
<div class="legend-group">
<h4>Context Window Tiers</h4>
<div class="legend-items">
<div class="legend-item"><span class="tier-indicator context-small"></span> Small (< 32K)</div>
<div class="legend-item"><span class="tier-indicator context-medium"></span> Medium (32K - 128K)</div>
<div class="legend-item"><span class="tier-indicator context-large"></span> Large (128K - 256K)</div>
<div class="legend-item"><span class="tier-indicator context-very-large"></span> Very Large (256K - 1M)</div>
<div class="legend-item"><span class="tier-indicator context-ultra"></span> Ultra (> 1M)</div>
</div>
</div>
</div>
<div class="table-controls">
<input type="text" id="search" placeholder="Search models..." />
<select id="quadrant-filter">
<option value="">All Quadrants</option>
<option value="Low Cost / High Context">Low Cost / High Context</option>
<option value="High Cost / High Context">High Cost / High Context</option>
<option value="Low Cost / Low Context">Low Cost / Low Context</option>
<option value="High Cost / Low Context">High Cost / Low Context</option>
</select>
</div>
<div class="table-wrapper">
<table id="models-table">
<thead>
<tr>
<th class="sortable" data-sort="model_name">Model Name <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="displayVendor">Vendor <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="context_length">Context <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="input_price_usd_per_m">Input ($/M) <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="output_price_usd_per_m">Output ($/M) <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="output_input_multiple">Out/In Multiple <span class="sort-indicator"></span></th>
<th class="sortable" data-sort="quadrant">Quadrant <span class="sort-indicator"></span></th>
</tr>
</thead>
<tbody id="table-body">
<!-- Populated by JavaScript -->
</tbody>
</table>
</div>
</section>
</div>
<div class="tab-content" id="tab-quadrants">
<section class="data-notes">
<h3>Quadrant Analysis</h3>
<ul>
<li><strong>Overview:</strong> Models divided into four quadrants based on median cost and context values</li>
<li><strong>Green (Low Cost / High Context):</strong> Best value - low prices with large context windows</li>
<li><strong>Blue (High Cost / High Context):</strong> Premium models with extensive context capabilities</li>
<li><strong>Orange (Low Cost / Low Context):</strong> Budget-friendly for simple tasks</li>
<li><strong>Red (High Cost / Low Context):</strong> Specialized models with limited context</li>
</ul>
</section>
<section class="chart-section">
<h2>All Quadrants Overview</h2>
<p>Complete view showing all models across quadrants with median division lines.</p>
<div class="chart-container">
<canvas id="allQuadrantsChart"></canvas>
</div>
</section>
<section class="quadrant-details">
<h2>Individual Quadrant Deep Dive</h2>
<p>Detailed view of each quadrant's models for easier comparison within each category.</p>
<div class="quadrant-grid">
<div class="quadrant-detail">
<h3 style="color: #10b981;">Low Cost / High Context</h3>
<div class="chart-container-small">
<canvas id="quadrant-lchc"></canvas>
</div>
</div>
<div class="quadrant-detail">
<h3 style="color: #2563eb;">High Cost / High Context</h3>
<div class="chart-container-small">
<canvas id="quadrant-hchc"></canvas>
</div>
</div>
<div class="quadrant-detail">
<h3 style="color: #f59e0b;">Low Cost / Low Context</h3>
<div class="chart-container-small">
<canvas id="quadrant-lclc"></canvas>
</div>
</div>
<div class="quadrant-detail">
<h3 style="color: #ef4444;">High Cost / Low Context</h3>
<div class="chart-container-small">
<canvas id="quadrant-hclc"></canvas>
</div>
</div>
</div>
</section>
</div>
<div class="tab-content" id="tab-analysis">
<section class="data-notes">
<h3>Analysis Notes</h3>
<ul>
<li><strong>Quadrant Divisions:</strong> Based on median values for cost and context window size</li>
<li><strong>Logarithmic Scale:</strong> Cost axis uses log scale to better visualize the wide price range</li>
<li><strong>Median Lines:</strong> Dashed lines show median cost and context values across all models</li>
</ul>
</section>
<section class="chart-section">
<h2>Cost vs Context Quadrant Analysis</h2>
<p>Interactive scatter plot showing the relationship between pricing and context window size. Hover over points for detailed model information.</p>
<div class="chart-container">
<canvas id="quadrantChart"></canvas>
</div>
</section>
</div>
</main>
<footer>
<p>Data source: <a href="https://openrouter.ai/" target="_blank">OpenRouter API</a> | Analysis: <a href="https://github.com/danielrosehill/OR-Models-With-Tools-0811" target="_blank">GitHub Repository</a></p>
<p class="disclaimer">Prices are subject to change. For current pricing, always refer to OpenRouter's live API.</p>
</footer>
<script src="app.js"></script>
</body>
</html>
|