| <!DOCTYPE html> |
| <html lang="en"> |
| <head> |
| <meta charset="UTF-8"> |
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> |
| <title>LLMBench | AI Quality Engineering Platform</title> |
| |
| <link href="https://fonts.googleapis.com/css2?family=Playfair+Display:ital,wght@0,400..900;1,400..900&family=Plus+Jakarta+Sans:ital,wght@0,200..800;1,200..800&family=JetBrains+Mono:wght@400;500;600&display=swap" rel="stylesheet"> |
| |
| <script src="https://cdn.plot.ly/plotly-2.24.0.min.js"></script> |
| <link rel="stylesheet" href="style.css?v=1.0.1"> |
| </head> |
| <body> |
| |
| <div class="vp-corner tl"></div> |
| <div class="vp-corner tr"></div> |
| <div class="vp-corner bl"></div> |
| <div class="vp-corner br"></div> |
|
|
| |
| <header class="masthead"> |
| <div class="brand"> |
| <span class="brand-logo">LLMBench</span> |
| <span class="tagline">AI Quality & LLM Engineering Platform</span> |
| </div> |
| <nav class="sidebar-menu"> |
| <a href="#dashboard" class="menu-item active">Dashboard</a> |
| <a href="#datasets" class="menu-item">Datasets</a> |
| <a href="#arena" class="menu-item">Prompt Arena</a> |
| <a href="#hub" class="menu-item">Evaluation Hub</a> |
| <a href="#rca" class="menu-item">Agent RCA</a> |
| <a href="#cost" class="menu-item">Cost Analytics</a> |
| </nav> |
| <div class="top-actions"> |
| <a href="https://github.com/Jyotiraditya21-bug/LLMBench" target="_blank" class="top-btn github-btn"> |
| <svg width="14" height="14" viewBox="0 0 24 24" fill="currentColor" style="vertical-align:middle;margin-right:6px;"><path d="M12 0c-6.626 0-12 5.373-12 12 0 5.302 3.438 9.8 8.207 11.387.599.111.793-.261.793-.577v-2.234c-3.338.726-4.033-1.416-4.033-1.416-.546-1.387-1.333-1.756-1.333-1.756-1.089-.745.083-.729.083-.729 1.205.084 1.839 1.237 1.839 1.237 1.07 1.834 2.807 1.304 3.492.997.107-.775.418-1.305.762-1.604-2.665-.305-5.467-1.334-5.467-5.931 0-1.311.469-2.381 1.236-3.221-.124-.303-.535-1.524.117-3.176 0 0 1.008-.322 3.301 1.23.957-.266 1.983-.399 3.003-.404 1.02.005 2.047.138 3.006.404 2.291-1.552 3.297-1.23 3.297-1.23.653 1.653.242 2.874.118 3.176.77.84 1.235 1.911 1.235 3.221 0 4.609-2.807 5.624-5.479 5.921.43.372.823 1.102.823 2.222v3.293c0 .319.192.694.801.576 4.765-1.589 8.199-6.086 8.199-11.386 0-6.627-5.373-12-12-12z"/></svg>GitHub |
| </a> |
| <button class="top-btn guide-btn" onclick="openProductGuide()"> |
| How to Use |
| </button> |
| </div> |
| |
| <div class="scroll-progress-container"> |
| <div id="scroll-progress" class="scroll-progress-bar"></div> |
| </div> |
| </header> |
|
|
| <div class="app-layout"> |
|
|
| |
| <main class="main-content"> |
| |
| <section id="dashboard" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>Dashboard</h1> |
| <p class="tab-subtitle">LLM Evaluation, Regression Testing & AI Quality Monitoring Platform</p> |
| </header> |
| |
| <div class="kpi-grid"> |
| <div class="kpi-card"> |
| <div class="kpi-label">Total Datasets</div> |
| <div id="kpi-datasets" class="kpi-value">0</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Prompt Templates</div> |
| <div id="kpi-prompts" class="kpi-value">0</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Evaluation Runs</div> |
| <div id="kpi-runs" class="kpi-value">0</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Total Spent (USD)</div> |
| <div id="kpi-cost" class="kpi-value">$0.0000</div> |
| </div> |
| </div> |
|
|
| |
| <div class="flex-row" style="gap: 30px; margin-bottom: 30px; flex-wrap: wrap;"> |
| <div class="card col-half" style="margin: 0; flex: 1; min-width: 320px;"> |
| <div class="card-header-grip" style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 12px;"> |
| <h2 style="margin: 0;">Model Leaderboard</h2> |
| <div class="grip-handle"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></div> |
| </div> |
| <p class="card-description">Aggregated benchmarks graded by LLM-as-a-Judge.</p> |
| <div class="table-container"> |
| <table style="width: 100%; font-size: 0.9rem; border-collapse: collapse;"> |
| <thead> |
| <tr style="border-bottom: 1px solid var(--border-color); text-align: left;"> |
| <th style="padding: 8px 0;">Rank</th> |
| <th style="padding: 8px 0;">Model</th> |
| <th style="padding: 8px 0;">Accuracy</th> |
| <th style="padding: 8px 0;">Avg Latency</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr style="border-bottom: 1px solid rgba(194,125,56,0.08);"> |
| <td style="padding: 10px 0;">1</td> |
| <td style="padding: 10px 0;"><strong>Claude 3.5 Sonnet</strong></td> |
| <td style="padding: 10px 0;"><span class="badge badge-status-completed" style="font-size:0.75rem; padding: 2px 6px;">4.50 / 5.0</span></td> |
| <td style="padding: 10px 0; font-family: var(--font-mono); font-size: 0.8rem;">850 ms</td> |
| </tr> |
| <tr style="border-bottom: 1px solid rgba(194,125,56,0.08);"> |
| <td style="padding: 10px 0;">2</td> |
| <td style="padding: 10px 0;"><strong>GPT-4o</strong></td> |
| <td style="padding: 10px 0;"><span class="badge badge-status-completed" style="font-size:0.75rem; padding: 2px 6px;">4.50 / 5.0</span></td> |
| <td style="padding: 10px 0; font-family: var(--font-mono); font-size: 0.8rem;">1240 ms</td> |
| </tr> |
| <tr style="border-bottom: 1px solid rgba(194,125,56,0.08);"> |
| <td style="padding: 10px 0;">3</td> |
| <td style="padding: 10px 0;"><strong>Gemini 1.5 Pro</strong></td> |
| <td style="padding: 10px 0;"><span class="badge badge-status-completed" style="font-size:0.75rem; padding: 2px 6px;">4.20 / 5.0</span></td> |
| <td style="padding: 10px 0; font-family: var(--font-mono); font-size: 0.8rem;">980 ms</td> |
| </tr> |
| <tr> |
| <td style="padding: 10px 0;">4</td> |
| <td style="padding: 10px 0;"><strong>Gemini 1.5 Flash</strong></td> |
| <td style="padding: 10px 0;"><span class="badge badge-status-completed" style="font-size:0.75rem; padding: 2px 6px;">3.80 / 5.0</span></td> |
| <td style="padding: 10px 0; font-family: var(--font-mono); font-size: 0.8rem;">420 ms</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </div> |
| |
| <div class="card col-half" style="margin: 0; flex: 1; min-width: 320px; display: flex; flex-direction: column; justify-content: space-between;"> |
| <div> |
| <div class="card-header-grip" style="display: flex; justify-content: space-between; align-items: center; margin-bottom: 15px;"> |
| <h2 style="margin: 0;">Platform Diagnostics</h2> |
| <div class="grip-handle"><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span><span></span></div> |
| </div> |
| <div style="display: grid; grid-template-columns: repeat(2, 1fr); gap: 15px; font-size: 0.9rem;"> |
| <div style="display: flex; flex-direction: column;"> |
| <span style="color: var(--text-muted); font-size: 0.75rem; text-transform: uppercase; font-family: var(--font-mono);">Web Server</span> |
| <span style="color: var(--accent-emerald); font-weight: 600; margin-top: 4px;">● Online</span> |
| </div> |
| <div style="display: flex; flex-direction: column;"> |
| <span style="color: var(--text-muted); font-size: 0.75rem; text-transform: uppercase; font-family: var(--font-mono);">Celery Queue</span> |
| <span style="color: var(--accent-emerald); font-weight: 600; margin-top: 4px;">● Connected</span> |
| </div> |
| <div style="display: flex; flex-direction: column;"> |
| <span style="color: var(--text-muted); font-size: 0.75rem; text-transform: uppercase; font-family: var(--font-mono);">Database</span> |
| <span style="color: var(--accent-emerald); font-weight: 600; margin-top: 4px;">● Connected</span> |
| </div> |
| <div style="display: flex; flex-direction: column;"> |
| <span style="color: var(--text-muted); font-size: 0.75rem; text-transform: uppercase; font-family: var(--font-mono);">Worker Pools</span> |
| <span style="color: var(--accent-green); font-weight: 600; margin-top: 4px;">1 solo pool</span> |
| </div> |
| </div> |
| </div> |
| <div style="border-top: 1px solid var(--border-color); padding-top: 15px; margin-top: 20px;"> |
| <span style="color: var(--text-muted); font-size: 0.75rem; text-transform: uppercase; font-family: var(--font-mono); display: block; margin-bottom: 10px;">Quick Shortcuts</span> |
| <div style="display: flex; gap: 8px;"> |
| <a href="#hub" class="btn primary-btn top-btn" style="text-decoration: none; font-size: 0.75rem; padding: 6px 12px;">Trigger Eval</a> |
| <a href="#datasets" class="btn top-btn github-btn" style="text-decoration: none; font-size: 0.75rem; padding: 6px 12px;">Upload Cases</a> |
| </div> |
| </div> |
| </div> |
| </div> |
|
|
| <div class="card chart-card"> |
| <h2>LLM Cost-to-Quality Efficiency Frontier</h2> |
| <p class="card-description">Quality vs. Cost Scale (Bubble Size = Latency)</p> |
| <div id="frontier-chart" class="chart-container"></div> |
| </div> |
|
|
| <div class="card"> |
| <h2>Recent Evaluation Runs</h2> |
| <div class="table-container"> |
| <table> |
| <thead> |
| <tr> |
| <th>Run ID</th> |
| <th>Dataset ID</th> |
| <th>Prompt ID</th> |
| <th>Status</th> |
| <th>Accuracy</th> |
| <th>Hallucination Rate</th> |
| <th>Cost</th> |
| <th>Timestamp</th> |
| </tr> |
| </thead> |
| <tbody id="recent-runs-table-body"> |
| <tr> |
| <td colspan="8" class="text-center text-muted">Loading evaluation runs...</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </div> |
| </section> |
|
|
| |
| <section id="datasets" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>Dataset Management</h1> |
| <p class="tab-subtitle">Manage evaluation test suites, categories, and test cases.</p> |
| </header> |
|
|
| <div class="card expander-card"> |
| <div class="expander-header" onclick="toggleExpander('create-dataset-expander')"> |
| <span class="expander-title">Create New Dataset</span> |
| <span class="expander-icon">+</span> |
| </div> |
| <div id="create-dataset-expander" class="expander-content hidden"> |
| <form id="create-dataset-form" class="form-grid"> |
| <div class="form-group"> |
| <label for="ds-name">Dataset Name</label> |
| <input type="text" id="ds-name" required placeholder="e.g. Customer Support Core QA"> |
| </div> |
| <div class="form-group"> |
| <label for="ds-version">Version</label> |
| <input type="text" id="ds-version" value="1.0.0" required placeholder="e.g. 1.0.0"> |
| </div> |
| <div class="form-group"> |
| <label for="ds-category">Primary Evaluation Domain</label> |
| <select id="ds-category"> |
| <option value="factual">factual</option> |
| <option value="reasoning" selected>reasoning</option> |
| <option value="hallucination">hallucination</option> |
| <option value="tone">tone</option> |
| <option value="safety">safety</option> |
| <option value="adversarial">adversarial</option> |
| <option value="rag">rag</option> |
| </select> |
| </div> |
| <div class="form-group full-width"> |
| <label for="ds-description">Description</label> |
| <textarea id="ds-description" placeholder="Enter dataset goals or source notes..."></textarea> |
| </div> |
| <button type="submit" class="btn primary-btn">Create Dataset</button> |
| </form> |
| </div> |
| </div> |
|
|
| <div class="card selector-card"> |
| <label for="ds-selector">Select Dataset to Inspect / Manage</label> |
| <select id="ds-selector" onchange="loadSelectedDataset()"> |
| <option value="">Loading datasets...</option> |
| </select> |
| </div> |
|
|
| <div id="dataset-details-container" class="hidden"> |
| <div class="card bg-accent-pine"> |
| <h2 id="inspect-ds-title">Dataset Details</h2> |
| <div class="metadata-row"> |
| <span>Version: <strong id="inspect-ds-version">1.0.0</strong></span> |
| <span>Category: <strong id="inspect-ds-category">reasoning</strong></span> |
| </div> |
| <p id="inspect-ds-description" class="text-muted">No description provided.</p> |
| </div> |
|
|
| |
| <div class="inner-tabs-container"> |
| <div class="inner-tabs"> |
| <button class="inner-tab-btn active" onclick="switchInnerTab('cases-list', this)">Test Cases List</button> |
| <button class="inner-tab-btn" onclick="switchInnerTab('cases-upload', this)">Batch Upload JSON</button> |
| <button class="inner-tab-btn" onclick="switchInnerTab('cases-add-single', this)">Add Single Case</button> |
| </div> |
| |
| <div class="inner-tab-content"> |
| |
| <div id="inner-tab-cases-list" class="inner-tab-panel active"> |
| <div class="table-container"> |
| <table> |
| <thead> |
| <tr> |
| <th>ID</th> |
| <th>Category</th> |
| <th>Question</th> |
| <th>Expected Output (Ground Truth)</th> |
| <th>Actions</th> |
| </tr> |
| </thead> |
| <tbody id="testcases-table-body"> |
| <tr> |
| <td colspan="5" class="text-center text-muted">No test cases in this dataset.</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| <div class="card inline-form-card" style="margin-top: 24px;"> |
| <h3>Delete Test Case</h3> |
| <div class="flex-row"> |
| <select id="delete-tc-selector"> |
| <option value="">Select ID...</option> |
| </select> |
| <button class="btn danger-btn" onclick="deleteTestCase()">Delete Selected Case</button> |
| </div> |
| </div> |
| </div> |
|
|
| |
| <div id="inner-tab-cases-upload" class="inner-tab-panel"> |
| <h3>Upload JSON Test Cases Array</h3> |
| <pre class="code-block"><code>[ |
| { |
| "question": "What is diabetes?", |
| "ground_truth": "Diabetes is a chronic disease where the body cannot regulate glucose...", |
| "category": "factual", |
| "meta_data": {"difficulty": "easy"} |
| } |
| ]</code></pre> |
| <div class="form-group"> |
| <label for="upload-json-input">Paste JSON Array Content</label> |
| <textarea id="upload-json-input" rows="8" placeholder="Paste valid JSON list here..."></textarea> |
| </div> |
| <button class="btn primary-btn" onclick="uploadBatchTestCases()">Upload and Save Batch</button> |
| </div> |
|
|
| |
| <div id="inner-tab-cases-add-single" class="inner-tab-panel"> |
| <h3>Append Single Test Case</h3> |
| <form id="add-single-case-form" class="form-grid"> |
| <div class="form-group full-width"> |
| <label for="tc-question">Question / Input Prompt</label> |
| <textarea id="tc-question" required placeholder="e.g. Write a brief history of AI."></textarea> |
| </div> |
| <div class="form-group full-width"> |
| <label for="tc-ground-truth">Ground Truth / Expected Output</label> |
| <textarea id="tc-ground-truth" required placeholder="e.g. Artificial Intelligence started in the 1950s with Alan Turing..."></textarea> |
| </div> |
| <div class="form-group"> |
| <label for="tc-category">TestCase Category Tag</label> |
| <input type="text" id="tc-category" value="reasoning" required> |
| </div> |
| <div class="form-group"> |
| <label for="tc-metadata">Metadata (JSON dict)</label> |
| <input type="text" id="tc-metadata" value="{}"> |
| </div> |
| <button type="submit" class="btn primary-btn">Save Test Case</button> |
| </form> |
| </div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| |
| <section id="arena" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>Prompt Arena & Benchmark</h1> |
| <p class="tab-subtitle">Compare prompt templates and rank versions by accuracy, latency, and cost.</p> |
| </header> |
|
|
| <div class="card expander-card"> |
| <div class="expander-header" onclick="toggleExpander('create-prompt-expander')"> |
| <span class="expander-title">Register Prompt Template Version</span> |
| <span class="expander-icon">+</span> |
| </div> |
| <div id="create-prompt-expander" class="expander-content hidden"> |
| <form id="create-prompt-form" class="form-grid"> |
| <div class="form-group"> |
| <label for="pr-name">Template Name</label> |
| <input type="text" id="pr-name" required placeholder="e.g. Reasoning Scaffold Prompt"> |
| </div> |
| <div class="form-group"> |
| <label for="pr-version">Version Tag</label> |
| <input type="text" id="pr-version" required placeholder="e.g. V1.0"> |
| </div> |
| <div class="form-group full-width"> |
| <label for="pr-system">System Instruction Context</label> |
| <textarea id="pr-system" placeholder="e.g. You are a helpful assistant..."></textarea> |
| </div> |
| <div class="form-group full-width"> |
| <label for="pr-user">User Template Text</label> |
| <textarea id="pr-user" required placeholder="Answer the question: {{question}}"></textarea> |
| </div> |
| <div class="form-group full-width"> |
| <label for="pr-description">Release Notes</label> |
| <input type="text" id="pr-description" placeholder="What changed in this template?"> |
| </div> |
| <button type="submit" class="btn primary-btn">Register Template</button> |
| </form> |
| </div> |
| </div> |
|
|
| <div class="card"> |
| <h2>Prompt Arena Comparison Playground</h2> |
| <div class="comparison-selectors form-grid"> |
| <div class="form-group"> |
| <label for="arena-group-selector">Select Prompt Template Group</label> |
| <select id="arena-group-selector" onchange="loadArenaGroupVersions()"> |
| <option value="">Loading template groups...</option> |
| </select> |
| </div> |
| <div class="form-group"> |
| <label for="arena-base-selector">Select Baseline Version (A)</label> |
| <select id="arena-base-selector" onchange="renderArenaComparison()"> |
| <option value="">Select Baseline...</option> |
| </select> |
| </div> |
| <div class="form-group"> |
| <label for="arena-comp-selector">Select Comparison Version (B)</label> |
| <select id="arena-comp-selector" onchange="renderArenaComparison()"> |
| <option value="">Select Comparison...</option> |
| </select> |
| </div> |
| </div> |
|
|
| <div id="arena-comparison-sidebyside" class="flex-row hidden"> |
| <div class="comparison-box"> |
| <div class="box-header">Baseline Prompt (A)</div> |
| <div class="box-body"> |
| <strong>System Prompt:</strong> |
| <pre><code id="arena-base-system">None</code></pre> |
| <strong>User Template:</strong> |
| <pre><code id="arena-base-user">User Template</code></pre> |
| </div> |
| </div> |
| <div class="comparison-box"> |
| <div class="box-header">Comparison Prompt (B)</div> |
| <div class="box-body"> |
| <strong>System Prompt:</strong> |
| <pre><code id="arena-comp-system">None</code></pre> |
| <strong>User Template:</strong> |
| <pre><code id="arena-comp-user">User Template</code></pre> |
| </div> |
| </div> |
| </div> |
| </div> |
|
|
| <div id="arena-regression-panel" class="card hidden"> |
| <h2>Regression Check & Deltas</h2> |
| <div class="form-grid"> |
| <div class="form-group"> |
| <label for="arena-base-run-selector">Select Run for Baseline (A)</label> |
| <select id="arena-base-run-selector"> |
| <option value="">Select Baseline Run...</option> |
| </select> |
| </div> |
| <div class="form-group"> |
| <label for="arena-comp-run-selector">Select Run for Comparison (B)</label> |
| <select id="arena-comp-run-selector"> |
| <option value="">Select Comparison Run...</option> |
| </select> |
| </div> |
| </div> |
| <button class="btn primary-btn" onclick="triggerRegressionReport()">Trigger Regression Analysis Report</button> |
|
|
| |
| <div id="regression-report-container" class="hidden" style="margin-top: 30px;"> |
| <h3 class="success-heading">Regression Report Generated Successfully</h3> |
| |
| <div class="kpi-grid"> |
| <div class="kpi-card"> |
| <div class="kpi-label">Average Accuracy Delta</div> |
| <div id="reg-kpi-accuracy" class="kpi-value">+0.00</div> |
| <div id="reg-sub-accuracy" class="kpi-label" style="text-transform:none; margin-top:5px;">Baseline: 0.00</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Avg Latency Delta</div> |
| <div id="reg-kpi-latency" class="kpi-value">+0.0 ms</div> |
| <div class="kpi-label" style="text-transform:none; margin-top:5px;">Lower is better</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Cost Delta</div> |
| <div id="reg-kpi-cost" class="kpi-value">+$0.00000</div> |
| <div class="kpi-label" style="text-transform:none; margin-top:5px;">Lower is better</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Hallucination Rate Delta</div> |
| <div id="reg-kpi-hallucination" class="kpi-value">+0.00%</div> |
| <div class="kpi-label" style="text-transform:none; margin-top:5px;">Lower is better</div> |
| </div> |
| </div> |
|
|
| <div class="chart-card" style="margin-top: 30px;"> |
| <h3>Metric Dimensions Overview (Radar Plot)</h3> |
| <div id="arena-radar-chart" class="chart-container"></div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| |
| <section id="hub" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>Evaluation Hub & Pipeline</h1> |
| <p class="tab-subtitle">Schedule evaluation runs, monitor real-time execution, and inspect historical logs.</p> |
| </header> |
|
|
| <div class="inner-tabs-container"> |
| <div class="inner-tabs"> |
| <button class="inner-tab-btn active" onclick="switchInnerTab('hub-trigger', this)">Trigger New Evaluation Run</button> |
| <button class="inner-tab-btn" onclick="switchInnerTab('hub-history', this)">Historical Execution Logs</button> |
| </div> |
|
|
| <div class="inner-tab-content"> |
| |
| <div id="inner-tab-hub-trigger" class="inner-tab-panel active"> |
| <h3>Run Configurations</h3> |
| <form id="trigger-run-form" class="form-grid"> |
| <div class="form-group full-width"> |
| <label for="hub-ds-selector">Select Evaluation Dataset</label> |
| <select id="hub-ds-selector" required> |
| <option value="">Loading datasets...</option> |
| </select> |
| </div> |
| <div class="form-group full-width"> |
| <label for="hub-prompt-selector">Select Prompt Template (Optional)</label> |
| <select id="hub-prompt-selector"> |
| <option value="">None (Send raw questions directly)</option> |
| </select> |
| </div> |
| <div class="form-group full-width"> |
| <label>Select LLM Model Providers Suite</label> |
| <div class="checkbox-tags-group"> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="gpt-4o" checked> |
| <span class="checkbox-tag">gpt-4o</span> |
| </label> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="gpt-3.5-turbo"> |
| <span class="checkbox-tag">gpt-3.5-turbo</span> |
| </label> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="claude-3-5-sonnet" checked> |
| <span class="checkbox-tag">claude-3-5-sonnet</span> |
| </label> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="claude-3-haiku"> |
| <span class="checkbox-tag">claude-3-haiku</span> |
| </label> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="gemini-1.5-flash" checked> |
| <span class="checkbox-tag">gemini-1.5-flash</span> |
| </label> |
| <label class="checkbox-tag-label"> |
| <input type="checkbox" name="hub-models" value="gemini-1.5-pro"> |
| <span class="checkbox-tag">gemini-1.5-pro</span> |
| </label> |
| </div> |
| </div> |
| <button type="submit" class="btn primary-btn">Start Evaluation Suite</button> |
| </form> |
|
|
| |
| <div id="hub-run-status-container" class="hidden" style="margin-top: 30px;"> |
| <div class="card bg-accent-pine"> |
| <h3>Execution Status</h3> |
| <p>Current Pipeline Run Status: <strong id="hub-run-status">PENDING</strong></p> |
| <div class="progress-bar-container"> |
| <div id="hub-run-progress" class="progress-bar-fill" style="width: 0%;"></div> |
| </div> |
| <p id="hub-run-message" class="text-muted" style="margin-top: 10px;"></p> |
| </div> |
| </div> |
| </div> |
|
|
| |
| <div id="inner-tab-hub-history" class="inner-tab-panel"> |
| <div class="card selector-card"> |
| <label for="hub-history-run-selector">Select Evaluation Run to Inspect</label> |
| <select id="hub-history-run-selector" onchange="loadHistoryRunDetails()"> |
| <option value="">Loading execution runs...</option> |
| </select> |
| </div> |
|
|
| <div id="hub-history-details-container" class="hidden"> |
| <h3 id="inspect-run-heading">Run Metadata — Status: COMPLETED</h3> |
| <p id="inspect-run-timestamp" class="text-muted">Created At: N/A</p> |
|
|
| <div class="kpi-grid" style="margin-top: 20px;"> |
| <div class="kpi-card"> |
| <div class="kpi-label">Average Accuracy (1-5)</div> |
| <div id="inspect-kpi-accuracy" class="kpi-value">0.00</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Hallucination Rate (%)</div> |
| <div id="inspect-kpi-hallucination" class="kpi-value">0.00%</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Avg Latency (ms)</div> |
| <div id="inspect-kpi-latency" class="kpi-value">0 ms</div> |
| </div> |
| <div class="kpi-card"> |
| <div class="kpi-label">Total Cost (USD)</div> |
| <div id="inspect-kpi-cost" class="kpi-value">$0.0000</div> |
| </div> |
| </div> |
|
|
| <h3 style="margin-top: 40px;">Granular Test Results</h3> |
| <div class="table-container"> |
| <table> |
| <thead> |
| <tr> |
| <th>TestCase ID</th> |
| <th>Model</th> |
| <th>Question</th> |
| <th>Ground Truth</th> |
| <th>Model Output</th> |
| <th>Scores</th> |
| <th>Judge Explanation</th> |
| <th>Latency</th> |
| <th>Cost</th> |
| </tr> |
| </thead> |
| <tbody id="inspect-run-results-table-body"> |
| <tr> |
| <td colspan="9" class="text-center text-muted">No results found for this run.</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| |
| <section id="rca" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>AI Agent Console</h1> |
| <p class="tab-subtitle">Leverage intelligent agent reasoning to debug failures and generate adversarial test cases.</p> |
| </header> |
|
|
| <div class="inner-tabs-container"> |
| <div class="inner-tabs"> |
| <button class="inner-tab-btn active" onclick="switchInnerTab('rca-debugger', this)">Root Cause Analysis (RCA)</button> |
| <button class="inner-tab-btn" onclick="switchInnerTab('rca-redteam', this)">AI Red Team Auditor</button> |
| </div> |
|
|
| <div class="inner-tab-content"> |
| |
| <div id="inner-tab-rca-debugger" class="inner-tab-panel active"> |
| <h3>Failure Debugger & Optimizer Agent</h3> |
| <p>When prompts change or database schema updates occur, evaluation scores can degrade. Select a run to analyze failures.</p> |
| |
| <div class="card selector-card" style="margin-top: 20px;"> |
| <label for="rca-run-selector">Select Run to Analyze</label> |
| <select id="rca-run-selector"> |
| <option value="">Loading completed runs...</option> |
| </select> |
| </div> |
| <button class="btn primary-btn" onclick="triggerRcaReport()">Generate Root Cause Report</button> |
|
|
| <div id="rca-report-result" class="card bg-accent-pine hidden" style="margin-top: 30px;"> |
| <h3>Root Cause Analysis Report</h3> |
| <div id="rca-report-markdown" class="markdown-body"> |
| Report output... |
| </div> |
| </div> |
| </div> |
|
|
| |
| <div id="inner-tab-rca-redteam" class="inner-tab-panel"> |
| <h3>Adversarial Dataset Generator</h3> |
| <p>Generate ambiguous variations, info-traps, and injection payloads to stresstest model robustness.</p> |
| |
| <div class="card selector-card" style="margin-top: 20px;"> |
| <label for="rca-seed-selector">Select Seed Dataset</label> |
| <select id="rca-seed-selector"> |
| <option value="">Loading seed datasets...</option> |
| </select> |
| </div> |
| <button class="btn primary-btn" onclick="triggerRedTeamSuite()">Generate Adversarial Attacks</button> |
|
|
| <div id="redteam-results-container" class="hidden" style="margin-top: 30px;"> |
| <div class="card bg-accent-pine" style="margin-bottom: 20px;"> |
| <h3 class="success-heading">Adversarial Suite Created Successfully</h3> |
| <p>Saved as new Dataset: <strong id="redteam-dataset-details">Name (ID)</strong></p> |
| </div> |
|
|
| <h3>Generated Adversarial Test Cases</h3> |
| <div class="table-container"> |
| <table> |
| <thead> |
| <tr> |
| <th>Attack Type</th> |
| <th>Question</th> |
| <th>Expected Output (Ground Truth)</th> |
| <th>Original Question Reference</th> |
| </tr> |
| </thead> |
| <tbody id="redteam-table-body"> |
| <tr> |
| <td colspan="4" class="text-center text-muted">No adversarial cases generated.</td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| |
| <section id="cost" class="scroll-section"> |
| <header class="tab-header"> |
| <h1>Cost Optimizer & Analytics</h1> |
| <p class="tab-subtitle">Analyze model spending, efficiency ratios, and get LLM routing recommendations.</p> |
| </header> |
|
|
| <div class="flex-row"> |
| <div class="card col-half"> |
| <h2>Cost per Query by Model</h2> |
| <div id="cost-bar-chart" class="chart-container"></div> |
| </div> |
| <div class="card col-half"> |
| <h2>Accuracy-to-Cost Efficiency Ratio</h2> |
| <p class="card-description">Calculated as (Accuracy Score / Cost in Millidollars). Higher means more value per dollar.</p> |
| <div id="value-bar-chart" class="chart-container"></div> |
| </div> |
| </div> |
|
|
| <div class="card" style="margin-top: 30px;"> |
| <h2>Cost Optimizer Recommendations</h2> |
| <div id="cost-recommendations-list"> |
| <div class="info-alert">Loading recommendation metrics...</div> |
| </div> |
| </div> |
| </main> |
| <footer class="app-footer"> |
| <div class="footer-content"> |
| <span>LLMBench © 2026. All rights reserved.</span> |
| <span>Designed & Built by Jyotiraditya</span> |
| </div> |
| </footer> |
| </div> |
|
|
| |
| <div id="guide-modal" class="modal-overlay hidden" onclick="closeProductGuideOnOverlay(event)"> |
| <div class="modal-card"> |
| <div class="modal-header"> |
| <h2>Product Guide & How to Use</h2> |
| <button class="close-btn" onclick="closeProductGuide()">×</button> |
| </div> |
| <div class="modal-body"> |
| <h3>Welcome to LLMBench</h3> |
| <p>LLMBench is a premium engineering dashboard for evaluating Large Language Models. Follow these simple steps to benchmark and audit prompts:</p> |
| |
| <ol> |
| <li> |
| <strong>Manage Datasets:</strong> |
| Go to the <em>Datasets</em> tab to create evaluation folders, batch import test cases, or inspect stored ground truths. |
| </li> |
| <li> |
| <strong>Trigger Evaluations:</strong> |
| In the <em>Evaluation Hub</em>, select your target dataset and prompt version, select the model suites (e.g. GPT-4, Claude, Gemini), and start the execution queue. The pipeline runs asynchronously in Celery and will poll live metrics. |
| </li> |
| <li> |
| <strong>Prompt Arena Benchmarking:</strong> |
| Navigate to the <em>Prompt Arena</em>, select your Prompt Template Group, and load your baseline (A) and comparison (B) prompt runs to view full, multi-dimensional accuracy, latency, and cost regression deltas. |
| </li> |
| <li> |
| <strong>Failure Diagnostics:</strong> |
| Use the <em>Agent RCA</em> tab to aggregate failed evaluation outputs, allowing LLM reasoning agents to pinpoint failure modes and suggest prompt scaffolds. |
| </li> |
| </ol> |
| </div> |
| <div class="modal-footer"> |
| <button class="btn primary-btn" onclick="closeProductGuide()">Get Started</button> |
| </div> |
| </div> |
| </div> |
|
|
| |
| <script src="app.js?v=1.0.1"></script> |
| </body> |
| </html> |
|
|
