updates to index.html

CLAUDE.md

@@ -2,138 +2,43 @@
 
 ## Project Overview
 - FLaME: Holistic Financial Language Model Evaluation
-- Static website built with Bulma CSS framework
-- No build system (pure HTML/CSS/JavaScript)
-- Research paper for ACL Annual Advances in Research (Feb 2025)
+- LaTeX paper with static website built using Bulma CSS
+- Research project for ACL Annual Advances in Research (Feb 2025)
 - Hosted on HuggingFace Spaces
 
-## Serving the Website
-- For local testing: `python -m http.server 8000` (will serve from current directory)
-- Open browser to http://localhost:8000
+## Commands
+- Build LaTeX paper: `pdflatex FLaME.tex && bibtex FLaME && pdflatex FLaME.tex && pdflatex FLaME.tex`
+- Local website testing: `python -m http.server 8000`
+- Fix tooltips: `./fix_tooltips.sh`
 
 ## Code Style Guidelines
-- HTML: Follow semantic HTML5 practices
-- CSS: Follow Bulma framework conventions
+- HTML: Semantic HTML5, Bulma framework conventions
+- CSS: Follow Bulma conventions, keep in static/css/
 - JavaScript:
-  - Use camelCase for variables and functions
-  - Use 2-space indentation
-  - Include semicolons after statements
-  - Use function declarations with 'function' keyword
-  - Prefer vanilla JS with minimal dependencies
-
-## Website Design Guidelines
-- Color scheme: 
-  - Primary: Deep blue (#004d99) for finance theme
-  - Secondary: Orange (#ff6b00) for "flame" accent
-  - Light background (#f8f9fa) for content areas
-- Card-based layout for content organization
-- Interactive elements with hover effects
-- Mobile-responsive design
-- Navigation menu with fixed positioning
-- Footer with institutional information and resource links
-
-## Media Guidelines
-- Image formats: Prefer .jpg for photos, .svg for vector graphics
-- Video formats: Use .mp4 for compatibility
-- Optimize media files for web delivery
-- Paper figures are in PDF format in FLaME/content/figures/
-- For web display, convert PDF figures to PNG/JPG formats
-
-## Structure
-- Keep all CSS in static/css/
-- Keep all JavaScript in static/js/
-- Keep media files in appropriate subdirectories
-- Paper content in FLaME/content/
-- Use section IDs for navigation linking (e.g., #abstract, #methodology)
-
-## Interactive Components
-- Navbar with smooth scrolling to sections
-- Performance indicator bars for result visualization
-- Card layouts for key findings with hover effects
-- Methodology workflow diagram with step visualization
-- Interactive feature highlights with icons
-- Getting started guide with numbered steps
-
-## Responsive Design
-- Mobile-friendly navigation menu (hamburger on small screens)
-- Stacked cards on mobile devices
-- Adjusted typography and spacing for different screen sizes
-- Media queries for breakpoints at 768px
-
-## FLaME Research Paper Information
-
-### Authors
-- Oopy Goopy, General Munchkin Man, L'il Jim Bob, Larry
-- Affiliation: Georgia Institute of Technology
-
-### Paper Focus and Objective
-- First comprehensive benchmarking framework for evaluating language models on financial NLP tasks
-- Addresses gaps in existing evaluation methodologies for financial language models
-- Provides standardized evaluation framework with open-source implementation
-
-### Key Components
-
-#### Taxonomy
-- Organized by three dimensions: tasks, domains, and languages
-- Six core FinNLP tasks:
-  1. Text classification
-  2. Sentiment analysis 
-  3. Information retrieval
-  4. Causal analysis
-  5. Text summarization
-  6. Question answering
-- Domains categorized by data source, origination, time period, etc.
-- Currently focuses on English language
-
-#### Datasets
-Selected based on:
-- Financial domain relevance
-- Fair usage licensing
-- Annotation quality
-- Task substance
-
-Key datasets include:
-- Banking: Banking77, FiQA, FinRED
-- Investment: FPB, Headlines, SubjectiveQA
-- Accounting: FinQA, TaT-QA, ConvFinQA
-- Corporate: ECTSum, EDTSum, FinCausal
-- Monetary Policy: FOMC, FNXL
-- Cross-domain: FinBench, NumClaim, ReFINED
-
-#### Models Evaluated
-- Proprietary closed-source: GPT-4o & o1-mini, Gemini-1.5, Claude3, Cohere Command R
-- Open-weight: Llama-3, DeepSeekV3 & R-1, Qwen-2 & QwQ, Mistral, Gemma-1 & 2, Mixtral, WizardLM2, DBRX
-- Used deterministic decoding (temperature 0.0, top p of 0.9, repetition penalty of 1)
-
-#### Evaluation Process
-- Two-stage approach: generation and extraction
-- Task-specific metrics: accuracy, F1 scores, precision, recall, BLEU scores
-- Standardized zero-shot evaluation
-
-### Key Findings
-- No single model performs best across all tasks
-- Performance varies significantly based on domain and task structure
-- Open-weight models show strong cost/performance efficiency
-- Numeric reasoning tasks remain challenging for all models
-- Inconsistent scaling: larger parameter sizes don't guarantee higher performance
-- Models struggle with consistent numeric formats and longer label sets
-- Top performers: DeepSeek R1, OpenAI o1-mini, Claude 3.5 Sonnet
-
-### Limitations
-- Limited dataset size and diversity
-- Focus on zero-shot scenarios only
-- English-language focus
-- No evaluation of advanced prompting techniques
-- Doesn't capture full breadth of real-world financial scenarios
-
-### Future Directions
-- More advanced prompt engineering
-- Domain-adaptive training for numeric/causal tasks
-- Benchmarking efficiency trade-offs
-- Multi-lingual coverage expansion
-
-### Resources
-- Paper PDF: FLaME/FLaME__ACL_AAR_Feb_2025_.pdf
-- ArXiv: https://arxiv.org/abs/2402.14017
+  - Use camelCase for variables/functions
+  - 2-space indentation
+  - Include semicolons
+  - Prefer vanilla JS
+  - Store in static/js/
+- LaTeX: Follow ACL style guidelines in acl_formatting.md
+
+## Paper Structure
+- Main source in FLaME.tex
+- Content modularized in FLaME/content/ directory
+- Six core task areas: text classification, sentiment analysis, info retrieval, causal analysis, summarization, QA
+- Datasets in FLaME/content/datasets/
+- Results in FLaME/content/tables/
+
+## Website Design
+- Color scheme: Deep blue (#004d99), Orange (#ff6b00), Light bg (#f8f9fa)
+- Card-based responsive layout
+- Interactive elements with tooltips
+- Results displayed in task-specific tables
+- Media files optimized for web delivery
+- Convert PDF figures to JPG/PNG for web display
+
+## Deployment
+- Website deploys to HuggingFace Spaces
+- Paper available as PDF at FLaME/FLaME.pdf
 - GitHub: https://github.com/flame-benchmark/flame
 - HuggingFace: https://huggingface.co/spaces/flame-benchmark/flame

FLaME/FLaME.pdf:Zone.Identifier

@@ -0,0 +1,4 @@
+[ZoneTransfer]
+ZoneId=3
+ReferrerUrl=https://www.overleaf.com/project/67380d90965fe3bdf7157e6c
+HostUrl=https://www.overleaf.com/download/project/67380d90965fe3bdf7157e6c/build/195acbb99c5-1bdebd47a7495259/output/output.pdf?compileGroup=priority&clsiserverid=clsi-pre-emp-c2d-c-f-bzgb&enable_pdf_caching=true&popupDownload=true

FLaME/FLaME[ACLAARFeb2025]/content/0_authors.tex

@@ -1 +1 @@
-\author{Oopy Goopy, {\bf General Munchkin Man}, {\bf L'il Jim Bob}, {\bf Larry} \\ Georgia Institute of Technology}
+\author{Glenn Matlin, {\bf Mika Okamoto}, {\bf Huzaifa Pardwala}, {\bf Yang Yang}, {\bf Sudheer Chava} \\ Georgia Institute of Technology}

FLaME/content/0_authors.tex

@@ -1 +1 @@
-\author{Oopy Goopy, {\bf General Munchkin Man}, {\bf L'il Jim Bob}, {\bf Larry} \\ Georgia Institute of Technology}
+\author{Glenn Matlin, {\bf Mika Okamoto}, {\bf Huzaifa Pardwala}, {\bf Yang Yang}, {\bf Sudheer Chava} \\ Georgia Institute of Technology}

index.html

@@ -93,14 +93,15 @@
           <h1 class="title is-1 publication-title">FLaME: Holistic Financial Language Model Evaluation</h1>
           <div class="is-size-5 publication-authors">
             <span class="author-block">
-              <a href="#" target="_blank">Oopy Goopy</a><sup>1</sup>,</span>
+              <a href="#" target="_blank">Glenn Matlin</a><sup>1</sup>,</span>
             <span class="author-block">
-              <a href="#" target="_blank">General Munchkin Man</a><sup>1</sup>,</span>
+              <a href="#" target="_blank">Mika Okamoto</a><sup>1</sup>,</span>
             <span class="author-block">
-              <a href="#" target="_blank">L'il Jim Bob</a><sup>1</sup>,
-            </span>
+              <a href="#" target="_blank">Huzaifa Pardwala</a><sup>1</sup>,</span>
+            <span class="author-block">
+              <a href="#" target="_blank">Yang Yang</a><sup>1</sup>,</span>
             <span class="author-block">
-              <a href="#" target="_blank">Larry</a><sup>1</sup>
+              <a href="#" target="_blank">Sudheer Chava</a><sup>1</sup>
             </span>
           </div>
 
@@ -112,7 +113,7 @@
             <div class="publication-links">
               <!-- PDF Link. -->
               <span class="link-block">
-                <a href="FLaME/FLaME__ACL_AAR_Feb_2025_.pdf" target="_blank"
+                <a href="FLaME/FLaME.pdf" target="_blank"
                    class="external-link button is-normal is-rounded is-dark">
                   <span class="icon">
                       <i class="fas fa-file-pdf"></i>
@@ -3911,15 +3912,15 @@
     
           <hr>
     
-          <p class="has-text-weight-bold mb-3">🔍 Key Insights from Model Analysis</p>
+          <p class="has-text-weight-bold mb-3"><span class="icon has-text-primary"><i class="fa-solid fa-magnifying-glass"></i></span> Key Insights from Model Analysis</p>
     
           <div class="notification is-info is-light py-3 px-4">
-            <p><strong>🏆 No single dominant model:</strong> DeepSeek R1 leads in complex multi-step QA, while Claude 3.5 excels in sentiment tasks. GPT-4o is strong in classification and summarization.</p>
-            <p><strong>⚖️ Inconsistent scaling:</strong> Larger models don’t always outperform smaller ones—DeepSeek R1 trails in summarization despite excelling in QA.</p>
-            <p><strong>🛠️ Open-weight models:</strong> Many open-weight models like DeepSeek-V3 and Llama 3.1 70B offer competitive performance while being cost-effective.</p>
-            <p><strong>💰 Cost-performance disparities:</strong> Running DeepSeek R1 can cost up to <strong>$260</strong> per million tokens, while Claude 3.5 Sonnet and o1-mini cost around <strong>$105</strong>, and Meta’s Llama 3.1 8B only <strong>$4</strong>.</p>
-            <p><strong>📉 Numeric reasoning challenges:</strong> Even the best models struggle with financial numeric reasoning tasks, achieving low F1 scores (<strong>≤ 0.06</strong>).</p>
-            <p><strong>🔢 Step-by-step deductions:</strong> Multi-turn financial QA (e.g., ConvFinQA) significantly reduces model accuracy due to complex dependencies.</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-trophy"></i></span> No single dominant model:</strong> DeepSeek R1 leads in complex multi-step QA, while Claude 3.5 excels in sentiment tasks. GPT-4o is strong in classification and summarization.</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-balance-scale"></i></span> Inconsistent scaling:</strong> Larger models don’t always outperform smaller ones—DeepSeek R1 trails in summarization despite excelling in QA.</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-tools"></i></span> Open-weight models:</strong> Many open-weight models like DeepSeek-V3 and Llama 3.1 70B offer competitive performance while being cost-effective.</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-coins"></i></span> Cost-performance disparities:</strong> Running DeepSeek R1 can cost up to <strong>$260</strong> per million tokens, while Claude 3.5 Sonnet and o1-mini cost around <strong>$105</strong>, and Meta’s Llama 3.1 8B only <strong>$4</strong>.</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-chart-line"></i></span> Numeric reasoning challenges:</strong> Even the best models struggle with financial numeric reasoning tasks, achieving low F1 scores (<strong>≤ 0.06</strong>).</p>
+            <p><strong><span class="icon"><i class="fa-solid fa-list-ol"></i></span> Step-by-step deductions:</strong> Multi-turn financial QA (e.g., ConvFinQA) significantly reduces model accuracy due to complex dependencies.</p>
           </div>
         </div>
       </div>
@@ -4200,27 +4201,27 @@
             </div>
 
             <div class="notification is-info is-light py-2 px-3 mb-3">
-              <p class="has-text-weight-bold mb-1">🧠 Few-Shot & Chain-of-Thought</p>
+              <p class="has-text-weight-bold mb-1"><span class="icon has-text-primary"><i class="fa-solid fa-brain"></i></span> Few-Shot & Chain-of-Thought</p>
               <p class="is-size-7 mb-0">Investigating in-context learning techniques such as few-shot, chain-of-thought, and retrieval-augmented generation (RAG).</p>
             </div>
 
             <div class="notification is-info is-light py-2 px-3 mb-3">
-              <p class="has-text-weight-bold mb-1">📊 Domain-Adaptive Training</p>
+              <p class="has-text-weight-bold mb-1"><span class="icon has-text-primary"><i class="fa-solid fa-chart-line"></i></span> Domain-Adaptive Training</p>
               <p class="is-size-7 mb-0">Evaluating fine-tuning strategies to enhance model understanding of financial-specific terminology and reasoning.</p>
             </div>
 
             <div class="notification is-info is-light py-2 px-3 mb-3">
-              <p class="has-text-weight-bold mb-1">🔍 Expanded Dataset Coverage</p>
+              <p class="has-text-weight-bold mb-1"><span class="icon has-text-primary"><i class="fa-solid fa-database"></i></span> Expanded Dataset Coverage</p>
               <p class="is-size-7 mb-0">Curating datasets from underrepresented financial sectors such as insurance, derivatives, and central banking.</p>
             </div>
 
             <div class="notification is-info is-light py-2 px-3 mb-3">
-              <p class="has-text-weight-bold mb-1">⚖️ Efficiency & Cost Benchmarking</p>
+              <p class="has-text-weight-bold mb-1"><span class="icon has-text-primary"><i class="fa-solid fa-balance-scale"></i></span> Efficiency & Cost Benchmarking</p>
               <p class="is-size-7 mb-0">Developing detailed trade-off analyses between accuracy, latency, and cost to optimize real-world usability.</p>
             </div>
 
             <div class="notification is-info is-light py-2 px-3 mb-3">
-              <p class="has-text-weight-bold mb-1">📈 Advanced Evaluation Metrics</p>
+              <p class="has-text-weight-bold mb-1"><span class="icon has-text-primary"><i class="fa-solid fa-chart-bar"></i></span> Advanced Evaluation Metrics</p>
               <p class="is-size-7 mb-0">Moving beyond traditional accuracy metrics by incorporating trustworthiness, robustness, and interpretability measures.</p>
             </div>
 
@@ -4299,7 +4300,7 @@
 
                 <div class="feature-item mb-3">
                   <p class="has-text-weight-bold mb-1">
-                    <span class="icon has-text-primary"><i class="fas fa-check"></i></span> 📊 Reproducible Benchmarking
+                    <span class="icon has-text-primary"><i class="fas fa-check"></i></span> Reproducible Benchmarking
                   </p>
                   <p class="is-size-7 ml-4">Ensures consistent evaluation metrics and transparent methodology.</p>
                 </div>
@@ -4391,7 +4392,7 @@
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon has-text-primary"><i class="fa-solid fa-calculator"></i></span> 📊 Numerical Reasoning & Question Answering
+            📊 Numerical Reasoning & Question Answering
           </p>
           <ul>
             <li><strong>FinQA</strong> – Multi-step financial numerical reasoning.</li>
@@ -4405,7 +4406,7 @@
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon has-text-primary"><i class="fa-solid fa-file-lines"></i></span> 📝 Text Summarization
+            📝 Text Summarization
           </p>
           <ul>
             <li><strong>ECTSum</strong> – Earnings call transcript summarization.</li>
@@ -4418,7 +4419,7 @@
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon has-text-primary"><i class="fa-solid fa-search"></i></span> 🔎 Information Retrieval
+            🔎 Information Retrieval
           </p>
           <ul>
             <li><strong>FiNER-ORD</strong> – Named entity recognition for financial documents.</li>
@@ -4434,7 +4435,7 @@
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon has-text-primary"><i class="fa-solid fa-comment-alt"></i></span> 😐 Sentiment Analysis
+            😐 Sentiment Analysis
           </p>
           <ul>
             <li><strong>FiQA (Task 1)</strong> – Aspect-based sentiment analysis.</li>
@@ -4449,7 +4450,7 @@
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon has-text-primary"><i class="fa-solid fa-tags"></i></span> 🏷️ Text Classification
+            🏷️ Text Classification
           </p>
           <ul>
             <li><strong>Numerical Claim Detection</strong> – Fine-grained investor claim detection.</li>
@@ -4461,11 +4462,11 @@
         </div>
       </div>
 
-      <!--  Causal Analysis -->
+      <!-- 🧠 Causal Analysis -->
       <div class="column is-6">
         <div class="dataset-category box">
           <p class="has-text-weight-bold">
-            <span class="icon"><i class="fa-solid fa-brain"></i></span> 🧠 Causal Analysis
+            🧠 Causal Analysis
           </p>
           <ul>
             <li><strong>FinCausal</strong> – Causal reasoning in financial news.</li>
@@ -4493,7 +4494,6 @@
         <pre><code>@article{flame2025,
   author    = {Goopy, Oopy and Man, General Munchkin and Bob, L'il Jim and Larry},
   title     = {FLaME: Holistic Financial Language Model Evaluation},
-  journal   = {ACL Annual Advances in Research},
   year      = {2025},
   month     = {February},
 }</code></pre>
@@ -4510,7 +4510,7 @@
           <h4 class="has-text-white mb-4"><span class="flame">FLaME</span>: Financial Language Model Evaluation</h4>
           
           <div class="footer-links mb-5">
-            <a class="icon-link mr-3" target="_blank" href="FLaME/FLaME__ACL_AAR_Feb_2025_.pdf" title="Download PDF">
+            <a class="icon-link mr-3" target="_blank" href="FLaME/FLaME.pdf" title="Download PDF">
               <i class="fas fa-file-pdf fa-lg"></i>
             </a>
             <a class="icon-link mr-3" href="https://arxiv.org/abs/2402.14017" target="_blank" title="View on arXiv">
@@ -4525,7 +4525,7 @@
           </div>
           
           <div class="institution-info mb-4">
-            <p class="has-text-white-ter">Georgia Institute of Technology | ACL 2025</p>
+            <p class="has-text-white-ter">Georgia Institute of Technology</p>
           </div>
           
           <p class="has-text-white-ter is-size-7">