README.md

metadata

title: GRM Leaderboard
colorFrom: gray
colorTo: blue
sdk: gradio
sdk_version: 5.23.0
app_file: app.py
pinned: false

GRM Leaderboard

Static Gradio Space for comparing language models on a game-focused evaluation suite.

What This Project Is

• Single-repo Hugging Face Space
• Frontend-only app with no database or external backend
• Static benchmark registry and static model score data stored in Python files
• Runtime leaderboard calculation from those local data files

Runtime

• Platform: Hugging Face Spaces
• UI framework: Gradio
• Entry point: app.py
• Dependencies: requirements.txt
• Space metadata: this README frontmatter
• app.py includes compatibility handling for both Gradio 5 and Gradio 6

Main Page Flow

Tab 1:

• Overview
• Leaderboard
• Per-Benchmark Score Breakdown
• Evaluation Suite
• Benchmark Details

Tab 2:

• GRM-Bench authored benchmark families

File Ownership

• app.py: page layout, tabs, CSS, overview copy, table rendering, and GRM-Bench authored sections
• benchmarks.py: benchmark registry, category assignments, descriptions, summaries, and weights
• scores.py: per-model benchmark scores on a 0.0 to 1.0 scale
• scoring.py: category scoring, GRM score calculation, and ranking logic
• requirements.txt: runtime dependencies
• README.md: Space metadata and maintainer handoff notes

Data Model

• benchmarks.py stores BENCHMARKS as a list of dicts
• Each benchmark entry includes: name, category, calc_weight, description, summary, paper
• Valid categories are ROLEPLAY, ACTIONS, and GENERAL
• scores.py stores MODEL_SCORES keyed by model display name
• Each model score dict is keyed by benchmark name
• Missing scores are skipped during weighted averaging

Scoring

• Category score = sum(score x weight) / sum(weight)
• GRM score = average of Roleplay, Actions, and General category scores
• scores.py values stay on a 0.0 to 1.0 scale
• Displayed leaderboard values are converted to 0 to 100

How To Update The Site

Update model scores:

• Edit scores.py
• Change benchmark values for an existing model or add a new model block

Update evaluation suite rows or benchmark descriptions:

• Edit benchmarks.py
• The evaluation table and benchmark detail sections are generated from this registry

Add a new benchmark:

• Add the benchmark entry to benchmarks.py
• Set its category and calc_weight
• Add corresponding values in scores.py for each model you want included

Update the authored GRM-Bench tab:

• Edit GRM_BENCH_SECTIONS in app.py

Update page structure, copy, or styling:

• Edit app.py

Local Development

• Install dependencies: pip install -r requirements.txt
• Run locally: python app.py
• The app launches a local Gradio server using the same static content as the Space

Deployment Notes

• The live Space deploys from the remote main branch
• README frontmatter controls the Space runtime metadata
• requirements.txt must match imports used by app.py
• Current scores in scores.py are placeholder/static values unless replaced with real outputs

Maintenance Notes

• The UI uses Python-generated HTML tables, not Gradio Dataframes
• Leaderboard order is recalculated on each launch from scores.py
• Gradio theme and CSS are injected conditionally based on the installed Gradio major version
• If page scrolling behaves oddly, inspect the root overflow and flex overrides in app.py