README.md

---
title: GRM Leaderboard
colorFrom: gray
colorTo: blue
sdk: gradio
sdk_version: "5.23.0"
app_file: app.py
pinned: false
---

# GRM Leaderboard

Static Gradio Space for comparing language models on a game-focused evaluation suite.

## What This Project Is

- Single-repo Hugging Face Space
- Frontend-only app with no database or external backend
- Static benchmark registry and static model score data stored in Python files
- Runtime leaderboard calculation from those local data files

## Runtime

- Platform: Hugging Face Spaces
- UI framework: Gradio
- Entry point: app.py
- Dependencies: requirements.txt
- Space metadata: this README frontmatter
- app.py includes compatibility handling for both Gradio 5 and Gradio 6

## Main Page Flow

Tab 1:
- Overview
- Leaderboard
- Per-Benchmark Score Breakdown
- Evaluation Suite
- Benchmark Details

Tab 2:
- GRM-Bench authored benchmark families

## File Ownership

- app.py: page layout, tabs, CSS, overview copy, table rendering, and GRM-Bench authored sections
- benchmarks.py: benchmark registry, category assignments, descriptions, summaries, and weights
- scores.py: per-model benchmark scores on a 0.0 to 1.0 scale
- scoring.py: category scoring, GRM score calculation, and ranking logic
- requirements.txt: runtime dependencies
- README.md: Space metadata and maintainer handoff notes

## Data Model

- benchmarks.py stores BENCHMARKS as a list of dicts
- Each benchmark entry includes: name, category, calc_weight, description, summary, paper
- Valid categories are ROLEPLAY, ACTIONS, and GENERAL
- scores.py stores MODEL_SCORES keyed by model display name
- Each model score dict is keyed by benchmark name
- Missing scores are skipped during weighted averaging

## Scoring

- Category score = sum(score x weight) / sum(weight)
- GRM score = average of Roleplay, Actions, and General category scores
- scores.py values stay on a 0.0 to 1.0 scale
- Displayed leaderboard values are converted to 0 to 100

## How To Update The Site

Update model scores:
- Edit scores.py
- Change benchmark values for an existing model or add a new model block

Update evaluation suite rows or benchmark descriptions:
- Edit benchmarks.py
- The evaluation table and benchmark detail sections are generated from this registry

Add a new benchmark:
- Add the benchmark entry to benchmarks.py
- Set its category and calc_weight
- Add corresponding values in scores.py for each model you want included

Update the authored GRM-Bench tab:
- Edit GRM_BENCH_SECTIONS in app.py

Update page structure, copy, or styling:
- Edit app.py

## Local Development

- Install dependencies: pip install -r requirements.txt
- Run locally: python app.py
- The app launches a local Gradio server using the same static content as the Space

## Deployment Notes

- The live Space deploys from the remote main branch
- README frontmatter controls the Space runtime metadata
- requirements.txt must match imports used by app.py
- Current scores in scores.py are placeholder/static values unless replaced with real outputs

## Maintenance Notes

- The UI uses Python-generated HTML tables, not Gradio Dataframes
- Leaderboard order is recalculated on each launch from scores.py
- Gradio theme and CSS are injected conditionally based on the installed Gradio major version
- If page scrolling behaves oddly, inspect the root overflow and flex overrides in app.py