Leaderboard
Every evaluated model, every benchmark, one table.
Sort
No models evaluated on this generation yet.
The 7-2026 tiers are brand new — run
python script.py --model <your-model> and open a PR with the
leaderboard.json it prints. Or view the
.
Text-to-image
FID on MS-COCO — lower is betterNo text-to-image models evaluated yet.
Entry schema & how to submit
Run the universal script — it fetches the live leaderboard and merges your run into it, writing an upload-ready models.json next to the results:
python script.py --model your-org/your-model
# then upload benchlabs_results/<model>/models.json to this Space (PR welcome).
# Same model id: only the tiers you just ran are replaced; everything else kept.
# 7-2026 runs report both modes:
# metrics.generative exact_match · hybrid_score (alias-aware, per-item routing)
# metrics.loglikelihood acc · acc_norm · soft_score · soft_score_norm
# model_revision + script_sha256 pin exactly what was evaluated.
#
# Text-to-image entries (FID / MS-COCO) live in a separate top-level
# "t2i_models" array in models.json — see add_t2i_schema.py for the entry shape.