Spaces:
Running
Complementary tool: automated 5-metric LLM evaluation beyond ELO scoring
Hi AtlaAI team 👋
Judge Arena's ELO-based ranking is great for preference-based evaluation. For teams who need quantitative, reproducible benchmarking alongside preference scores, I built a complementary tool.
LLM Evaluation Framework gives you numbers, not just rankings:
→ 🎯 Accuracy % — verifiable, reproducible across runs
→ ⚡ Latency ms — p50/p95/p99, so you can set SLA thresholds
→ 💰 Cost $ — exact token-count pricing per model
→ 🔍 Hallucination Rate — 0.0-1.0 score, runs locally
→ 🧠 Reasoning Quality — 1-10 chain-of-thought depth
ELO tells you which model humans prefer. This tells you which model wins on your specific metrics at your specific budget.
Live demo: https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework
Open source, 71 tests, free forever. Happy to discuss how quantitative + preference evals complement each other!