Spaces:
Sleeping
LLM evaluation framework to benchmark agent backbone models systematically
Hi TraceMind team 👋
MCP-powered agent evaluation is exactly where AI is heading. For teams building agents, selecting the right backbone LLM requires systematic evaluation beyond vibes.
I built an open source LLM Evaluation Framework that gives you hard numbers:
→ 🧠 Reasoning Quality — 1-10 scoring on chain-of-thought depth, directly relevant to agent quality
→ 🔍 Hallucination Rate — hallucinated function calls and parameters are a critical agent failure mode
→ 💰 Cost per 1K tokens — agent tasks run many LLM calls, cost scales fast
→ ⚡ Latency p95 — real-time agents need tight latency budgets
→ 🎯 Accuracy — task-level accuracy
Live demo (no API key): https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework
Open source, free forever. Happy to discuss how evaluation fits into agent development workflows!