Spaces:

---

FinanceMTEB

/

FinMTEB

like 10

Sleeping

Hi FinanceMTEB team 👋

Financial text evaluation is extremely high-stakes — models deployed in finance must be accurate and must NOT hallucinate. Your benchmark is doing critical work here.

I built an open source LLM Evaluation Framework with particular relevance for finance:

→ 🔍 Hallucination Rate — in finance, a confident wrong answer is a compliance failure
→ 💰 Cost per 1K tokens — high-volume financial data processing makes cost critical
→ ⚡ Latency p95 — real-time trading and compliance workflows have strict SLAs
→ 🎯 Accuracy — 4-strategy cascade scorer
→ 🧠 Reasoning Quality — step-by-step financial reasoning scoring

Live demo: https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework

Would love to discuss how these metrics apply to financial NLP model selection!