Spaces:

---

Duplicated from  akhaliq/LFM2.5-8B-A1B

LiquidAI

/

LFM2.5-8B-A1B

like 40

Running on Zero

Hi LiquidAI team 👋

LFM2.5 looks impressive! For teams evaluating whether to adopt it for production, the key question is always: how does it compare on accuracy/cost/hallucination vs GPT-4o-mini or Gemini Flash?

I built an open source LLM Evaluation Framework that answers this in one command:

llm-eval compare \
  --models gpt-4o-mini \
  --models gemini/gemini-1.5-flash \
  --benchmark mmlu --samples 100

→ 🎯 Accuracy % — MMLU + TruthfulQA side by side
→ 💰 Cost per 1K tokens — production cost comparison
→ ⚡ Latency p95 — real-world tail latency
→ 🔍 Hallucination Rate — overconfident wrong outputs
→ 🧠 Reasoning Quality

Live demo (no API key): https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework

Open source. Would love to add LFM2.5 to our benchmark comparison!