Spaces:

---

locuslab

/

tofu_leaderboard

like 13

Sleeping

Hi TOFU team!

LLM unlearning evaluation is fascinating and critical for privacy compliance. For teams measuring model quality before/after unlearning, I built a framework that captures the full quality picture.

LLM Evaluation Framework measures:

• Accuracy — did unlearning degrade general task accuracy?
• Hallucination Rate — unlearned models sometimes fill gaps with hallucinations
• Reasoning Quality — does unlearning affect chain-of-thought coherence?
• Cost per 1K tokens — production cost after fine-tuning/unlearning
• Latency p95 — inference latency before vs after unlearning

The before/after comparison across all 5 metrics gives a much richer picture than accuracy alone.

Live demo: https://huggingface.co/spaces/vigneshwar234/llm-eval-demo
GitHub: https://github.com/vignesh2027/LLM-Evaluation-Framework

Would love to discuss evaluation approaches for unlearned models!