Hawky-ai/longmemeval-results
Viewer • Updated • 8 • 91
DJ Sri Vigneshwar
Sri-Vigneshwar-DJ
AI & ML interests
Co-Founder, CTO @Hawky.ai - Creative Intelligence for Performance Marketing
Recent Activity
updated a dataset 7 days ago
Hawky-ai/longmemeval-results updated a Space 8 days ago
Hawky-ai/pm-agi-leaderboard updated a dataset 8 days ago
Hawky-ai/pm-agi-benchmarkOrganizations
updated a
dataset 7 days ago
updated a
Space 8 days ago
updated a
dataset 8 days ago
published an article 8 days ago
Article
Feather DB on LongMemEval: embedded retrieval beats full-context GPT-4o for $2.40
posted an update 9 days ago
Post
105
We ran Feather DB v0.8.0 on LongMemEval (ICLR 2025) — 500 questions across real multi-session conversations, up to 115K tokens each.
**Score: 0.693** · GPT-4o full-context baseline: 0.640
Full 500-question run with Gemini-Flash: **$2.40**
Per-axis breakdown:
→ Info-extraction: **0.942**
→ Knowledge-update: **0.714**
→ Multi-session: **0.606**
→ Temporal: **0.477** ← the hard one, Phase 9 addresses this
Architecture: Hybrid BM25+dense · adaptive temporal decay · embedded (no server) · p50 = 0.19ms · MIT
Raw results + audit JSONs: Hawky-ai/longmemeval-results
We ran Feather DB v0.8.0 on LongMemEval (ICLR 2025) — 500 questions across real multi-session conversations, up to 115K tokens each.
**Score: 0.693** · GPT-4o full-context baseline: 0.640
Full 500-question run with Gemini-Flash: **$2.40**
Per-axis breakdown:
→ Info-extraction: **0.942**
→ Knowledge-update: **0.714**
→ Multi-session: **0.606**
→ Temporal: **0.477** ← the hard one, Phase 9 addresses this
Architecture: Hybrid BM25+dense · adaptive temporal decay · embedded (no server) · p50 = 0.19ms · MIT
pip install feather-dbRaw results + audit JSONs: Hawky-ai/longmemeval-results
published a
dataset 11 days ago
updated a
Space 15 days ago
updated a
dataset 15 days ago
published a
dataset 15 days ago
updated a
Space 15 days ago
liked a
model 22 days ago
updated a
Space about 1 month ago
updated a
collection about 1 month ago
published a
Space about 1 month ago
liked a
model about 1 month ago
liked 2
models about 2 months ago
updated 2
collections about 2 months ago