README.md

metadata

title: MEBench Viewer
emoji: 🧩
colorFrom: green
colorTo: blue
sdk: static
pinned: false
license: apache-2.0

MEBench Viewer

Static viewer for MEBench (paper, EMNLP 2025; HF dataset) — a cross-document multi-entity QA benchmark. Questions ask a model to retrieve, consolidate and reason over facts scattered across many entities that belong to a shared topic (e.g. the members of the Ivy League), across 8 question types in 3 reasoning categories.

Two tabs:

• 📚 Corpus — the entity documents. A dropdown selects a document by title (filter by title text and/or topic, or use prev/next). Each shows the entity's topic membership(s), a link to Wikipedia, and its intro text.
• ❓ Eval — the questions, with a split selector (train / test / single_test). A dropdown selects a question (filter by topic and/or question type, or search). Each shows the question, the gold answer (train answers are SQL-result tables; ∅ marks empty/null results), the supporting documents (clickable entity chips that jump to the Corpus tab; train questions range over a whole topic, so they link to all documents of that topic), and metadata (qid, reasoning category, type, topic, relation edge, extracted properties, hops/class, and the ground-truth SQL for train).

Splits

Split	Questions	Notes
train	5,764	includes the ground-truth SQL; answers are SQL-result tables
test	1,373	per-entity questions (single/multi-hop), NL answers
single_test	5,940	single-entity questions, NL answers

All 7 topics are university associations: Ivy League, Group of Eight, Russell Group, C9 League, Universitas 21, Association of American Universities, and the European University Association.

The corpus

MEBench ships questions only — in the benchmark, each "document" is an entity's Wikipedia intro paragraph, generated live by the pipeline and not included in the dataset files. This viewer therefore reconstructs the corpus from the entities that actually appear in the dataset (the Entity field of the test + single_test splits, which together cover all 7 topics): 172 unique entities, each fetched from Wikipedia (extracts, intro only) and tagged with the topic(s) it appears under. 3 / 172 entities had no English-Wikipedia intro and show a placeholder. It is a representative subset of each topic's full membership, not the exhaustive corpus.

Data layout

File	Contents
sets.json	split manifest (per-split question count, topics, type histogram)
corpus.json	entity index: {title, topics, size, wiki_url, file} (shared across splits)
corpus/*.txt	one Wikipedia-intro shard per entity (lazy-loaded)
eval_<split>.json	questions: {qid, topic, type, category, question, answer, edge, properties, supporting_titles, topic_level, sql?, class?, hops?, entity?}

Rebuilding the data

python scripts/build_data.py \
    --data-dir /mnt/ramdisk/blobstore/timchen0618/data/mebench

Reads train/test/single_test.jsonl, (re)fetches Wikipedia intros (cached to <data-dir>/wiki_cache.jsonl), and rewrites corpus.json + corpus/ + eval_<split>.json + sets.json. The raw MEBench JSONLs (gated on Hugging Face) are not committed here.

Local dev

python -m http.server 8000   # then open http://localhost:8000/

Serves statically — no build step. Push to the HF Space remote to deploy.

Dataset license: apache-2.0.