README.md

metadata

title: Oolong Viewer
emoji: 🍵
colorFrom: yellow
colorTo: gray
sdk: static
pinned: false
license: mit

Oolong Viewer

Static viewer for Oolong (paper, HF datasets) — a benchmark for long-context reasoning and aggregation, where a model must reason over a single very long context window (up to ~132K tokens) to answer counting / timeline / comparison style questions.

Two datasets, selectable at the top of the sidebar:

• synth — oolong-synth: synthetic contexts built from labeled NLP datasets (agnews, imdb, metaphors, …). 6,500 questions over 208 context windows.
• real — oolong-real toy_dnd config: Critical Role Dungeons & Dragons transcripts (campaign1 = test, campaign2 = validation). 949 questions over 22 context windows.

Two tabs:

• 📚 Corpus — the context windows (the long documents). A dropdown selects a context by title (filter by title text, or prev/next). Each shows its metadata (source dataset / campaign / episodes, token length, how many questions use it, context_window_id) and its full text. Context windows are reused across many questions, so the corpus is far smaller than the question set.
• ❓ Eval — the questions. A dropdown selects a question (search + one dropdown per dataset facet — task group / answer type / source, or question type / split / campaign). Each shows the question, the gold answer, its supporting context window (a clickable chip that jumps to it in the Corpus tab), and all remaining metadata.

Long contexts are capped

Oolong is a long-context benchmark: some context windows are up to ~14 MB of text. To keep the viewer responsive and every file under Hugging Face's 10 MB non-LFS limit, each context-window shard is capped at 2 MB (with a truncated in viewer note); the full contexts live on the HF datasets. The context_window_text_with_labels variant (synth) is not included.

Data layout

File	Contents
sets.json	manifest: per-set question/context counts + filter facets
corpus_<set>.json	context-window index: {cwid, title, size, truncated, n_questions, file, …}
corpus_<set>/*.txt	one (capped) context-window shard, lazy-loaded
eval_<set>.json	questions: {id, context_window_id, question, answer, meta{…}}

Rebuilding the data

python scripts/build_data.py --data-dir /mnt/tmp/oolong

Reads the downloaded oolong-synth/ (parquet) + oolong-real/toy_dnd/ (jsonl), dedupes context windows into shards, projects the questions, and rewrites corpus_<set>.json + corpus_<set>/ + eval_<set>.json + sets.json. The raw datasets (12 GB / 1.4 GB) are not committed here. Use --only synth|real to rebuild a single set.

Local dev

python -m http.server 8000   # then open http://localhost:8000/

Serves statically — no build step. Push to the HF Space remote to deploy.

Dataset license: see the Oolong repo / dataset cards.