README.md

metadata

title: DotCache Paper Demo
sdk: gradio
python_version: 3.10.13
sdk_version: 6.11.0
app_file: space_app.py
colorFrom: blue
colorTo: green
thumbnail: >-
  https://huggingface.co/spaces/DeanoCalver/DotCache-Arena/resolve/main/banner.png
models:
  - Qwen/Qwen3.5-4B
  - Qwen/Qwen3.5-9B
  - Qwen/Qwen3.5-27B
startup_duration_timeout: 1h
preload_from_hub:
  - Qwen/Qwen3.5-4B

🧠 DotCache Paper Demo

Paper-backed Qwen 4B / 9B / 27B results from the current DotCache draft

---

🚀 Try it

👇 Launch the demo and inspect the paper-backed rows or try the live lane on supported Qwen models
👉 Run the Playground

---

⚡ TL;DR

• The paper's headline result is a completed Qwen3.5 4B / 9B / 27B matrix
• The promoted systems profile delivers roughly 2x to 4x decode speedups
• Compact-task correctness stays unchanged across the reported Qwen rows
• The learned execution path remains strongly M3-heavy
• LongBench is presented as a sanity check, not a full quality frontier

---

🧪 What you can do here

This Space now centers the paper's main sections:

• Compact-task matrix
• Backend-truth decode rows
• LongBench QA mini-pack sanity check

The preset mode uses benchmark-backed fixtures extracted from the bundled matrix. The live lane now replays the bundled Qwen benchmark rows directly where the required selector artifact is available.

---

🧠 What is DotCache?

DotCache explores a simple shift in how KV compression is used:

Instead of treating compressed KV as storage, treat it as an execution format

In most pipelines:

KV (compressed) → decompress → attention

In DotCache:

KV (compressed) → attention directly

This removes widening overhead and changes the runtime behavior of attention itself.

---

⚙️ Key ideas

📄 Page-based KV

• KV is stored in small pages:
  • (layer, head, token range, K/V)
• Enables selective access and execution

🧮 Compressed-domain attention

• Decode is fused into:
  • score (Q·K)
  • mix (attention-weighted V)
• Avoids reconstructing full tensors

🎯 Adaptive page selection

• Not all pages are equal
• A policy (or learned selector) decides:
  • which pages stay low-bit
  • which escape to high precision (M3)

🔀 Asymmetric K/V handling

• Keys and values are treated differently
• Because:
  • keys affect scoring
  • values affect mixing

---

📊 What we’re seeing so far

✅ Qwen scaling is the headline

• The draft reports consistent multi-x decode wins on Qwen 4B, 9B, and 27B
• The systems profile preserves task success on the reported compact-task rows

✅ The learned path is still mostly M3

• The current result is about execution structure, not aggressive compression ratio
• Selector overhead stays small while backend score and mix dominate the remaining cost

✅ LongBench is intentionally framed narrowly

• The mini-pack is a sanity check showing no regression on the reported rows
• It is not yet presented as a full long-context reasoning frontier

---

⚠️ What’s still open

This is still active research:

• ❌ LongBench coverage is still a mini-pack
• ❌ The promoted Qwen lanes are strongly M3-heavy, so the draft is not yet a strong compression-ratio result
• ❌ External matched-budget baselines remain future work

The paper's current claim is that execution structure already buys meaningful serving wins.

---

🧭 Why this matters

If low-bit KV is already viable, the next question is:

What is the cheapest way to run attention on it?

DotCache suggests:

• compression is not just about memory
• it’s about execution paths and decisions

---

🧩 Example insight

The paper's current learned Qwen lanes are not sparse low-bit winners. They are mostly high-fidelity M3 pages, yet they still run much faster. That points to the runtime execution path itself as the main optimization.

---

🔗 Links

• 💻 Repo: (link)
• 📄 Technical write-up: (link)
• 🧠 Paper draft (WIP): DotCacheArXiv.tex

---

🧠 One-line takeaway

DotCache turns page format into a serving-time execution policy.