Spaces:
Paused
Paused
| title: DotCache Paper Demo | |
| sdk: gradio | |
| python_version: 3.10.13 | |
| sdk_version: 6.11.0 | |
| app_file: space_app.py | |
| colorFrom: blue | |
| colorTo: green | |
| thumbnail: https://huggingface.co/spaces/DeanoCalver/DotCache-Arena/resolve/main/banner.png | |
| models: | |
| - Qwen/Qwen3.5-4B | |
| - Qwen/Qwen3.5-9B | |
| - Qwen/Qwen3.5-27B | |
| startup_duration_timeout: 1h | |
| preload_from_hub: | |
| - Qwen/Qwen3.5-4B | |
| # 🧠 DotCache Paper Demo | |
| > Paper-backed Qwen 4B / 9B / 27B results from the current DotCache draft | |
| --- | |
| ## 🚀 Try it | |
| 👇 Launch the demo and inspect the paper-backed rows or try the live lane on supported Qwen models | |
| 👉 **[Run the Playground](https://huggingface.co/spaces/DeanoCalver/DotCache-Arena)** | |
| --- | |
| ## ⚡ TL;DR | |
| - The paper's headline result is a completed **Qwen3.5 4B / 9B / 27B** matrix | |
| - The promoted systems profile delivers roughly **2x to 4x decode speedups** | |
| - Compact-task correctness stays unchanged across the reported Qwen rows | |
| - The learned execution path remains strongly **M3-heavy** | |
| - LongBench is presented as a sanity check, not a full quality frontier | |
| --- | |
| ## 🧪 What you can do here | |
| This Space now centers the paper's main sections: | |
| - Compact-task matrix | |
| - Backend-truth decode rows | |
| - LongBench QA mini-pack sanity check | |
| The preset mode uses benchmark-backed fixtures extracted from the bundled matrix. The live lane now replays the bundled Qwen benchmark rows directly where the required selector artifact is available. | |
| --- | |
| ## 🧠 What is DotCache? | |
| DotCache explores a simple shift in how KV compression is used: | |
| > Instead of treating compressed KV as storage, treat it as an **execution format** | |
| In most pipelines: | |
| KV (compressed) → decompress → attention | |
| In DotCache: | |
| KV (compressed) → attention directly | |
| This removes widening overhead and changes the runtime behavior of attention itself. | |
| --- | |
| ## ⚙️ Key ideas | |
| ### 📄 Page-based KV | |
| - KV is stored in small pages: | |
| - (layer, head, token range, K/V) | |
| - Enables selective access and execution | |
| ### 🧮 Compressed-domain attention | |
| - Decode is fused into: | |
| - score (Q·K) | |
| - mix (attention-weighted V) | |
| - Avoids reconstructing full tensors | |
| ### 🎯 Adaptive page selection | |
| - Not all pages are equal | |
| - A policy (or learned selector) decides: | |
| - which pages stay low-bit | |
| - which escape to high precision (M3) | |
| ### 🔀 Asymmetric K/V handling | |
| - Keys and values are treated differently | |
| - Because: | |
| - keys affect scoring | |
| - values affect mixing | |
| --- | |
| ## 📊 What we’re seeing so far | |
| ### ✅ Qwen scaling is the headline | |
| - The draft reports consistent multi-x decode wins on Qwen 4B, 9B, and 27B | |
| - The systems profile preserves task success on the reported compact-task rows | |
| ### ✅ The learned path is still mostly M3 | |
| - The current result is about execution structure, not aggressive compression ratio | |
| - Selector overhead stays small while backend score and mix dominate the remaining cost | |
| ### ✅ LongBench is intentionally framed narrowly | |
| - The mini-pack is a sanity check showing no regression on the reported rows | |
| - It is not yet presented as a full long-context reasoning frontier | |
| --- | |
| ## ⚠️ What’s still open | |
| This is still **active research**: | |
| - ❌ LongBench coverage is still a mini-pack | |
| - ❌ The promoted Qwen lanes are strongly M3-heavy, so the draft is not yet a strong compression-ratio result | |
| - ❌ External matched-budget baselines remain future work | |
| > The paper's current claim is that execution structure already buys meaningful serving wins. | |
| --- | |
| ## 🧭 Why this matters | |
| If low-bit KV is already viable, the next question is: | |
| > **What is the cheapest way to *run attention* on it?** | |
| DotCache suggests: | |
| - compression is not just about memory | |
| - it’s about **execution paths and decisions** | |
| --- | |
| ## 🧩 Example insight | |
| The paper's current learned Qwen lanes are not sparse low-bit winners. They are mostly high-fidelity `M3` pages, yet they still run much faster. That points to the runtime execution path itself as the main optimization. | |
| --- | |
| ## 🔗 Links | |
| - 💻 Repo: *(link)* | |
| - 📄 Technical write-up: *(link)* | |
| - 🧠 Paper draft (WIP): `DotCacheArXiv.tex` | |
| --- | |
| ## 🧠 One-line takeaway | |
| > DotCache turns page format into a serving-time execution policy. | |