---
title: DotCache Paper Demo
sdk: gradio
python_version: 3.10.13
sdk_version: 6.11.0
app_file: space_app.py
colorFrom: blue
colorTo: green
thumbnail: https://huggingface.co/spaces/DeanoCalver/DotCache-Arena/resolve/main/banner.png
models:
  - Qwen/Qwen3.5-4B
  - Qwen/Qwen3.5-9B
  - Qwen/Qwen3.5-27B
startup_duration_timeout: 1h
preload_from_hub:
  - Qwen/Qwen3.5-4B
---


# 🧠 DotCache Paper Demo

> Paper-backed Qwen 4B / 9B / 27B results from the current DotCache draft

---

## 🚀 Try it

👇 Launch the demo and inspect the paper-backed rows or try the live lane on supported Qwen models  
👉 **[Run the Playground](https://huggingface.co/spaces/DeanoCalver/DotCache-Arena)**

---

## ⚡ TL;DR

- The paper's headline result is a completed **Qwen3.5 4B / 9B / 27B** matrix
- The promoted systems profile delivers roughly **2x to 4x decode speedups**
- Compact-task correctness stays unchanged across the reported Qwen rows
- The learned execution path remains strongly **M3-heavy**
- LongBench is presented as a sanity check, not a full quality frontier

---

## 🧪 What you can do here

This Space now centers the paper's main sections:

- Compact-task matrix
- Backend-truth decode rows
- LongBench QA mini-pack sanity check

The preset mode uses benchmark-backed fixtures extracted from the bundled matrix. The live lane now replays the bundled Qwen benchmark rows directly where the required selector artifact is available.

---

## 🧠 What is DotCache?

DotCache explores a simple shift in how KV compression is used:

> Instead of treating compressed KV as storage, treat it as an **execution format**

In most pipelines:

KV (compressed) → decompress → attention

In DotCache:

KV (compressed) → attention directly

This removes widening overhead and changes the runtime behavior of attention itself.

---

## ⚙️ Key ideas

### 📄 Page-based KV
- KV is stored in small pages:
  - (layer, head, token range, K/V)
- Enables selective access and execution

### 🧮 Compressed-domain attention
- Decode is fused into:
  - score (Q·K)
  - mix (attention-weighted V)
- Avoids reconstructing full tensors

### 🎯 Adaptive page selection
- Not all pages are equal
- A policy (or learned selector) decides:
  - which pages stay low-bit
  - which escape to high precision (M3)

### 🔀 Asymmetric K/V handling
- Keys and values are treated differently
- Because:
  - keys affect scoring
  - values affect mixing

---

## 📊 What we’re seeing so far

### ✅ Qwen scaling is the headline
- The draft reports consistent multi-x decode wins on Qwen 4B, 9B, and 27B
- The systems profile preserves task success on the reported compact-task rows

### ✅ The learned path is still mostly M3
- The current result is about execution structure, not aggressive compression ratio
- Selector overhead stays small while backend score and mix dominate the remaining cost

### ✅ LongBench is intentionally framed narrowly
- The mini-pack is a sanity check showing no regression on the reported rows
- It is not yet presented as a full long-context reasoning frontier

---

## ⚠️ What’s still open

This is still **active research**:

- ❌ LongBench coverage is still a mini-pack
- ❌ The promoted Qwen lanes are strongly M3-heavy, so the draft is not yet a strong compression-ratio result
- ❌ External matched-budget baselines remain future work

> The paper's current claim is that execution structure already buys meaningful serving wins.

---

## 🧭 Why this matters

If low-bit KV is already viable, the next question is:

> **What is the cheapest way to *run attention* on it?**

DotCache suggests:
- compression is not just about memory
- it’s about **execution paths and decisions**

---

## 🧩 Example insight

The paper's current learned Qwen lanes are not sparse low-bit winners. They are mostly high-fidelity `M3` pages, yet they still run much faster. That points to the runtime execution path itself as the main optimization.

---

## 🔗 Links

- 💻 Repo: *(link)*
- 📄 Technical write-up: *(link)*
- 🧠 Paper draft (WIP): `DotCacheArXiv.tex`

---

## 🧠 One-line takeaway

> DotCache turns page format into a serving-time execution policy.