Note Start here. Pope et al. give you the actual arithmetic — memory bandwidth vs. FLOPs, why decode is bandwidth-bound and prefill is compute-bound. Every decision below follows from this.

Note Scaling laws. The baseline everyone quotes; read it to know what it does *not* say about inference.

Note Chinchilla. Compute-optimal training gives you smaller models for the same quality — which is really an inference-cost result.

Note Multi-query attention. Shazeer's short paper is the origin of every KV-cache reduction that followed.

Note GQA. The interpolation between MQA and MHA that essentially every modern open model now ships.

Note FlashAttention. IO-awareness as a design principle — the same reasoning applies far beyond attention.

Note FlashAttention-2. Read for the work-partitioning discussion; it is a lesson in occupancy.

Note PagedAttention / vLLM. Virtual memory applied to the KV cache. The single highest-leverage serving idea of the last few years.

Note Speculative decoding. Trades cheap parallel compute for expensive sequential memory traffic. Know the acceptance-rate math before you promise a speedup.

Note SGLang / RadixAttention. Prefix sharing across requests — huge for agent and few-shot workloads with common prefixes.

Note Mooncake. Disaggregated prefill/decode in production. This is the shape serving fleets are converging on.

Note H2O. KV-cache eviction — the beginning of treating cache entries as unequal.

Note StreamingLLM / attention sinks. Why naive KV eviction collapses, and the cheap fix.

Note Switch Transformers. MoE from the systems side: capacity factors and routing cost, not just the quality win.

Note Mixtral. The report that made sparse MoE the default assumption for open frontier models.

Note DeepSeek-V2. Multi-head latent attention — an aggressive KV-cache compression that actually shipped at scale.

Note The papers above, reduced to numbers on one fully-specified machine. Prefill vs decode on a bandwidth-limited unified-memory part, measured rather than argued.