inference-engineering-academy/llm-inference-notes
Updated • 48
LLM Inference Engineering: notes & tools
Worked-math notes on KV cache, prefill/decode, batching, quantization, serving engines + KV-cache calculator. Free course: inference-engineering.xyz
-
LLM KV Cache & GPU Memory Calculator
🧮KV cache size, weights and max batch for LLM serving
Note Interactive KV-cache memory calculator