---
license: mit
tags:
- kv-cache
- quantization
- efficient-inference
- llm-optimization
- memory-optimization
- inference
---

# KV-Cache-Compression-Lab: Systematic Analysis of KV-Cache Quantization for Efficient LLM Inference

**Author:** Samudhan S (@samudhan2003)  
**Portfolio:** Part of 5-project Efficient Inference Series

## Abstract

The Key-Value (KV) cache is a major memory bottleneck in autoregressive LLM inference, scaling linearly with sequence length and batch size. This lab presents a systematic empirical analysis of KV-cache compression techniques (FP16 baseline vs INT8 vs INT4) and their interaction with weight quantization, measuring the trade-off between VRAM reduction, latency improvement, and quality degradation on sub-1B scale models.

## Motivation

During decoding, KV-cache memory often exceeds model weight memory:
- For Llama-2