research/optimization-catalog.md

Optimization Catalog — Detailed Research

Chef et Dev | Gemma Speed Challenge

---

1. FP8 Quantization (Priority: HIGH)

Why FP8?

• A10G has native FP8 tensor cores (compute capability 8.9)
• Expected 2x throughput vs BF16 for matmul operations
• Minimal PPL impact if calibration is correct

Implementation

# vLLM command line:
--quantization fp8
--load-format fp8
--dtype bfloat16  # compute dtype stays BF16 for stability

Calibration Strategy

1. Use auto-calibration (vLLM handles this)
2. Validate PPL on ground_truth_tokens after each change
3. If PPL > 2.42, switch to fp8_e4m3fnz (more conservative)

Expected Gains

• Throughput: +15-25%
• PPL impact: minimal if calibrated
• Risk: medium (must validate carefully)

---

2. AWQ Int4 Quantization (Priority: HIGH)

Why AWQ?

• Pre-computed AWQ weights available for Gemma-4
• Better PPL stability than GGUF
• Proven technique for 8B models

Implementation

# Requires AWQ weight files:
--model /path/to/gemma-4-E4B-it-awq
--quantization awq
--load-format awq
--dtype float16

Expected Gains

• Throughput: +20-30% (int4 matmul on A10G)
• PPL impact: moderate (usually stays under 2.42)
• Risk: low-medium (well-studied technique)

---

3. FlashInfer Attention Backend (Priority: MEDIUM)

Why FlashInfer?

• Faster than default Triton for gemm-based attention
• Better memory bandwidth utilization
• Already bundled with vLLM 0.22.0

Implementation

# Should be automatic with vLLM 0.22.0
# Explicitly set if needed:
--attention-backend flashinfer

Expected Gains

• Throughput: +5-10%
• PPL impact: negligible
• Risk: low

---

4. KV Cache Quantization (Priority: MEDIUM)

Why KV Cache Quantization?

• KV cache dominates VRAM for long contexts
• Quantizing KV cache frees memory for larger batch sizes
• Can be combined with other optimizations

Implementation

--quantization fp8  # includes KV cache quantization
# Or explicit KV cache quantization:
--kv-cache-dtype fp8

Expected Gains

• Throughput: +10-15% (indirect, via memory savings)
• PPL impact: small
• Risk: medium

---

5. CUDA Graph Optimization (Priority: MEDIUM)

Why CUDA Graphs?

• Eliminates Python/CUDA overhead per call
• Critical for single-stream benchmark
• vLLM handles this automatically, but needs warmup

Implementation

# Warmup calls before benchmark:
for i in range(20):
    await client.completions.create(
        model="gemma-4-e4b-it",
        prompt=[...],  # representative prompt
        max_tokens=64,
        temperature=0.0
    )

Expected Gains

• Latency reduction: 5-15ms per request
• Throughput: +5-10% (indirect)
• Risk: low

---

6. Scheduler Parameter Tuning (Priority: LOW)

Key Parameters

• --max-num-batched-tokens 512 — already set
• --max-num-seqs 64 — increase for more parallelism
• --scheduler-delay-factor 0.05 — dynamic scheduling
• --num-scheduler-steps 10 — batching frequency

Expected Gains

• Throughput: +2-5%
• PPL impact: negligible
• Risk: low

---

7. Layer Removal (Priority: LOW)

Why Layer Removal?

• Fewer layers = less computation
• Can be effective if PPL allows

Implementation

• Modify model config to skip top N layers
• Must validate PPL after each removal

Expected Gains

• Throughput: +10-20% (proportional to layers removed)
• PPL impact: HIGH (often exceeds cap)
• Risk: HIGH

---

Optimization Strategy

Recommended Order:

1. Baseline (BF16) → establish reference
2. FlashInfer → quick win, low risk
3. FP8 quantization → high gain, validate PPL
4. CUDA graph warmup → optimize timing
5. AWQ int4 → if FP8 doesn't meet target
6. KV cache quantization → combined with FP8
7. Scheduler tuning → fine-tuning phase

Key Constraint: PPL < 2.42

• Every optimization MUST be validated against PPL
• Reference PPL: ~2.30
• 5% margin: 2.30 * 1.05 = 2.415

---

Last updated: 2026-06-11 by chef-et-dev