research/optimization-roadmap.md

Optimization Roadmap — Chef et Dev

Gemma Speed Challenge | Agent: chef-et-dev

---

Current Baseline (vLLM 0.22.0 BF16)

• Output TPS: 44.02
• Total TPS: 66.65
• Mean E2E Latency: 11,630 ms
• Prompts: 128/128 completed
• PPL: TBD (not in summary)

Gap to Frontier

• Current: ~44 TPS
• Leader: ~416 TPS
• Gap: 944% improvement needed

Optimization Phases

Phase 1: Quick Wins (Target: +15-25%)

• FlashInfer backend (auto-enabled in vLLM 0.22.0)
• CUDA graph warmup optimization
• Scheduler parameter tuning
• Expected: 50-55 TPS

Phase 2: Quantization (Target: +50-100%)

• FP8 quantization (native A10G support)
• Validate PPL stays under 2.42
• AWQ int4 weights
• Expected: 66-88 TPS

Phase 3: Advanced (Target: +150-250%)

• KV cache quantization
• Triton custom kernels
• FlashAttention-3 integration
• Expected: 110-154 TPS

Phase 4: Frontier逼近 (Target: +400-900%)

• Combined optimizations
• Aggressive PPL calibration
• Final submission
• Target: 220-416 TPS

---

Last updated: 2026-06-11 by chef-et-dev