SpecInfer: Accelerating Generative LLM Serving with Speculative Inference and Token Tree Verification
Paper • 2305.09781 • Published • 4
AI & ML interests
ML algorithms and systems
ZMC2019
authored 10
papers about 1 year ago
GNNPipe: Scaling Deep GNN Training with Pipelined Model Parallelism
Paper • 2308.10087 • Published • 1
Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
Paper • 2402.12374 • Published • 4
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
Paper • 2404.11912 • Published • 17
SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
Paper • 2406.02532 • Published • 13
MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding
Paper • 2408.11049 • Published • 14
MINI-SEQUENCE TRANSFORMER: Optimizing Intermediate Memory for Long Sequences Training
Paper • 2407.15892 • Published
Sirius: Contextual Sparsity with Correction for Efficient LLMs
Paper • 2409.03856 • Published • 1
MagicPIG: LSH Sampling for Efficient LLM Generation
Paper • 2410.16179 • Published
CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Models
Paper • 2502.00433 • Published
beidic
authored a
paper over 1 year ago
Hanyuezhuohua
authored 9
papers over 1 year ago
APE: Faster and Longer Context-Augmented Generation via Adaptive Parallel Encoding
Paper • 2502.05431 • Published • 6
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
Paper • 2402.09398 • Published
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
Paper • 2205.13542 • Published
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity
Paper • 2406.02913 • Published
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Paper • 2301.08739 • Published
Improving Domain Generalization with Domain Relations
Paper • 2302.02609 • Published
VcLLM: Video Codecs are Secretly Tensor Codecs
Paper • 2407.00467 • Published • 2
It Takes Two: On the Seamlessness between Reward and Policy Model in RLHF
Paper • 2406.07971 • Published
S$^{2}$FT: Efficient, Scalable and Generalizable LLM Fine-tuning by Structured Sparsity
Paper • 2412.06289 • Published