How to Train Data-Efficient LLMs
Paper
• 2402.09668 • Published
• 43
Efficient LMs
-
-
SliceGPT: Compress Large Language Models by Deleting Rows and Columns
Paper • 2401.15024 • Published • 73 -
SHERL: Synthesizing High Accuracy and Efficient Memory for Resource-Limited Transfer Learning
Paper • 2407.07523 • Published • 6 -
Spectra: A Comprehensive Study of Ternary, Quantized, and FP16 Language Models
Paper • 2407.12327 • Published • 79 -
Compact Language Models via Pruning and Knowledge Distillation
Paper • 2407.14679 • Published • 39 -
DDK: Distilling Domain Knowledge for Efficient Large Language Models
Paper • 2407.16154 • Published • 22 -
Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning
Paper • 2408.00690 • Published • 25 -
MiniCPM-V: A GPT-4V Level MLLM on Your Phone
Paper • 2408.01800 • Published • 92 -
MobileQuant: Mobile-friendly Quantization for On-device Language Models
Paper • 2408.13933 • Published • 16 -
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models
Paper • 2409.17066 • Published • 28 -
Addition is All You Need for Energy-efficient Language Models
Paper • 2410.00907 • Published • 151 -
Selective Attention Improves Transformer
Paper • 2410.02703 • Published • 25