Pretraining Transformers with Quantized Softmax in Attention
Paper • 2609.33591 • Published • 4
Shangzhen Zhu
ArgonnSZ
·
AI & ML interests
None yet
Recent Activity
submitted a paper about 16 hours ago
Pretraining Transformers with Quantized Softmax in Attention upvoted a paper about 16 hours ago
Softmax Reparameterization for Output-Head Quantization upvoted a paper about 20 hours ago
Approximating Softmax in Pretrained LLMs: Model Sensitivity and Kernel AccelerationOrganizations
