BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
Paper • 2412.03594 • Published
AI & ML interests
build frontier intelligence for everyday life
Recent Activity
Papers
DoSA: Dots Sparse Attention
VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?
constfrost
authored 2
papers 7 days ago
fxmeng
submitted a
paper to Daily Papers 9 days ago
whk-sjtu
updated a
Space 15 days ago
debingzhang
authored a
paper 11 months ago
ygfrancois
updated a
model 12 months ago
redmoe-ai-v1
updated a
collection 12 months ago
redmoe-ai-v1
published a
model 12 months ago
Challenges and Limitations in Fine-Tuning the Dots.OCR Model
12
#39 opened 12 months ago
by 
JamesGs
redmoe-ai-v1
updated a
collection 12 months ago
redmoe-ai-v1
updated a
model 12 months ago
fix autoprocessor
#30 opened about 1 year ago
by 
kingsley01
Will it be supported by vLLM officially?
👍👀 8
1
#20 opened about 1 year ago
by 
riverclouds
没打算提交PR 给 vllm 和 transformers 吗？
👍 1
1
#28 opened about 1 year ago
by 
CHONGYOEYAT
Adding `transformers` as a library, and also mentioning the `custom_code` tag
#29 opened about 1 year ago
by 
ariG23498
AttributeError: module 'fitz' has no attribute 'open'
1
#21 opened about 1 year ago
by 
kaqmoh2025-A
Update VisionSdpaAttention to support memory efficient backend.
#27 opened about 1 year ago
by 
warrenwjk
yonghenglh6
updated a
model about 1 year ago
Getting these attached issues, please anyone help me
#15 opened about 1 year ago
by 
BipinBisht
Update modeling_dots_vision.py
🔥 1
#19 opened about 1 year ago
by 
chenj123