Transformers CI dashboard
👁
21
Track the transformers CI across GPUs
AI & ML interests
None defined yet.
Recent Activity
View all activity
Include NVIDIA flash attention CI results in NVIDIA numbers
#17 opened 1 day ago
by 
badaoui
Revert "Show overall failure rates with two decimals (#15)"
#16 opened 7 days ago
by badaoui
Show overall failure rates with two decimals
#15 opened 7 days ago
by badaoui
Show last real data instead of sample data when loading fails
#14 opened 9 days ago
by badaoui
Fix mask None-ness
#5 opened 22 days ago
by 
cyrilvallez
badaoui
updated a
Space 23 days ago
bezzam
submitted a
paper to Daily Papers about 2 months ago
Post
4191
🚨 I've just published Sentence Transformers v6.0, introducing MultiVectorEncoder: ColBERT-style late interaction models are now a fourth model type, for training, inference, and interpretation, alongside the dense, sparse, and reranker models! Details:
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
Where a regular embedding model compresses a whole text into one vector, a multi-vector model keeps one vector per token and scores query against document with the MaxSim operator. That preserves token-level matching information that a single vector has to average away. It is also the state of the art for visual document retrieval, where a text query is matched against page images directly, charts and tables included, with no OCR step in between.
Any PyLate, Stanford ColBERT, or ColPali checkpoint loads straight into the same familiar API: model.encode_query(), model.encode_document(), and model.similarity() just work, whether the documents are texts or page images.
Does it help? LightOn trained LateOn (multi-vector) and DenseOn (dense) on the same data with the same 149M ModernBERT backbone, and the multi-vector model wins on 9 of the 13 NanoBEIR datasets: 0.6868 vs 0.6764 mean NDCG@10. The price is a bigger index, and the new HierarchicalTokenPooling module halves it at roughly no retrieval cost.
Antoine Chaffin, Raphaël Sourty, and I wrote a blog post walking through multi-vector models in practice: loading the various checkpoint formats, encoding and scoring, plugging them into a search stack, running them on page images, and keeping the index affordable. Check it out if you want to get started, or just point your Agent to the URL: https://huggingface.co/blog/multi-vector-encoder
pip install sentence-transformers==6.0.0
Release notes: https://github.com/huggingface/sentence-transformers/releases/tag/v6.0.0
Post
2466
432 GB of ultra-fast HBM4 and up to 23.3 TB/s of memory bandwidth on a single GPU 🤯.
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
- 1 reply
Gausson
authored 7
papers 3 months ago
SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator
Paper • 2412.12094 • Published • 11
Self-Adjust Softmax
Paper • 2502.18277 • Published
Answer-Consistent Chain-of-thought Reinforcement Learning For Multi-modal Large Langauge Models
Paper • 2510.10104 • Published
LightGNN: Simple Graph Neural Network for Recommendation
Paper • 2501.03228 • Published
Pre-training for Recommendation Unlearning
Paper • 2505.22649 • Published
UniPool: A Globally Shared Expert Pool for Mixture-of-Experts
Paper • 2605.06665 • Published • 12
Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
Paper • 2607.13125 • Published • 140
cyrilvallez
updated a
model 3 months ago
Update AMD hardware label from MI325 to MI300
#13 opened 3 months ago
by badaoui
Optimize group beam search by sharing the initial prefill across beams
3
#6 opened 3 months ago
by 
lavrenko
Possible performance issue: group beam search appears to prefill identical prefixes once per beam
3
#5 opened 3 months ago
by lavrenko