LeaderboardV3
📈
3
Explore and compare text embedding benchmark results
AI & ML interests
Massive Text Embeddings Benchmark
Recent Activity
View all activity
Papers
MAEB: Massive Audio Embedding Benchmark
HUME: Measuring the Human-Model Performance Gap in Text Embedding Task
Samoed
updated a
Space 1 day ago
Feat Regional Analysis
1
#1 opened 6 days ago
by 
SmileXing
Samoed
updated a
Space 1 day ago
VORTEXRAG: 7-Layer RAG — Causal Drift Filtering + Context Poison Guard [paper + code + demo]
#186 opened 2 days ago
by 
vigneshwar234
Post
942
Human brains don't recreate every pixel to understand the world!
Most current models in genomics, proteomics, and single-cell transcriptomics rely on generative objectives like masked language modeling or next token prediction. While effective, these architectures waste significant capacity reconstructing raw, noisy sequence details that may not carry functional biological meaning.
But a promising, more efficient alternative is emerging: Joint-Embedding Predictive Architecture (JEPA)
Originally introduced by Yann LeCun for computer vision, JEPA is a non-generative, self-supervised learning (SSL) framework. Instead of predicting raw inputs, it operates as a world model that predicts abstract semantic embeddings in latent space.
Recently, the JEPA framework (and its more efficient LeJEPA variant) has been adapted into the biological sciences to develop performing foundation models and to improve on already existing ones.
It's interesting how each adaptation modified and tailored JEPA to suit its specific biological domain, whether by experimenting with different backbones or complementing the objective with other loss terms.
For example, JEPA-DNA and ProteinJEPA used JEPA as a continual pre-training framework to enhance existing foundation models without training from scratch, while Cell-JEPA and JEPA-DNA employed a hybrid objective that combines the JEPA loss with a traditional language modeling loss.
The article below provides an overview of these implementations, along with others that came out this year. As always, your thoughts and feedback are welcome and highly appreciated!
Link to the article is in the first comment 👇
Most current models in genomics, proteomics, and single-cell transcriptomics rely on generative objectives like masked language modeling or next token prediction. While effective, these architectures waste significant capacity reconstructing raw, noisy sequence details that may not carry functional biological meaning.
But a promising, more efficient alternative is emerging: Joint-Embedding Predictive Architecture (JEPA)
Originally introduced by Yann LeCun for computer vision, JEPA is a non-generative, self-supervised learning (SSL) framework. Instead of predicting raw inputs, it operates as a world model that predicts abstract semantic embeddings in latent space.
Recently, the JEPA framework (and its more efficient LeJEPA variant) has been adapted into the biological sciences to develop performing foundation models and to improve on already existing ones.
It's interesting how each adaptation modified and tailored JEPA to suit its specific biological domain, whether by experimenting with different backbones or complementing the objective with other loss terms.
For example, JEPA-DNA and ProteinJEPA used JEPA as a continual pre-training framework to enhance existing foundation models without training from scratch, while Cell-JEPA and JEPA-DNA employed a hybrid objective that combines the JEPA loss with a traditional language modeling loss.
The article below provides an overview of these implementations, along with others that came out this year. As always, your thoughts and feedback are welcome and highly appreciated!
Link to the article is in the first comment 👇
- 3 replies
Samoed
updated a
dataset 3 days ago
Samoed
updated a
Space 3 days ago
orionweller
updated a
Space 5 days ago
MTEB Leaderboard is really slow for loading & processing queries
👍 3
3
#185 opened 8 days ago
by 
Alanox
MTEB Leaderboard is really slow for loading & processing queries
👍 3
3
#185 opened 8 days ago
by Alanox
Feat Regional Analysis
1
#1 opened 6 days ago
by SmileXing
Samoed
published a
dataset 6 days ago
AdnanElAssadi
updated a
dataset 6 days ago
Samoed
published 2
Spaces 7 days ago
Post
124
Things rarely go as we expect!
In 2017, Google released the Transformer architecture. While it was clear the model was promising, absolutely no one (including its authors) anticipated the pervasive global revolution it would create!
The authors actually viewed the Transformer as just a stepping stone for a much more ambitious project: The MultiModel.
Their ultimate goal was to build a single deep learning architecture capable of jointly learning massive, diverse tasks across entirely different domains (in 2017). A One Model To Learn Them All.
In fact, the MultiModel paper was published in the exact same month as Attention Is All You Need!
But history had other plans. The building block eclipsed the grand design!
So, have you heard about the MultiModel before? 😀
In 2017, Google released the Transformer architecture. While it was clear the model was promising, absolutely no one (including its authors) anticipated the pervasive global revolution it would create!
The authors actually viewed the Transformer as just a stepping stone for a much more ambitious project: The MultiModel.
Their ultimate goal was to build a single deep learning architecture capable of jointly learning massive, diverse tasks across entirely different domains (in 2017). A One Model To Learn Them All.
In fact, the MultiModel paper was published in the exact same month as Attention Is All You Need!
But history had other plans. The building block eclipsed the grand design!
So, have you heard about the MultiModel before? 😀
- 1 reply
AdnanElAssadi
updated a
dataset 11 days ago
AdnanElAssadi
published a
dataset 11 days ago
AdnanElAssadi
updated a
dataset 12 days ago