Org card: the Fusion Embedding family (both generations, Ember, leaderboards)

README.md

@@ -9,34 +9,71 @@ pinned: false
 
 ![Eximius Labs](assets/banner.png)
 
-This is EximiusLabs' home for open model weights and multimodal embedding research. Everything here gives developers and researchers a production-ready starting point for cross-modal retrieval and semantic search, backed by the research behind the Fusion embedding stack.
-
-## The Fusion Family: Unified Multimodal Embeddings
-
-The Fusion family is EximiusLabs' lineup of embedding models that map text, images, video, and audio into a single shared semantic space. Each model targets a specific retrieval profile, from lightweight edge-scale vectors to full-fidelity cross-modal search, and ships with open weights and reproducible training recipes.
-
-### Fusion Embedding 1 (Preview)
-
-The flagship embedding model extends a vision-language base into audio without touching the base weights. Fusion Embedding 1 builds on **Qwen3-VL-Embedding-2B** and adds a trained connector (~16M parameters) that maps frozen **Qwen2.5-Omni** audio features into the base model's space. The result is one embedding covering four modalities, with retrieval in any direction between them, instead of a stack of modality-specific encoders.
-
-*   Trained exclusively on audio-text pairs, the model still generalizes to unseen pairings, reaching audio-to-image R@10 of 0.407 on VGGSound-AV with no image-audio supervision.
-
-*   On VGGSound-AV it reaches audio-text R@10 of 0.625 and 0.645, ahead of ImageBind, LanguageBind, and Gemini Embedding 2 in both directions.
-
-*   A single forward pass yields nested Matryoshka embeddings truncatable to {2048, 1536, 1024, 512, 256, 128, 64} dimensions, trading storage and latency for accuracy without re-encoding.
-
-*   Only the FusionResampler connector is trained. The base model and audio tower stay frozen, so existing Qwen3-VL-Embedding pipelines gain audio without regression or re-deployment.
-
-### Architecture
-
-A perceiver-resampler connector (width 384, 64 latent queries) is trained with contrastive learning (InfoNCE) across a Matryoshka ladder. Training runs in two stages: a contrastive phase on ~484K caption pairs, followed by connector-only fine-tuning on AudioCaps. Inputs are 16 kHz mono audio in 30-second windows. The connector ships as a ~60 MB distribution, with the frozen towers downloaded separately.
-
-## Fusion Training Data
-
-Fusion Embedding 1 was trained on ~484K open audio-caption pairs drawn from AudioCaps (45K clips), FSD50K, WavCaps / AudioSet_SL, and a 318K-clip LAION-FreeSound subset. Evaluation sets (AudioCaps test, Clotho, VGGSound, ESC-50) were excluded by clip ID to keep benchmarks honest.
-
-## Preview Status
-
-Fusion Embedding 1 is an English-language preview release under **CC-BY-NC-4.0**. It is optimized for sound-event retrieval; speech and music are still undertrained, and audio-text retrieval currently sits below fully fine-tuned CLAP specialists. We're releasing it early to invite research and feedback on unified multimodal embedding spaces.
-
-[eximiuslabs.com](https://eximiuslabs.com/) · [github.com/Eximius-Labs](https://github.com/Eximius-Labs)
+This is Eximius Labs' home for open model weights and multimodal embedding
+research. Everything here gives developers and researchers a production-ready
+starting point for cross-modal retrieval and semantic search, backed by the
+research behind the Fusion Embedding stack.
+
+## The Fusion Embedding family
+
+One embedding space for text, images, video, and audio, built on a single
+design constraint: the base model is never retrained. Both generations are
+live on the public MTEB leaderboards for audio (MAEB) and video (MVEB),
+evaluated through the official `mteb` harness anyone can rerun.
+
+**[fusion-embedding-1-2b-preview](https://huggingface.co/EximiusLabs/fusion-embedding-1-2b-preview)** (v0.3) —
+generation 1 trains a 16.4M-parameter connector between a frozen
+Qwen2.5-Omni audio tower and a byte-frozen Qwen3-VL-Embedding-2B base:
+under 1% of the model. AudioCaps audio-to-text R@10 0.741, at the frontier
+of models that keep their base frozen. Aligned on audio-text only, it
+retrieves images from sound at 29x chance with zero paired audio-visual
+training data.
+
+**[fusion-embedding-2-2b-preview](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview)** (v0.2) —
+generation 2 adds modality-gated adapters inside the frozen decoder layers
+(+44.2M parameters), opened only for audio inputs. For text, images, and
+video the adapters never execute and outputs are bit-for-bit identical to
+the base model, verified after every training run. The family's best
+text-to-audio results on every benchmark: AudioCaps R@10 0.785, Clotho
+zero-shot 0.485.
+
+**[fusion-embedding-2-ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember)** (v0.1) —
+Ember is the thermal sense for fusion-embedding-2 and the first *sense
+pack*: a separately loadable adapter pack that adds a modality while every
+existing vector stays valid. Text-to-thermal retrieval R@10 0.785 on a
+2,000-image gallery against a 0.224 frozen baseline. Sense packs are named
+for the physical trace their sensor reads; Ember reads heat.
+
+## How it works
+
+A perceiver-resampler connector maps frozen audio-tower features into the
+base's input space; gated bottleneck adapters add in-layer capacity per
+modality. Gates are mutually exclusive by input modality, so non-gated
+modalities execute the base's exact computation. A single forward pass
+yields nested Matryoshka embeddings truncatable to
+{2048, 1024, 512, 256, 64} dimensions. Training runs in hours on a single
+GPU; connectors and packs ship as small artifacts with the frozen towers
+downloaded separately.
+
+```python
+import mteb
+model = mteb.get_model("EximiusLabs/fusion-embedding-2-2b-preview")
+```
+
+## Training data
+
+The audio corpus is ~500K open audio-caption pairs (AudioCaps and
+AudioCaps 2.0, FSD50K, WavCaps/AudioSet-SL, a LAION-FreeSound subset);
+Ember trains on IR-TD real thermal imagery with restricted sources removed.
+Evaluation sets are excluded from training by clip id at ingestion, and
+every reported number traces to a result record in the open repository.
+
+## Status
+
+English-language research previews. Model weights are released under
+CC-BY-NC-4.0 (Ember under its own research license; see each model card).
+Sound-event retrieval is the strongest axis; speech and music are
+undertrained in the current corpus. We release early to invite research and
+feedback on unified multimodal embedding spaces.
+
+[eximiuslabs.com](https://eximiuslabs.com/) · [github.com/Eximius-Labs](https://github.com/Eximius-Labs)