Eximius Labs

This is EximiusLabs' home for open model weights and multimodal embedding research. Everything here gives developers and researchers a production-ready starting point for cross-modal retrieval and semantic search, backed by the research behind the Fusion embedding stack.

The Fusion Family: Unified Multimodal Embeddings

The Fusion family is EximiusLabs' lineup of embedding models that map text, images, video, and audio into a single shared semantic space. Each model targets a specific retrieval profile, from lightweight edge-scale vectors to full-fidelity cross-modal search, and ships with open weights and reproducible training recipes.

Fusion Embedding 1 (Preview)

The flagship embedding model extends a vision-language base into audio without touching the base weights. Fusion Embedding 1 builds on Qwen3-VL-Embedding-2B and adds a trained connector (~16M parameters) that maps frozen Qwen2.5-Omni audio features into the base model's space. The result is one embedding covering four modalities, with retrieval in any direction between them, instead of a stack of modality-specific encoders.

• Trained exclusively on audio-text pairs, the model still generalizes to unseen pairings, reaching audio-to-image R@10 of 0.407 on VGGSound-AV with no image-audio supervision.

• On VGGSound-AV it reaches audio-text R@10 of 0.625 and 0.645, ahead of ImageBind, LanguageBind, and Gemini Embedding 2 in both directions.

• A single forward pass yields nested Matryoshka embeddings truncatable to {2048, 1536, 1024, 512, 256, 128, 64} dimensions, trading storage and latency for accuracy without re-encoding.

• Only the FusionResampler connector is trained. The base model and audio tower stay frozen, so existing Qwen3-VL-Embedding pipelines gain audio without regression or re-deployment.

Architecture

A perceiver-resampler connector (width 384, 64 latent queries) is trained with contrastive learning (InfoNCE) across a Matryoshka ladder. Training runs in two stages: a contrastive phase on ~484K caption pairs, followed by connector-only fine-tuning on AudioCaps. Inputs are 16 kHz mono audio in 30-second windows. The connector ships as a ~60 MB distribution, with the frozen towers downloaded separately.

Fusion Training Data

Fusion Embedding 1 was trained on ~484K open audio-caption pairs drawn from AudioCaps (45K clips), FSD50K, WavCaps / AudioSet_SL, and a 318K-clip LAION-FreeSound subset. Evaluation sets (AudioCaps test, Clotho, VGGSound, ESC-50) were excluded by clip ID to keep benchmarks honest.

Preview Status

Fusion Embedding 1 is an English-language preview release under CC-BY-NC-4.0. It is optimized for sound-event retrieval; speech and music are still undertrained, and audio-text retrieval currently sits below fully fine-tuned CLAP specialists. We're releasing it early to invite research and feedback on unified multimodal embedding spaces.

eximiuslabs.com · github.com/Eximius-Labs