---
title: README
emoji: 🌍
colorFrom: blue
colorTo: blue
sdk: static
pinned: false
---

![Eximius Labs](assets/banner.png)

This is EximiusLabs' home for open model weights and multimodal embedding research. Everything here gives developers and researchers a production-ready starting point for cross-modal retrieval and semantic search, backed by the research behind the Fusion embedding stack.

## The Fusion Family: Unified Multimodal Embeddings

The Fusion family is EximiusLabs' lineup of embedding models that map text, images, video, and audio into a single shared semantic space. Each model targets a specific retrieval profile, from lightweight edge-scale vectors to full-fidelity cross-modal search, and ships with open weights and reproducible training recipes.

### Fusion Embedding 1 (Preview)

The flagship embedding model extends a vision-language base into audio without touching the base weights. Fusion Embedding 1 builds on **Qwen3-VL-Embedding-2B** and adds a trained connector (~16M parameters) that maps frozen **Qwen2.5-Omni** audio features into the base model's space. The result is one embedding covering four modalities, with retrieval in any direction between them, instead of a stack of modality-specific encoders.

*   Trained exclusively on audio-text pairs, the model still generalizes to unseen pairings, reaching audio-to-image R@10 of 0.407 on VGGSound-AV with no image-audio supervision.

*   On VGGSound-AV it reaches audio-text R@10 of 0.625 and 0.645, ahead of ImageBind, LanguageBind, and Gemini Embedding 2 in both directions.

*   A single forward pass yields nested Matryoshka embeddings truncatable to {2048, 1536, 1024, 512, 256, 128, 64} dimensions, trading storage and latency for accuracy without re-encoding.

*   Only the FusionResampler connector is trained. The base model and audio tower stay frozen, so existing Qwen3-VL-Embedding pipelines gain audio without regression or re-deployment.

### Architecture

A perceiver-resampler connector (width 384, 64 latent queries) is trained with contrastive learning (InfoNCE) across a Matryoshka ladder. Training runs in two stages: a contrastive phase on ~484K caption pairs, followed by connector-only fine-tuning on AudioCaps. Inputs are 16 kHz mono audio in 30-second windows. The connector ships as a ~60 MB distribution, with the frozen towers downloaded separately.

## Fusion Training Data

Fusion Embedding 1 was trained on ~484K open audio-caption pairs drawn from AudioCaps (45K clips), FSD50K, WavCaps / AudioSet_SL, and a 318K-clip LAION-FreeSound subset. Evaluation sets (AudioCaps test, Clotho, VGGSound, ESC-50) were excluded by clip ID to keep benchmarks honest.

## Preview Status

Fusion Embedding 1 is an English-language preview release under **CC-BY-NC-4.0**. It is optimized for sound-event retrieval; speech and music are still undertrained, and audio-text retrieval currently sits below fully fine-tuned CLAP specialists. We're releasing it early to invite research and feedback on unified multimodal embedding spaces.

[eximiuslabs.com](https://eximiuslabs.com/) · [github.com/Eximius-Labs](https://github.com/Eximius-Labs)