Sentio / README.md

Sentio model card, config and modeling code

943d95f verified 3 days ago

11.5 kB

---
license: apache-2.0
pipeline_tag: feature-extraction
base_model:
- tencent/WeMM-Embedding-9B
- Qwen/Qwen2.5-Omni-3B
tags:
- audio
- multimodal-embedding
- audio-retrieval
- audio-classification
- video-moment-retrieval
---
# Sentio

![Sentio architecture](resources/sentio_architecture.png)

Sentio is WeMM-Embedding-9B with ears, and with sharper eyes for video moments.

WeMM-Embedding-9B is the strongest open embedding model for text, images, video and visual
documents, but it cannot hear. [Wianmu-5B](https://huggingface.co/eduardo-terres2/Wianmu-5B) gave
the 4B member of the family an audio tower by freezing both the embedding model and a pretrained
audio encoder and training only a small connector between them, the recipe of Tianmu-Emb-Uni.
Sentio does the same to the 9B, with two additions: before the audio stage, the 9B tower gets a
small LoRA fine-tune for video moment retrieval, so the vision side also moves; and during the audio
stage the audio encoder gets a LoRA of its own, trained together with the connector. The name is
Latin for "I perceive": one model that hears, sees and reads into one space. Total 10.20B
parameters, of which 152M (1.5%) were trained.

## How it works

Two experts, one that understands audio and one that understands everything else, and a trained
piece in the middle that moves audio into the other model's embedding space.

The audio side is the audio tower of Qwen2.5-Omni-3B (the encoder only, 638M parameters) with a
LoRA on its attention and MLP projections. Everything else goes through WeMM-Embedding-9B. Audio
lands in WeMM's 4096-d space, so an audio clip can be compared against a caption, a label name, an
image or a video directly, and nothing that was indexed with the tower has to be re-indexed.

```
waveform
  -> Whisper-style mel features (128 bins, up to 30 s)
  -> Qwen2.5-Omni audio tower + LoRA (r=16), masked-mean pooled over frames  -> 2048-d
  -> connector: 3-layer MLP, 2048 -> 4096 -> 8192 -> 4096, LayerNorm + GELU
  -> modality adapter fusion
  -> RMSNorm projection
  -> L2 normalize  -> 4096-d embedding
```

**Two stages, three trained pieces.**

1. *Stage 1, the tower.* WeMM-Embedding-9B gets a LoRA (r=16, α=32, 43.3M parameters) trained with
   InfoNCE on video moment-retrieval pairs mixed with a replay of visual-document, video and image
   pairs so that nothing already strong is lost. This is the only change to the non-audio path: a
   text, image, video or document embedding from Sentio is the embedding of WeMM-9B with this LoRA
   applied. It lifts MMEB-V2 from 80.58 to 80.68 and V-MRET from 58.46 to 61.53.
2. *Stage 2, the ears.* With the tower frozen (LoRA included), the audio path is trained: a LoRA on
   the audio tower (r=16, α=32, dropout 0.05, on q/k/v/out projections and both MLP layers of every
   encoder block, 11.8M parameters) together with the audio head — connector, adapter fusion,
   projection norm and prototypes, 96.8M parameters, almost all of it the connector (75.5M).

**Adapter fusion.** One audio clip can be matched against very different things. For a caption you
care about meaning, for another audio clip about acoustic detail, for a video about timing, and for
a label about the category. So there are four small bottleneck adapters, one each for text, audio,
video and label. A gate looks at the audio vector and mixes the four outputs with soft weights, with
a small entropy bonus during training so the gate doesn't collapse onto one adapter.

| Adapter | Input | Down projection | Activation | Bottleneck | Up projection | Output | Residual | Parameters |
|---|---|---|---|---|---|---|---|---|
| text  | 4096 | Linear 4096 → 512, no bias | GELU | 512 | Linear 512 → 4096, no bias | 4096 | yes | 4,194,304 |
| audio | 4096 | Linear 4096 → 512, no bias | GELU | 512 | Linear 512 → 4096, no bias | 4096 | yes | 4,194,304 |
| video | 4096 | Linear 4096 → 512, no bias | GELU | 512 | Linear 512 → 4096, no bias | 4096 | yes | 4,194,304 |
| label | 4096 | Linear 4096 → 512, no bias | GELU | 512 | Linear 512 → 4096, no bias | 4096 | yes | 4,194,304 |
| gate  | 4096 | Linear 4096 → 1024, no bias + LayerNorm(1024) | GELU | 1024 | Linear 1024 → 4, with bias | 4 (softmax weights) | no | 4,200,452 |

**Shared prototypes.** 64 learnable vectors in the shared space. Each audio embedding and each target
embedding gets a soft assignment over them, and training pushes the audio and its matching target
to pick the same prototypes. They are only used for the loss; at inference you just use the embedding.

**Parameter count** (measured from the weight files):

| Component | Parameters | Trained |
|---|---|---|
| WeMM-Embedding-9B tower | 9,408M | frozen |
| vdr LoRA on the tower (stage 1) | 43.3M | yes |
| Qwen2.5-Omni-3B audio tower | 638M | frozen |
| LoRA on the audio tower (stage 2) | 11.8M | yes |
| Connector (3-layer MLP) | 75.5M | yes (stage 2) |
| Adapter fusion (4 adapters + gate) | 21.0M | yes (stage 2) |
| RMSNorm projection + 64 prototypes | 0.27M | yes (stage 2) |
| **Total** | **10.20B** | **152M (1.5%)** |

## Training

*Stage 2 (audio).* The loss is symmetric in-batch InfoNCE between audio and target embeddings
(temperature 0.07), which does most of the work. On top of it come the Tianmu prototype loss — a
contrastive term over prototype assignments (0.03), a symmetric KL between the audio and target
assignments (0.01) and a balance term so all prototypes get used (0.005) — and a gate-entropy bonus
(0.01). For label datasets, batches are built so that each label appears at most once; otherwise two
clips of "dog bark" in one batch count as negatives of each other.

The data is 449k audio–target pairs over 37k distinct targets from seven public datasets, sampled
in natural proportion: NSynth (instrument labels, the bulk of every epoch), SpeechCommands
(spoken-word labels), FSD50K and RAVDESS (sound-event and emotion labels), Clotho (captions),
Flickr8k Audio (spoken captions paired with the image they describe) and AVE (clips paired with
their source video). Batch size 16 (smaller where a dataset has fewer distinct labels); learning
rate 1e-5 for the head and 1e-4 for the audio-tower LoRA, constant after 500 warm-up steps;
498,000 steps on one H200. Validation is the same in-batch InfoNCE on a held-out split every 2,500
steps; the released checkpoint is the one with the lowest validation loss (step ~408,000).

*Stage 1 (tower).* LoRA r=16, α=32 on the WeMM-9B tower, 600 steps of InfoNCE at temperature 0.045;
8/11 of each batch are DiDeMo moment–query pairs and 1/11 each MSR-VTT video–text, MSCOCO
text-to-image and ColPali page–query pairs, replayed so that video retrieval, image retrieval and
visual documents keep their scores while V-MRET moves.

## Evaluation

All 190 MMEB-V3 tasks with the [Tencent WeMM evaluation harness](https://github.com/Tencent/WeMM-Embedding/tree/main/mmeb_v3_eval),
with the Wianmu audio modifications for the 11 audio tasks; scored with the leaderboard's own code.
For the two TUTSound tasks, whose queries carry both an audio clip and text, the model has no joint
encoder, so the two unit embeddings are summed with equal weight and re-normalised.

MMEB-V3, leaderboard snapshot of 16 Oct 2026:

| Model | Overall | Overall-V3 (new) | Text | Audio | Agent | Image | Video | Visdoc |
|---|---|---|---|---|---|---|---|---|
| **Sentio** | **61.69** | **48.46** | **48.98** | 36.81 | **50.57** | 80.90 | **75.04** | 83.28 |
| Wianmu-5B-v0 | 60.45 | 47.50 | 48.13 | 42.93 | 48.00 | 79.73 | 71.56 | 81.99 |
| WeMM-Embedding-9B | 59.33 | 44.54 | 48.76 | — | 50.10 | 80.99 | 74.31 | 83.34 |
| Wianmu-3B-v0 | 58.16 | 44.37 | 45.30 | 41.16 | 44.11 | 78.67 | 70.79 | 80.80 |

MMEB-V2 (78 tasks) and the video block:

| Model | Overall-V2 | Image (V2) | Video | V-CLS | V-QA | V-RET | V-MRET | Visdoc |
|---|---|---|---|---|---|---|---|---|
| **Sentio** | 80.68 | 81.77 | 75.04 | 87.54 | 77.89 | 67.80 | 61.53 | 83.28 |
| WeMM-Embedding-9B | 80.58 | 81.87 | 74.31 | 87.36 | 77.69 | 67.38 | 58.46 | 83.34 |

Audio tasks (hit@1):

| Task | Sentio | Wianmu-3B-v0 | Wianmu-5B-v0 |
|---|---|---|---|
| NSynth | 76.10 | 74.10 | 60.90 |
| ESC-50 | 56.15 | 79.20 | 86.60 |
| UrbanSound8K | 55.92 | 64.05 | 66.63 |
| SpeechCommands | 89.50 | 89.80 | 89.90 |
| CREMA-D | 33.74 | 47.08 | 48.93 |
| Clotho | 8.48 | 13.61 | 16.56 |
| SoundDescs | 6.10 | 9.50 | 20.50 |
| TUTSound | 61.76 | 52.96 | 46.89 |
| TUTSound (hard) | 5.16 | 4.55 | 3.64 |
| AVE | 9.20 | 16.42 | 18.66 |
| SpeechCOCO | 2.80 | 1.50 | 13.00 |
| **A-CLS / A-RET / Audio-Overall** | **62.28 / 15.58 / 36.81** | 70.85 / 16.42 / 41.16 | 70.59 / 19.88 / 42.93 |

Sentio takes the top of the 190-task board on the strength of the 9B tower (text, agent, video and
visual documents) and is the first of a series: the audio block is where it trails the smaller
Wianmu models, and the next releases work on exactly that.

## Known limitations

- Audio is pooled into a single vector, so there is no fine timing inside a clip. Grounding and
  moment-level tasks suffer from this the most.
- Clips longer than 30 seconds get cut.
- The audio stage saw seven datasets, dominated by NSynth instrument labels. Together with the
  adaptable encoder this makes the model strong on instruments and sound events (NSynth, TUTSound)
  and weaker than the smaller Wianmu models on environmental scenes (ESC-50, UrbanSound8K), emotion
  (CREMA-D) and audio captions (Clotho, SoundDescs).
- The tower is frozen during the audio stage, so whatever WeMM-9B can't tell apart in text, Sentio
  can't either. Some label names also collide across datasets ("happy" is a spoken word in
  SpeechCommands and an emotion in RAVDESS, and both share one target vector).
- Most of the training targets are English.

## Usage

The repository holds the model card, `config.json` and `modeling_sentio.py`; the base checkpoints
(WeMM-Embedding-9B, Qwen2.5-Omni-3B) are downloaded from their own repositories, and Sentio's
trained parts (the vdr LoRA adapter, 43M, and `trainable.pt` with the audio head and the
audio-tower LoRA, 109M) are not distributed here.

```python
from modeling_sentio import Sentio
model = Sentio.from_pretrained(
    ".",
    wemm_path="path/to/WeMM-Embedding-9B",
    wemm_adapter_path="path/to/sentio/vdr_lora_adapter",
    qwen_omni_path="path/to/Qwen2.5-Omni-3B",
    trainable_path="path/to/sentio/trainable.pt",
)
audio = model.encode_audio(["dog.wav", "piano.wav"])          # (2, 4096)
text  = model.encode_text(["dog barking", "piano"])            # (2, 4096)
image = model.encode_image(["photo.jpg"])                      # (1, 4096)
scores = audio @ text.T
```

Preprocessing (128-bin mel features, 30-second head crop, mean pooling over valid frames) mirrors
the MMEB-V3 evaluation that produced the scores above.

## License

Sentio's own code, model card and trained parameters are released under Apache-2.0. The two base
models keep their own terms: WeMM-Embedding-9B is licensed by Tencent under Apache-2.0; the audio
encoder is the audio tower of Qwen2.5-Omni-3B, distributed by Alibaba Cloud under the
[Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-Omni-3B/blob/main/LICENSE), whose terms
apply to that component.

## Credits
Credits to [Wianmu-5B](https://huggingface.co/eduardo-terres2/Wianmu-5B), [Tianmu-Emb-Uni](https://huggingface.co/TianmuLab/Tianmu-Emb-Uni/tree/main) and the [WeMM-Embedding family](https://github.com/Tencent/WeMM-Embedding).

	---
	license: apache-2.0
	pipeline_tag: feature-extraction
	base_model:
	- tencent/WeMM-Embedding-9B
	- Qwen/Qwen2.5-Omni-3B
	tags:
	- audio
	- multimodal-embedding
	- audio-retrieval
	- audio-classification
	- video-moment-retrieval
	---

	# Sentio

	![Sentio architecture](resources/sentio_architecture.png)

	Sentio is WeMM-Embedding-9B with ears, and with sharper eyes for video moments.

	WeMM-Embedding-9B is the strongest open embedding model for text, images, video and visual
	documents, but it cannot hear. [Wianmu-5B](https://huggingface.co/eduardo-terres2/Wianmu-5B) gave
	the 4B member of the family an audio tower by freezing both the embedding model and a pretrained
	audio encoder and training only a small connector between them, the recipe of Tianmu-Emb-Uni.
	Sentio does the same to the 9B, with two additions: before the audio stage, the 9B tower gets a
	small LoRA fine-tune for video moment retrieval, so the vision side also moves; and during the audio
	stage the audio encoder gets a LoRA of its own, trained together with the connector. The name is
	Latin for "I perceive": one model that hears, sees and reads into one space. Total 10.20B
	parameters, of which 152M (1.5%) were trained.

	## How it works

	Two experts, one that understands audio and one that understands everything else, and a trained
	piece in the middle that moves audio into the other model's embedding space.

	The audio side is the audio tower of Qwen2.5-Omni-3B (the encoder only, 638M parameters) with a
	LoRA on its attention and MLP projections. Everything else goes through WeMM-Embedding-9B. Audio
	lands in WeMM's 4096-d space, so an audio clip can be compared against a caption, a label name, an
	image or a video directly, and nothing that was indexed with the tower has to be re-indexed.

	```
	waveform
	-> Whisper-style mel features (128 bins, up to 30 s)
	-> Qwen2.5-Omni audio tower + LoRA (r=16), masked-mean pooled over frames -> 2048-d
	-> connector: 3-layer MLP, 2048 -> 4096 -> 8192 -> 4096, LayerNorm + GELU
	-> modality adapter fusion
	-> RMSNorm projection
	-> L2 normalize -> 4096-d embedding
	```

	Two stages, three trained pieces.

	1. Stage 1, the tower. WeMM-Embedding-9B gets a LoRA (r=16, α=32, 43.3M parameters) trained with
	InfoNCE on video moment-retrieval pairs mixed with a replay of visual-document, video and image
	pairs so that nothing already strong is lost. This is the only change to the non-audio path: a
	text, image, video or document embedding from Sentio is the embedding of WeMM-9B with this LoRA
	applied. It lifts MMEB-V2 from 80.58 to 80.68 and V-MRET from 58.46 to 61.53.
	2. Stage 2, the ears. With the tower frozen (LoRA included), the audio path is trained: a LoRA on
	the audio tower (r=16, α=32, dropout 0.05, on q/k/v/out projections and both MLP layers of every
	encoder block, 11.8M parameters) together with the audio head — connector, adapter fusion,
	projection norm and prototypes, 96.8M parameters, almost all of it the connector (75.5M).

	Adapter fusion. One audio clip can be matched against very different things. For a caption you
	care about meaning, for another audio clip about acoustic detail, for a video about timing, and for
	a label about the category. So there are four small bottleneck adapters, one each for text, audio,
	video and label. A gate looks at the audio vector and mixes the four outputs with soft weights, with
	a small entropy bonus during training so the gate doesn't collapse onto one adapter.

	\| Adapter \| Input \| Down projection \| Activation \| Bottleneck \| Up projection \| Output \| Residual \| Parameters \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| text \| 4096 \| Linear 4096 → 512, no bias \| GELU \| 512 \| Linear 512 → 4096, no bias \| 4096 \| yes \| 4,194,304 \|
	\| audio \| 4096 \| Linear 4096 → 512, no bias \| GELU \| 512 \| Linear 512 → 4096, no bias \| 4096 \| yes \| 4,194,304 \|
	\| video \| 4096 \| Linear 4096 → 512, no bias \| GELU \| 512 \| Linear 512 → 4096, no bias \| 4096 \| yes \| 4,194,304 \|
	\| label \| 4096 \| Linear 4096 → 512, no bias \| GELU \| 512 \| Linear 512 → 4096, no bias \| 4096 \| yes \| 4,194,304 \|
	\| gate \| 4096 \| Linear 4096 → 1024, no bias + LayerNorm(1024) \| GELU \| 1024 \| Linear 1024 → 4, with bias \| 4 (softmax weights) \| no \| 4,200,452 \|

	Shared prototypes. 64 learnable vectors in the shared space. Each audio embedding and each target
	embedding gets a soft assignment over them, and training pushes the audio and its matching target
	to pick the same prototypes. They are only used for the loss; at inference you just use the embedding.

	Parameter count (measured from the weight files):

	\| Component \| Parameters \| Trained \|
	\|---\|---\|---\|
	\| WeMM-Embedding-9B tower \| 9,408M \| frozen \|
	\| vdr LoRA on the tower (stage 1) \| 43.3M \| yes \|
	\| Qwen2.5-Omni-3B audio tower \| 638M \| frozen \|
	\| LoRA on the audio tower (stage 2) \| 11.8M \| yes \|
	\| Connector (3-layer MLP) \| 75.5M \| yes (stage 2) \|
	\| Adapter fusion (4 adapters + gate) \| 21.0M \| yes (stage 2) \|
	\| RMSNorm projection + 64 prototypes \| 0.27M \| yes (stage 2) \|
	\| Total \| 10.20B \| 152M (1.5%) \|

	## Training

	Stage 2 (audio). The loss is symmetric in-batch InfoNCE between audio and target embeddings
	(temperature 0.07), which does most of the work. On top of it come the Tianmu prototype loss — a
	contrastive term over prototype assignments (0.03), a symmetric KL between the audio and target
	assignments (0.01) and a balance term so all prototypes get used (0.005) — and a gate-entropy bonus
	(0.01). For label datasets, batches are built so that each label appears at most once; otherwise two
	clips of "dog bark" in one batch count as negatives of each other.

	The data is 449k audio–target pairs over 37k distinct targets from seven public datasets, sampled
	in natural proportion: NSynth (instrument labels, the bulk of every epoch), SpeechCommands
	(spoken-word labels), FSD50K and RAVDESS (sound-event and emotion labels), Clotho (captions),
	Flickr8k Audio (spoken captions paired with the image they describe) and AVE (clips paired with
	their source video). Batch size 16 (smaller where a dataset has fewer distinct labels); learning
	rate 1e-5 for the head and 1e-4 for the audio-tower LoRA, constant after 500 warm-up steps;
	498,000 steps on one H200. Validation is the same in-batch InfoNCE on a held-out split every 2,500
	steps; the released checkpoint is the one with the lowest validation loss (step ~408,000).

	Stage 1 (tower). LoRA r=16, α=32 on the WeMM-9B tower, 600 steps of InfoNCE at temperature 0.045;
	8/11 of each batch are DiDeMo moment–query pairs and 1/11 each MSR-VTT video–text, MSCOCO
	text-to-image and ColPali page–query pairs, replayed so that video retrieval, image retrieval and
	visual documents keep their scores while V-MRET moves.

	## Evaluation

	All 190 MMEB-V3 tasks with the [Tencent WeMM evaluation harness](https://github.com/Tencent/WeMM-Embedding/tree/main/mmeb_v3_eval),
	with the Wianmu audio modifications for the 11 audio tasks; scored with the leaderboard's own code.
	For the two TUTSound tasks, whose queries carry both an audio clip and text, the model has no joint
	encoder, so the two unit embeddings are summed with equal weight and re-normalised.

	MMEB-V3, leaderboard snapshot of 16 Oct 2026:

	\| Model \| Overall \| Overall-V3 (new) \| Text \| Audio \| Agent \| Image \| Video \| Visdoc \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Sentio \| 61.69 \| 48.46 \| 48.98 \| 36.81 \| 50.57 \| 80.90 \| 75.04 \| 83.28 \|
	\| Wianmu-5B-v0 \| 60.45 \| 47.50 \| 48.13 \| 42.93 \| 48.00 \| 79.73 \| 71.56 \| 81.99 \|
	\| WeMM-Embedding-9B \| 59.33 \| 44.54 \| 48.76 \| — \| 50.10 \| 80.99 \| 74.31 \| 83.34 \|
	\| Wianmu-3B-v0 \| 58.16 \| 44.37 \| 45.30 \| 41.16 \| 44.11 \| 78.67 \| 70.79 \| 80.80 \|

	MMEB-V2 (78 tasks) and the video block:

	\| Model \| Overall-V2 \| Image (V2) \| Video \| V-CLS \| V-QA \| V-RET \| V-MRET \| Visdoc \|
	\|---\|---\|---\|---\|---\|---\|---\|---\|---\|
	\| Sentio \| 80.68 \| 81.77 \| 75.04 \| 87.54 \| 77.89 \| 67.80 \| 61.53 \| 83.28 \|
	\| WeMM-Embedding-9B \| 80.58 \| 81.87 \| 74.31 \| 87.36 \| 77.69 \| 67.38 \| 58.46 \| 83.34 \|

	Audio tasks (hit@1):

	\| Task \| Sentio \| Wianmu-3B-v0 \| Wianmu-5B-v0 \|
	\|---\|---\|---\|---\|
	\| NSynth \| 76.10 \| 74.10 \| 60.90 \|
	\| ESC-50 \| 56.15 \| 79.20 \| 86.60 \|
	\| UrbanSound8K \| 55.92 \| 64.05 \| 66.63 \|
	\| SpeechCommands \| 89.50 \| 89.80 \| 89.90 \|
	\| CREMA-D \| 33.74 \| 47.08 \| 48.93 \|
	\| Clotho \| 8.48 \| 13.61 \| 16.56 \|
	\| SoundDescs \| 6.10 \| 9.50 \| 20.50 \|
	\| TUTSound \| 61.76 \| 52.96 \| 46.89 \|
	\| TUTSound (hard) \| 5.16 \| 4.55 \| 3.64 \|
	\| AVE \| 9.20 \| 16.42 \| 18.66 \|
	\| SpeechCOCO \| 2.80 \| 1.50 \| 13.00 \|
	\| A-CLS / A-RET / Audio-Overall \| 62.28 / 15.58 / 36.81 \| 70.85 / 16.42 / 41.16 \| 70.59 / 19.88 / 42.93 \|

	Sentio takes the top of the 190-task board on the strength of the 9B tower (text, agent, video and
	visual documents) and is the first of a series: the audio block is where it trails the smaller
	Wianmu models, and the next releases work on exactly that.

	## Known limitations

	- Audio is pooled into a single vector, so there is no fine timing inside a clip. Grounding and
	moment-level tasks suffer from this the most.
	- Clips longer than 30 seconds get cut.
	- The audio stage saw seven datasets, dominated by NSynth instrument labels. Together with the
	adaptable encoder this makes the model strong on instruments and sound events (NSynth, TUTSound)
	and weaker than the smaller Wianmu models on environmental scenes (ESC-50, UrbanSound8K), emotion
	(CREMA-D) and audio captions (Clotho, SoundDescs).
	- The tower is frozen during the audio stage, so whatever WeMM-9B can't tell apart in text, Sentio
	can't either. Some label names also collide across datasets ("happy" is a spoken word in
	SpeechCommands and an emotion in RAVDESS, and both share one target vector).
	- Most of the training targets are English.

	## Usage

	The repository holds the model card, `config.json` and `modeling_sentio.py`; the base checkpoints
	(WeMM-Embedding-9B, Qwen2.5-Omni-3B) are downloaded from their own repositories, and Sentio's
	trained parts (the vdr LoRA adapter, 43M, and `trainable.pt` with the audio head and the
	audio-tower LoRA, 109M) are not distributed here.

	```python
	from modeling_sentio import Sentio

	model = Sentio.from_pretrained(
	".",
	wemm_path="path/to/WeMM-Embedding-9B",
	wemm_adapter_path="path/to/sentio/vdr_lora_adapter",
	qwen_omni_path="path/to/Qwen2.5-Omni-3B",
	trainable_path="path/to/sentio/trainable.pt",
	)
	audio = model.encode_audio(["dog.wav", "piano.wav"]) # (2, 4096)
	text = model.encode_text(["dog barking", "piano"]) # (2, 4096)
	image = model.encode_image(["photo.jpg"]) # (1, 4096)
	scores = audio @ text.T
	```

	Preprocessing (128-bin mel features, 30-second head crop, mean pooling over valid frames) mirrors
	the MMEB-V3 evaluation that produced the scores above.

	## License

	Sentio's own code, model card and trained parameters are released under Apache-2.0. The two base
	models keep their own terms: WeMM-Embedding-9B is licensed by Tencent under Apache-2.0; the audio
	encoder is the audio tower of Qwen2.5-Omni-3B, distributed by Alibaba Cloud under the
	[Qwen Research License](https://huggingface.co/Qwen/Qwen2.5-Omni-3B/blob/main/LICENSE), whose terms
	apply to that component.

	## Credits
	Credits to [Wianmu-5B](https://huggingface.co/eduardo-terres2/Wianmu-5B), [Tianmu-Emb-Uni](https://huggingface.co/TianmuLab/Tianmu-Emb-Uni/tree/main) and the [WeMM-Embedding family](https://github.com/Tencent/WeMM-Embedding).