Papers
arxiv:2606.18924

Who Wins the Conflict? Mechanistic Interpretability of Text Bias in Audio LLMs

Published on Sep 4
Authors:
,
,
,

Abstract

Text dominance in Audio LLMs arises from separate audio and text pathways converging late while the text pathway actively suppresses intact audio representations, and a training-free back-patching intervention reduces this bias by routing late-layer audio activations into earlier layers.

While Audio Large Language Models (Audio LLMs) excel at multimodal understanding, they suffer from text dominance, a bias where models favor text over acoustic evidence, potentially leading to hallucinated responses. However, the internal mechanisms underlying how these models behave when audio and textual inputs contradict each other remain unexplored. In this work, we present the first mechanistic analysis of this phenomenon by tracing the propagation of internal representations across layers. Our investigation reveals three key findings: (i) text dominance is consistently observed across models; (ii) while text and audio rely on functionally distinct pathways, they ultimately converge into a shared semantic space in late layers; and (iii) the text pathway does not erase audio information, but rather actively suppresses intact audio representations. Building on these insights, we leverage back-patching, a training-free intervention that routes late-layer audio activations back into earlier layers. This amplifies the audio representations, enabling them to overcome textual suppression. Our evaluation shows that back-patching consistently reduces text dominance, demonstrating a mechanistic route to mitigating text dominance under conflict.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2606.18924
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2606.18924 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2606.18924 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2606.18924 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.