| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8"> |
| <meta name="description" |
| content="Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention."> |
| <meta name="keywords" content="MTLA, MLLM, Grounding, Localized Attention, object detection, temporal localization"> |
| <meta name="viewport" content="width=device-width, initial-scale=1"> |
| <title>Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention</title> |
|
|
| <link href="https://fonts.googleapis.com/css?family=Google+Sans|Noto+Sans|Castoro" |
| rel="stylesheet"> |
|
|
| <link rel="stylesheet" href="./static/css/bulma.min.css"> |
| <link rel="stylesheet" href="./static/css/fontawesome.all.min.css"> |
| <link rel="stylesheet" |
| href="https://cdn.jsdelivr.net/gh/jpswalsh/academicons@1/css/academicons.min.css"> |
| <link rel="stylesheet" href="./static/css/index.css"> |
| <link rel="icon" href="./static/images/favicon.svg"> |
|
|
| <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script> |
| <script defer src="./static/js/fontawesome.all.min.js"></script> |
| </head> |
| <body> |
|
|
| <section class="hero"> |
| <div class="hero-body"> |
| <div class="container is-max-desktop"> |
| <div class="columns is-centered"> |
| <div class="column has-text-centered"> |
| <h1 class="title is-1 publication-title">Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention</h1> |
| <div class="is-size-5 publication-authors"> |
| <span class="author-block"> |
| Daniel Shalam, |
| </span> |
| <span class="author-block"> |
| Emanuel Ben Baruch, |
| </span> |
| <span class="author-block"> |
| Avi Ben Cohen, |
| </span> |
| <span class="author-block"> |
| Tal Remez |
| </span> |
| </div> |
|
|
| <div class="is-size-5 publication-authors"> |
| <span class="author-block">Amazon</span> |
| </div> |
|
|
| <div class="column has-text-centered"> |
| <div class="publication-links"> |
| |
| <span class="link-block"> |
| <a href="https://arxiv.org/pdf/2607.05978.pdf" target="_blank" |
| class="external-link button is-normal is-rounded is-dark"> |
| <span class="icon"> |
| <i class="fas fa-file-pdf"></i> |
| </span> |
| <span>Paper</span> |
| </a> |
| </span> |
| <span class="link-block"> |
| <a href="https://arxiv.org/abs/2607.05978" target="_blank" |
| class="external-link button is-normal is-rounded is-dark"> |
| <span class="icon"> |
| <i class="ai ai-arxiv"></i> |
| </span> |
| <span>arXiv</span> |
| </a> |
| </span> |
| |
| <span class="link-block"> |
| <a href="https://github.com/TalRemez/MTLA" target="_blank" |
| class="external-link button is-normal is-rounded is-dark"> |
| <span class="icon"> |
| <i class="fab fa-github"></i> |
| </span> |
| <span>Code</span> |
| </a> |
| </span> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </div> |
| </section> |
|
|
| <section class="hero teaser"> |
| <div class="container is-max-desktop"> |
| <div class="hero-body has-text-centered"> |
| <img src="https://raw.githubusercontent.com/TalRemez/MTLA/main/assets/method_pipeline.png" alt="Propose and Attend Pipeline" style="max-width: 100%; height: auto;"> |
| <h2 class="subtitle has-text-centered mt-4"> |
| Overview of the <strong>Propose and Attend</strong> framework. We propose Multi-Token Localized Attention (MTLA): a training-free, post-hoc score that measures how strongly a prediction's tokens attend to the region they claim. |
| </h2> |
| </div> |
| </div> |
| </section> |
|
|
| <section class="section"> |
| <div class="container is-max-desktop"> |
| |
| <div class="columns is-centered has-text-centered"> |
| <div class="column is-four-fifths"> |
| <h2 class="title is-3">Abstract</h2> |
| <div class="content has-text-justified"> |
| <p> |
| Multimodal large language models can emit localized predictions, bounding boxes for objects and temporal windows for video and audio events, but they hallucinate these regions prolifically. The model's own token log-probabilities are nearly uninformative: they conflate grounding quality with input ambiguity, and coordinate tokens become near-deterministic once the model commits. |
| </p> |
| <p> |
| We propose Multi-Token Localized Attention (MTLA): a training-free, post-hoc score that measures how strongly a prediction's tokens attend to the region they claim. Prior attention-based detectors, which sum attention over the entire input modality and read a single response token, are weaker special cases; we show that summing only within the claimed region and aggregating across all prediction tokens recovers a stronger grounding signal. |
| </p> |
| <p> |
| The same recipe applies almost trivially to other modalities and tasks: object detection in images and temporal localization in video and audio. Across multiple MLLM families and three modalities, MTLA improves hallucination AUROC by +7 to +38 over the best prior training-free baseline. Used as a confidence score for re-ranking, it nearly doubles the zero-shot COCO detection AP of an open-source 8B generalist (from 20.4 to 37.0), narrowing the gap to supervised detectors without any task-specific training. |
| </p> |
| </div> |
| </div> |
| </div> |
| |
| </div> |
| </section> |
|
|
| <section class="section" id="BibTeX"> |
| <div class="container is-max-desktop content"> |
| <h2 class="title">BibTeX</h2> |
| <pre><code>@article{shalam2026propose, |
| title={Propose and Attend: Training-free MLLM Grounding Confidence via Multi-Token Localized Attention}, |
| author={Shalam, Daniel and Baruch, Emanuel Ben and Cohen, Avi Ben and Remez, Tal}, |
| journal={arXiv preprint arXiv:2607.05978}, |
| year={2026} |
| }</code></pre> |
| </div> |
| </section> |
|
|
| <footer class="footer"> |
| <div class="container"> |
| <div class="columns is-centered"> |
| <div class="column is-8"> |
| <div class="content has-text-centered"> |
| <p> |
| This website is licensed under a <a rel="license" target="_blank" |
| href="http://creativecommons.org/licenses/by-sa/4.0/">Creative |
| Commons Attribution-ShareAlike 4.0 International License</a>. |
| </p> |
| <p> |
| Website source code borrowed from <a target="_blank" |
| href="https://github.com/nerfies/nerfies.github.io">Nerfies</a>. |
| </p> |
| </div> |
| </div> |
| </div> |
| </div> |
| </footer> |
|
|
| </body> |
| </html> |
|
|
