chengpingan's picture
Publish the research topic explorer
5c5dd05 verified
Raw History Blame Contribute Delete
5.57 kB
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="description" content="How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured.">
<title>Long-Context Preference Optimization</title>
<style>:root { --accent: #534689; }
:root { color-scheme: light; background: #f3f4f7; color: #252d39; font: 100%/1.65 system-ui, sans-serif; }
* { box-sizing: border-box; }
body { margin: 0; }
main { width: min(55rem, calc(100% - 2rem)); margin: 0 auto; padding: 2.75rem 0 3.5rem; }
header { border-top: .3rem solid var(--accent); padding: 1.35rem 0 1.1rem; }
h1 { max-width: 21ch; margin: 0 0 .65rem; font: bold clamp(2.2rem, 6vw, 3.65rem)/1.08 Georgia, serif; letter-spacing: -.035em; }
h2 { margin: 0 0 .4rem; font-size: 1.16rem; }
p { max-width: 72ch; margin: .55rem 0; }
nav, .buttons { display: flex; flex-wrap: wrap; gap: .4rem .95rem; margin: .8rem 0; }
a { color: var(--accent); text-underline-offset: .19em; }
a:focus-visible, summary:focus-visible, button:focus-visible, select:focus-visible, textarea:focus-visible { outline: 3px solid var(--accent); outline-offset: 3px; }
details, .reference { background: #fff; border: 1px solid #c4cbd4; border-radius: .65rem; margin-bottom: .65rem; }
summary { color: var(--accent); cursor: pointer; padding: .8rem 1rem; font-weight: 750; }
details[open] summary { background: #ebeff5; border-radius: .6rem .6rem 0 0; }
.body, .reference { padding: .65rem 1rem 1rem; }
.table-wrap { overflow-x: auto; }
table { width: 100%; border-collapse: collapse; font-variant-numeric: tabular-nums; }
caption, .note { text-align: left; color: #4f5968; font-size: .91rem; margin: .4rem 0; }
th, td { padding: .42rem .55rem; border-bottom: 1px solid #d8dde4; text-align: right; }
th:first-child { text-align: left; }
thead { background: #f3f5f8; font-size: .89rem; }
.reference { margin-top: 1.2rem; }
.widget { padding: .65rem .85rem; margin: .75rem 0; border-radius: .4rem; background: #f0f3f8; }
label { display: block; font-weight: 700; margin-bottom: .25rem; }
textarea { width: 100%; min-width: 0; resize: vertical; }
button, select, textarea { font: inherit; color: inherit; border: 1px solid #8794a4; background: white; border-radius: .28rem; padding: .3rem .55rem; }
button { cursor: pointer; }
.selected-row { background: #e4f1e8; box-shadow: inset .22rem 0 var(--accent); }
@media (max-width: 28rem) {
main { padding-top: 1.4rem; }
.body, .reference, summary { padding-inline: .7rem; }
th, td { padding-inline: .32rem; }
}
</style>
</head>
<body>
<main>
<header>
<h1>Long-Context Preference Optimization</h1>
<p>How can a model generate useful preference pairs when the evidence is scattered through a long document? This walkthrough follows chunk selection and shows what the LongMab ablations measured.</p>
<nav aria-label="Paper resources"><a href="https://aclanthology.org/2026.acl-long.161/">Paper</a>
<a href="https://arxiv.org/abs/2508.13993v2">arXiv v2</a>
<a href="https://aclanthology.org/2026.acl-long.161.bib">BibTeX</a>
<a href="https://github.com/NEUIR/LongMab-PO">Code</a></nav>
</header>
<details open>
<summary>Where do the bandit arms come from?</summary>
<div class="body"><p>Split the source document into chunks. LongMab initializes chunk utility with a probe based on embedding similarity, then uses an upper confidence bound to balance promising chunks against less-tested ones. The language model generates candidate answers from selected chunks, and feedback from those answers updates the bandit scores.</p></div>
</details><details>
<summary>How much did the bandit contribute?</summary>
<div class="body"><p>The same Llama-3.1-8B-Instruct backbone appears in the full and ablated runs. These are averages over five long-context question answering datasets in Table 2.</p><div class="table-wrap"><table><caption>Long-context QA averages for Llama-3.1-8B-Instruct</caption><thead><tr><th scope="col">Setting</th><th scope="col">Substring exact match</th><th scope="col">Token F1</th></tr></thead><tbody><tr><th scope="row">Direct preference optimization</th><td>35.90</td><td>45.09</td></tr><tr><th scope="row">LongMab without probe initialization</th><td>37.01</td><td>45.48</td></tr><tr><th scope="row">LongMab without upper confidence bound</th><td>35.78</td><td>44.19</td></tr><tr><th scope="row">Full LongMab</th><td>40.95</td><td>47.43</td></tr></tbody></table></div></div>
</details><details>
<summary>Where does this method fit in a long-context pipeline?</summary>
<div class="body"><p>The bandit runs when collecting candidate responses for preference training. Chosen and rejected responses then become DPO examples. In comparisons with inference-time retrieval or truncation, separate the training cost, the chunk budget during data construction and the evaluation-time context.</p></div>
</details>
<section class="reference" aria-label="Citation">
<h2>Paper and citation</h2>
<p><a href="https://aclanthology.org/2026.acl-long.161/">Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization</a><br>Shaohua Duan, Pengcheng Huang, Xinze Li, and colleagues. ACL 2026.</p>
<p><a href="https://aclanthology.org/2026.acl-long.161.pdf#page=7">Ablations in Tables 2 and 3</a></p>
</section>
</main>
</body>
</html>