Chopper: mmBERT
Reads the snippets your retriever found (from a database or a long conversation) and cuts out chops: exact quotes that answer the question, typically 3β5% of the text, never rewritten. Plain text in, plain text out, so the chops work with any answering model. Code: github.com/wiiiktor/chopper_demo.
| folder | model | trained on |
|---|---|---|
mmbert-base/ |
mmBERT-base chopper, 0.31B (recommended) | 102,654 rows in one mixed run: multi-hop (MuSiQue, HotpotQA, 2WikiMultiHopQA, MuSiQue unanswerable), 8-language rerank pools, scientific papers (QASPER), conversations (WildChat, Multi-Session Chat) |
mmbert-small/ |
mmBERT-small chopper, 0.14B | the same 102,654-row mix, same recipe |
mmbert-base-pl/, mmbert-small-pl/ |
the same two, trained further for Polish | + 15,652 Polish rows (PoQuAD, PolQA) mixed with 15,935 rows of the mix |
plmodernbert-base/ |
Polish documents: the recipe on OPI PIB's pl-ModernBERT-base, 0.15B, Apache 2.0 | the whole mix and the Polish rows fitted to its Polish tokenizer, + 12,753 Polish multi-hop rows (MuSiQue translated) |
Results
Multi-hop, one retrieval per question. 400 MuSiQue dev questions, 100 each at 1β4 hops, each with its own 20 snippets; Qwen3-14B (4-bit, no fine-tuning) answers from the given text only, gpt-5.6-luna judges. Correct answers; in brackets, only questions whose chops kept every hop and the answer:
| Qwen3-14B answering from | tokens | 1-hop | 2-hop | 3-hop | 4-hop |
|---|---|---|---|---|---|
| all 20 snippets | 2,620 | 84 | 28 | 22 | 12 |
| chops from mmBERT-base | 228 | 83 (82) | 48 (44) | 45 (35) | 29 (21) |
| chops from mmBERT-small | 217 | 72 (67) | 49 (41) | 32 (22) | 23 (11) |
| chops from Claude Opus 5.5 | 144 | 90 (90) | 64 (60) | 55 (40) | 43 (30) |
Validation (decoded at p > 0.5, gaps β€ 5 tokens merged, runs < 4 dropped; character F1 of kept text / share of questions whose answer is in the kept text):
| chopper | MuSiQue | HotpotQA | TriviaQA | QASPER | Multi-Session Chat | rerank holdout, 8 languages |
|---|---|---|---|---|---|---|
| base | .56 / .55 | .63 / .85 | .42 / .69 | .30 / .32 | .68 / .59 | .52 |
| small | .54 / .49 | .60 / .80 | .32 / .64 | .26 / .28 | .63 / .59 | .47 |
On pools whose answer sentences were deleted (8 languages), it chooses SEARCH 67% of the time (mmBERT-small: 33%).
Also a reranker. The same pass scores every passage (mean p(KEEP) of its 16 most likely tokens). Ranking 16 passages per query on the 8-language rerank holdout (400 queries, never trained on):
| nDCG@10 | Hit@1 | GPU time per query (RTX 3090) | |
|---|---|---|---|
| mmBERT-base chopper, one pass | 0.973 | 0.972 | 0.20 s, chops included |
| bge-reranker-v2-m3 | 0.984 | 0.985 | 0.20 s |
| Qwen3-Reranker-0.6B | 0.977 | 0.972 | 0.77 s |
| Qwen3-Reranker-4B | 0.986 | 0.990 | 3.3 s |
Never trained to rank, it is within about a point of dedicated rerankers at the speed of the fastest, so a pipeline can
drop its reranker (pip install chopper-rag β₯ 0.1.4: Chopper.rerank(), or mode="rerank" in the LangChain /
LlamaIndex / Haystack components). Caveats: DETAILS.md.
Polish
Qwen3-14B answering 856 Polish questions it was not trained on (356 PoQuAD dev, 500 PolQA test), from all 16 snippets or from the chops; % correct, judged by gpt-5.6-luna:
| Qwen3-14B answering from | tokens | PoQuAD | PolQA | all |
|---|---|---|---|---|
| all 16 snippets | 3,856 | 72.5 | 94.2 | 85.2 |
| chops from mmBERT-base | 231 | 63.2 | 82.6 | 74.5 |
| chops from mmBERT-small | 220 | 68.5 | 71.6 | 70.3 |
| chops from mmBERT-base-pl | 255 | 77.2 | 91.6 | 85.6 |
| chops from mmBERT-small-pl | 266 | 75.6 | 90.2 | 84.1 |
With the Polish chopper, Qwen3-14B answers as well as from all the snippets with 15Γ fewer tokens. The Polish training
leaves the other tests about where they were (MuSiQue hop study, HotpotQA, TriviaQA, QASPER, conversations, rerank holdout
within a few points); use the -pl folders when your documents are Polish.
pl-ModernBERT-base chopper (plmodernbert-base/, suggested by the OPI PIB AI Lab): the same recipe on OPI PIB's Polish encoder, plus a Polish multi-hop stage. On the same 856 questions 86.2% with 240 tokens; on 100 Polish two-hop questions Qwen3-14B answers 56% from its chops vs 43% with mmBERT-base-pl and 33% from all 20 paragraphs. It is the Polish default in pip install chopper-rag β₯ 0.1.9 (variant="base-pl"); for mixed-language text use mmbert-base-pl/ (variant="mmbert-base-pl"; pl-ModernBERT's tokenizer is Polish-only). Details in its folder README. A chopper on polish-roberta-base-8k is queued.
Input and output
Input: one string, up to 8,192 tokens: Question: <question> [SEP] <passage 1> [SEP] β¦ [SEP] <passage N>.
Output:
- per token, p(KEEP) from
keep.*inheads.safetensors. Keep tokens with p > 0.5 (the threshold is inchopper_config.json), merge runs separated by β€ 5 tokens, drop runs shorter than 4 tokens. - one ANSWER / SEARCH decision from
decision.*on the first token (0 = SEARCH, 1 = ANSWER).
Give the answering model only the chops:
Notes:
- <fragment 1>
- <fragment 2>
Question: <question>
Answer:
Minimal use
import json, torch
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer
from huggingface_hub import snapshot_download
d = snapshot_download("wiiiktor/chopper_mmbert", allow_patterns=["mmbert-base/*"]) + "/mmbert-base"
tok, enc = AutoTokenizer.from_pretrained(f"{d}/encoder"), AutoModel.from_pretrained(f"{d}/encoder").eval()
H, thr = load_file(f"{d}/heads.safetensors"), json.load(open(f"{d}/chopper_config.json"))["decode"]["pred_thr"]
text = "Question: " + question + "".join(f" {tok.sep_token} {p}" for p in passages)
x = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=8192)
off = x.pop("offset_mapping")[0]
with torch.no_grad():
h = enc(**x).last_hidden_state[0].float()
p_keep = torch.sigmoid(h @ H["keep.weight"].float().T + H["keep.bias"].float())[:, 0]
# kept characters of `text`: off[i] for every token i with p_keep[i] > thr (then merge gaps β€ 5, drop runs < 4)
Labels from commercial APIs. Claude Opus 5.5 (Anthropic) wrote the MuSiQue span labels used in training. They were used for an allowed purpose: training a non-competing model, a small information-extraction and summarization tool (the chopper), not a general-purpose chat or text-generation model. gpt-5.6-luna and GPT-6 Luna (OpenAI) were used only as YES/NO relevance filters for the rerank rows and as answer judges in evaluations; the resulting model is likewise an extraction tool that does not compete with OpenAI's products.
A complete retrieve β chop β answer loop over your own documents: operator-for-llm-with-mmbert.py in the GitHub repo.
License
MIT: commercial use allowed (weights and code). For comparison, Naver's Provence and XProvence context pruners are CC BY-NC-ND 4.0 (non-commercial, no derivatives).
- Downloads last month
- 169
Model tree for wiiiktor/chopper_mmbert
Base model
OPI-PIB/pl-ModernBERT-base