Chopper: mmBERT

Reads the snippets your retriever found (from a database or a long conversation) and cuts out chops: exact quotes that answer the question, typically 3–5% of the text, never rewritten. Plain text in, plain text out, so the chops work with any answering model. Code: github.com/wiiiktor/chopper_demo.

folder model trained on
mmbert-base/ mmBERT-base chopper, 0.31B (recommended) 102,654 rows in one mixed run: multi-hop (MuSiQue, HotpotQA, 2WikiMultiHopQA, MuSiQue unanswerable), 8-language rerank pools, scientific papers (QASPER), conversations (WildChat, Multi-Session Chat)
mmbert-small/ mmBERT-small chopper, 0.14B the same 102,654-row mix, same recipe
mmbert-base-pl/, mmbert-small-pl/ the same two, trained further for Polish + 15,652 Polish rows (PoQuAD, PolQA) mixed with 15,935 rows of the mix
plmodernbert-base/ Polish documents: the recipe on OPI PIB's pl-ModernBERT-base, 0.15B, Apache 2.0 the whole mix and the Polish rows fitted to its Polish tokenizer, + 12,753 Polish multi-hop rows (MuSiQue translated)

Results

Multi-hop, one retrieval per question. 400 MuSiQue dev questions, 100 each at 1–4 hops, each with its own 20 snippets; Qwen3-14B (4-bit, no fine-tuning) answers from the given text only, gpt-5.6-luna judges. Correct answers; in brackets, only questions whose chops kept every hop and the answer:

Qwen3-14B answering from tokens 1-hop 2-hop 3-hop 4-hop
all 20 snippets 2,620 84 28 22 12
chops from mmBERT-base 228 83 (82) 48 (44) 45 (35) 29 (21)
chops from mmBERT-small 217 72 (67) 49 (41) 32 (22) 23 (11)
chops from Claude Opus 5.5 144 90 (90) 64 (60) 55 (40) 43 (30)

Validation (decoded at p > 0.5, gaps ≀ 5 tokens merged, runs < 4 dropped; character F1 of kept text / share of questions whose answer is in the kept text):

chopper MuSiQue HotpotQA TriviaQA QASPER Multi-Session Chat rerank holdout, 8 languages
base .56 / .55 .63 / .85 .42 / .69 .30 / .32 .68 / .59 .52
small .54 / .49 .60 / .80 .32 / .64 .26 / .28 .63 / .59 .47

On pools whose answer sentences were deleted (8 languages), it chooses SEARCH 67% of the time (mmBERT-small: 33%).

Also a reranker. The same pass scores every passage (mean p(KEEP) of its 16 most likely tokens). Ranking 16 passages per query on the 8-language rerank holdout (400 queries, never trained on):

nDCG@10 Hit@1 GPU time per query (RTX 3090)
mmBERT-base chopper, one pass 0.973 0.972 0.20 s, chops included
bge-reranker-v2-m3 0.984 0.985 0.20 s
Qwen3-Reranker-0.6B 0.977 0.972 0.77 s
Qwen3-Reranker-4B 0.986 0.990 3.3 s

Never trained to rank, it is within about a point of dedicated rerankers at the speed of the fastest, so a pipeline can drop its reranker (pip install chopper-rag β‰₯ 0.1.4: Chopper.rerank(), or mode="rerank" in the LangChain / LlamaIndex / Haystack components). Caveats: DETAILS.md.

Polish

Qwen3-14B answering 856 Polish questions it was not trained on (356 PoQuAD dev, 500 PolQA test), from all 16 snippets or from the chops; % correct, judged by gpt-5.6-luna:

Qwen3-14B answering from tokens PoQuAD PolQA all
all 16 snippets 3,856 72.5 94.2 85.2
chops from mmBERT-base 231 63.2 82.6 74.5
chops from mmBERT-small 220 68.5 71.6 70.3
chops from mmBERT-base-pl 255 77.2 91.6 85.6
chops from mmBERT-small-pl 266 75.6 90.2 84.1

With the Polish chopper, Qwen3-14B answers as well as from all the snippets with 15Γ— fewer tokens. The Polish training leaves the other tests about where they were (MuSiQue hop study, HotpotQA, TriviaQA, QASPER, conversations, rerank holdout within a few points); use the -pl folders when your documents are Polish.

pl-ModernBERT-base chopper (plmodernbert-base/, suggested by the OPI PIB AI Lab): the same recipe on OPI PIB's Polish encoder, plus a Polish multi-hop stage. On the same 856 questions 86.2% with 240 tokens; on 100 Polish two-hop questions Qwen3-14B answers 56% from its chops vs 43% with mmBERT-base-pl and 33% from all 20 paragraphs. It is the Polish default in pip install chopper-rag β‰₯ 0.1.9 (variant="base-pl"); for mixed-language text use mmbert-base-pl/ (variant="mmbert-base-pl"; pl-ModernBERT's tokenizer is Polish-only). Details in its folder README. A chopper on polish-roberta-base-8k is queued.

Input and output

Input: one string, up to 8,192 tokens: Question: <question> [SEP] <passage 1> [SEP] … [SEP] <passage N>.

Output:

  • per token, p(KEEP) from keep.* in heads.safetensors. Keep tokens with p > 0.5 (the threshold is in chopper_config.json), merge runs separated by ≀ 5 tokens, drop runs shorter than 4 tokens.
  • one ANSWER / SEARCH decision from decision.* on the first token (0 = SEARCH, 1 = ANSWER).

Give the answering model only the chops:

Notes:
- <fragment 1>
- <fragment 2>

Question: <question>
Answer:

Minimal use

import json, torch
from safetensors.torch import load_file
from transformers import AutoModel, AutoTokenizer
from huggingface_hub import snapshot_download

d = snapshot_download("wiiiktor/chopper_mmbert", allow_patterns=["mmbert-base/*"]) + "/mmbert-base"
tok, enc = AutoTokenizer.from_pretrained(f"{d}/encoder"), AutoModel.from_pretrained(f"{d}/encoder").eval()
H, thr = load_file(f"{d}/heads.safetensors"), json.load(open(f"{d}/chopper_config.json"))["decode"]["pred_thr"]
text = "Question: " + question + "".join(f" {tok.sep_token} {p}" for p in passages)
x = tok(text, return_tensors="pt", return_offsets_mapping=True, truncation=True, max_length=8192)
off = x.pop("offset_mapping")[0]
with torch.no_grad():
    h = enc(**x).last_hidden_state[0].float()
p_keep = torch.sigmoid(h @ H["keep.weight"].float().T + H["keep.bias"].float())[:, 0]
# kept characters of `text`: off[i] for every token i with p_keep[i] > thr (then merge gaps ≀ 5, drop runs < 4)

Labels from commercial APIs. Claude Opus 5.5 (Anthropic) wrote the MuSiQue span labels used in training. They were used for an allowed purpose: training a non-competing model, a small information-extraction and summarization tool (the chopper), not a general-purpose chat or text-generation model. gpt-5.6-luna and GPT-6 Luna (OpenAI) were used only as YES/NO relevance filters for the rerank rows and as answer judges in evaluations; the resulting model is likewise an extraction tool that does not compete with OpenAI's products.

A complete retrieve β†’ chop β†’ answer loop over your own documents: operator-for-llm-with-mmbert.py in the GitHub repo.

License

MIT: commercial use allowed (weights and code). For comparison, Naver's Provence and XProvence context pruners are CC BY-NC-ND 4.0 (non-commercial, no derivatives).

Downloads last month
169
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for wiiiktor/chopper_mmbert

Quantized
(1)
this model

Dataset used to train wiiiktor/chopper_mmbert

Spaces using wiiiktor/chopper_mmbert 2