onebrain-ai/onebrain-rerank-v1

OneBrain Rerank v1

The first entry in the OneBrain model line — the cross-encoder reranker used by OneBrain native search (Tier-2 precision stage, CLI v3.4.7+).

v1 is not fine-tuned. It is a size/speed-optimized int8 build of BAAI/bge-reranker-v2-m3 (Apache-2.0). Future versions (v2+) may be fine-tuned for OneBrain's vault-search domain; each version ships as its own repo and is sha256-pinned by the OneBrain CLI.

Intended use

• For: reranking a small set of (query, passage) candidates returned by a first-stage retriever (BM25 / vector), producing a calibrated 0–1 relevance score. This is the Tier-2 precision stage of OneBrain native search.
• Not for: retrieval or embedding — it scores query–passage pairs, it does not produce vectors. It is a general (not domain-fine-tuned) reranker, so it is not tuned for any specialized corpus in v1.

Provenance

• Base: BAAI/bge-reranker-v2-m3 (Apache-2.0)
• ONNX export: rozgo/bge-reranker-v2-m3 (fp32, 2.27 GB)
• Quantization: dynamic int8 (QInt8 weights), onnxruntime 1.27.0, single-file output (DefaultTensorType=FLOAT for >2GB shape-inference skip)
• model_int8.onnx — 569,011,484 bytes · sha256 dd7b26f4a233732aefbe857bef026050582dc7c1bdb8aeda909080bf15b2ad88

Why int8?

OneBrain runs local-first and CPU-only (no GPU assumption; Raspberry-Pi-class hardware is the floor). A cross-encoder scores every candidate at query time, so both download size and CPU latency matter:

• Size — the fp32 ONNX export is 2.27 GB; this int8 build is 569 MB (~4× smaller). That is the difference between a reranker most users won't download and one that can ship on by default.
• CPU latency — dynamic-QInt8 integer kernels run faster on CPU. OneBrain's design target is ≤500 ms P50 for the Tier-2 stage on a warm daemon at the default candidate depth (a design target, not a published benchmark — no measured latency number is claimed here).
• Quality cost — negligible — quantization perturbs the raw logits slightly, but reranking only needs relative ordering and calibrated separation, both preserved. On the validation pairs below the sigmoid-score delta vs fp32 is ≤ 0.0046 and the pairwise ranking is identical.

int8 is not more accurate than fp32 — it is the same weights at ~¼ the size and lower CPU cost, with quality loss small enough to be irrelevant for ranking. That trade is what makes cross-encoder reranking viable on the hardware OneBrain targets.

Validation (int8 vs fp32, sigmoid(logit))

pair	fp32	int8	delta
"what is panda?" / panda passage	0.9541	0.9495	0.0046
"what is panda?" / unrelated (Eiffel Tower)	0.0000	0.0000	0.0000
"how to install rust compiler" / rustup passage	0.9848	0.9866	0.0018
"วิธีตั้งค่า daemon" / relevant Thai passage	0.9979	0.9978	0.0002
"วิธีตั้งค่า daemon" / unrelated Thai passage	0.0000	0.0000	0.0000

Pairwise ranking identical to fp32; max delta 0.0046.

Calibration (OneBrain vault golden set)

Measured with the int8 model on OneBrain's internal golden set — a ~585-note personal Obsidian vault (Thai/English mixed), 20 answerable + 10 known-no-answer queries. This is an internal evaluation on one real vault, not a public benchmark (MTEB/BEIR/MIRACL were not run).

bucket	top-hit sigmoid score
genuine relevant match	0.73 – 0.99
tangential / weak match	0.20 – 0.52
genuine no-answer query	0.003 – 0.066 (median 0.011)

The cross-encoder separates real matches from no-answer queries by ~an order of magnitude — the property a bi-encoder's overlapping cosine scores cannot provide, and the reason Tier-2 exists. OneBrain uses these measured bands: > 0.60 confident, 0.30 – 0.60 possible, < 0.30 no strong match (default gate 0.30).

Usage

Consumed automatically by the OneBrain CLI (onebrain search) — the model downloads on first reindex and is verified against the pinned sha256. Direct use: tokenize (query, passage) with the included tokenizer (max_length 512), feed input_ids + attention_mask (int64) → logits [batch, 1] (raw logit; apply sigmoid for a 0–1 relevance score). Multilingual (Thai/English first-class) is inherited from the bge-reranker-v2-m3 base.

Limitations

• Not fine-tuned (v1): a size/speed-optimized build of a general reranker, not adapted to any specific domain.
• Evaluation scope: validated on OneBrain's internal vault golden set, not on public reranking benchmarks — no MTEB/BEIR/MIRACL scores are claimed.
• 512-token input: longer passages are truncated; OneBrain chunks documents upstream so the reranker sees chunk-sized passages.
• int8 perturbation: quantization shifts logits slightly vs fp32 (≤ 0.0046 on the validation pairs) — negligible for ranking, but present.
• CPU-oriented: optimized for CPU int8 inference; users who need maximum fidelity on GPU may prefer the fp32 base model.

License

Apache-2.0, matching the base model.