Cut query latency ~6.8x: eval-gated reranker pool + doc-cap tuning

Profiling (per-stage timers on the live index) showed the cross-encoder
rerank of the candidate pool is ~96% of query time -- ~4.3s of the ~4.5s
total; BM25, fusion and the semantic stage round to zero. The pool size
and the per-document text cap are therefore the only latency levers that
matter.

Both are now env-sweepable (CANLEX_RERANK_POOL, CANLEX_RERANK_DOC_CHARS)
and were swept with the 159-question eval as the gate:

POOL=50 CHARS=3000 (old): Hit@1 .81 / Hit@3 .94 / Hit@5 .97 / MRR .87, ~4470 ms/q
POOL=24 CHARS=1500 : identical metrics, ~1730 ms/q
POOL=16 CHARS=1000 (new): Hit@1 .81 / Hit@3 .94 / Hit@5 .97 / MRR .88, ~660 ms/q

The most aggressive config holds the eval exactly (MRR +0.01) at 6.8x
lower latency, so it ships as the default. Not pushed below 16: the
promote-only reranker needs a meaningful pool to lift low-fusion
candidates, and tuning further would fit the eval set rather than real
robustness. The doc cap is benign in practice -- the median section is
~520 chars and sub-chunked pieces cap at 1800.

Also cuts the eval harness from ~13 min to ~2.3 min, and proportionally
lightens every query on the 2-vCPU Space. 57 tests pass. No corpus or
embedding change (embeddings.npz untouched).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

canlex/index.py

@@ -16,7 +16,14 @@ B = 0.75
 RRF_K = 60          # reciprocal-rank-fusion damping constant
 W_SEM = 2.0         # weight on the semantic retriever in the fusion (1.0 = equal; eval-tuned)
 CANDIDATES = 80     # hits each retriever contributes to the fusion
-RERANK_POOL = 50    # top fused candidates the cross-encoder rescores
+RERANK_POOL = int(os.environ.get("CANLEX_RERANK_POOL", "16"))
+                    # top fused candidates the cross-encoder rescores. Profiled
+                    # at ~96% of query latency (the pool is cross-encoded on
+                    # CPU), so this is THE latency knob; env-sweepable, and any
+                    # change must be eval-gated (a smaller pool limits which
+                    # low-fusion candidates the promote-only rerank can lift).
+                    # Swept 2026-06: 50 -> 16 (with doc cap 3000 -> 1000) held
+                    # the 159-Q eval exactly (MRR +0.01) at ~5.5x lower latency.
 MN_WEIGHT = float(os.environ.get("CANLEX_MN_WEIGHT", "0.0024"))
                     # title-match boost per unit of idf-weighted overlap between
                     # the query and a candidate's marginal note (section title)

canlex/rerank.py

@@ -1,5 +1,6 @@
 """Cross-encoder reranker over ONNX Runtime (local, key-free)."""
 import json
+import os
 
 import numpy as np
 import onnxruntime as ort
@@ -9,7 +10,14 @@ from tokenizers import Tokenizer
 RERANK_REPO = "Xenova/bge-reranker-base"
 RERANK_ONNX = "onnx/model_quantized.onnx"  # int8: ~3x faster on CPU, negligible quality loss
 MAX_TOKENS = 512
-_MAX_DOC_CHARS = 3000  # cap doc text before tokenizing (512 tokens ~ 2-3k chars)
+_MAX_DOC_CHARS = int(os.environ.get("CANLEX_RERANK_DOC_CHARS", "1000"))
+                       # cap doc text before tokenizing. Cross-encoder cost is
+                       # ~linear in tokens per doc, so halving this ~halves the
+                       # dominant query cost; env-sweepable, eval-gated (a
+                       # tighter cap scores long sections on less of their text
+                       # -- fine in practice: the median section is ~520 chars,
+                       # and sub-chunked pieces cap at 1800). Swept 2026-06:
+                       # 3000 -> 1000 (with pool 50 -> 16) held the eval exactly.
 
 
 class Reranker: