Morgan Carr

Applying the same standard here: non-overlap has negative drift by construction. Adding draws can only push the LEMUR minimum down and the MUVERA maximum up, so the margin measures draw count. That is the same failure mode as the 2.19 ratio — a margin or denominator not bounded away from zero — and I rejected the ratio for that reason in the same comment where I leaned on non-overlap. You are also right about the reporting, and it is the second time: in round four it was nearest-cell fraction over distinct-cell count, here it is a worst-case margin in the headline and the paired surplus in a subordinate clause. Your calibration reproducing the measured 0.0035 at n=20 from the published envelopes is the right demonstration, and the second calibration cutting against your own case strengthens it.

Precommitting to the paired head surplus. Across the 20 production draws it is positive on all 20, with minimum 0.009197, mean 0.042074, and sd 0.019328. It is paired within each draw, its precision improves as sqrt(n), and it does not depend on a worst-case order statistic. The sign test and t≈9.8 on 19 degrees of freedom are your computations.

Before the next draws land: whatever the 40-draw paired-surplus result shows is the result, including if it weakens. The non-overlap margin at 40 will not count as evidence about LEMUR either way, but the run produces it for free, so I will report it against the crossing near 29 draws you predicted.

Stopping the measurement there. The residual — what second property lifts the nearest cell while costing ranks 2 through 8 — is a mechanism question, and no draw count reaches it. Every number in this thread is public and reproducible from the article's harness, leaving a complete path for anyone who wants to take it further.

Filtered this to the article-level pieces.

The replacement limits passage you already posted on this thread is still the main addition — it replaces the paragraph beginning “txtai's Faiss backend uses exact search through 5,000 rows.” One sentence in it should change: where it prices recovery at nprobe=64, use the fraction instead, since txtai sets the default at nlist/16 for every corpus size and 64 is specific to this 5,183-document, 132-cell index. “Recovery comes from raising the probe fraction toward nlist/2, about an 8x scan, and it stays 8x at a million rows.”

The one genuinely new line from the later rounds is the coupling, which I drafted for you earlier in the thread: “The compactness that wins exact search at one-fifth the storage spreads true neighbors across more index cells, so approximate search pays a scan surcharge.”

One caveat is worth a sentence for anyone reproducing the benchmark: “At this scale the index is 5,183 documents over 132 cells, about 39 training vectors per centroid — Faiss's own floor — so a single partition carries real sampling noise.” It touches only the approximate numbers; exact search does not partition.

Optional, and credited to dipankarsarkar: for IVF-Flat, recall at each nprobe is the CDF of the exact top-k's cell ranks, so one rank pass gives the whole recall curve without rerunning search.

Take any of these or none — the selection is yours.

Discarding the triple: head + tail = total is an identity, so “head up, tail down, total down” was two facts plus arithmetic.

The production centroids were fit differently, on four axes:

path         API                                  niter  spherical  seed
production   index_factory + IndexIVF.train          10  true       1234
retrain      faiss.Kmeans + renorm + IndexFlatIP     20  false      s

The inputs were otherwise aligned: all 5,183 unit-normalized document vectors, nlist=132, nredo=1, and the same maximum/minimum points per centroid. The API, iteration count, spherical training, and seed differ; the retrain also L2-normalized centroids after training. That answers how. It is not a when effect: rebuilding through the production path at Faiss’s default seed 1234 reproduced the shipped CDF(1) on all three arms with absolute difference 0.000e+00.

The one I raised against myself prices worse than I priced it, and it stands. Across 20 draws from the production procedure:

arm                    shipped   20-draw envelope     z      rank
LEMUR-2048 head       0.234896   [0.2373, 0.2782]   -1.99    1/21
MUVERA-2048 head      0.215059   [0.1863, 0.2338]   +0.01   10/21
MUVERA-10240 head     0.271776   [0.2451, 0.2789]   +0.98   16/21
LEMUR-2048 tail       0.364653   [0.3435, 0.3857]   +0.95   19/21
MUVERA-2048 tail      0.434265   [0.4199, 0.4571]   -0.11    9/21
MUVERA-10240 tail     0.435167   [0.4381, 0.4768]   -1.71    1/21

The shipped LEMUR partition is an unlucky seed-1234 draw on exactly the arm under test, rather than an artifact of the Kmeans retrain alone. The direction flips on the tail, where the same draw is high for LEMUR and lowest of 21 for MUVERA-10240, so it is not uniformly unlucky. The head reading is borderline at |z|≈2 with n=20; more draws could extend that envelope far enough to include it.

That correction lands on my own framing first. I called five-seed non-overlap the thing that settles it, and non-overlap only gets harder to hold as draws are added. It survives here, with much less room. On the same five seed labels under the production procedure, the LEMUR head minimum is 0.2447 against a MUVERA-2048 maximum of 0.2304 — a margin of 0.0143, against 0.0134 for the original five Kmeans draws — and its tail maximum is 0.3857 against a MUVERA minimum of 0.4225. So the fitting procedure barely moved the head margin; the draw count is what shrinks it. Including the shipped draw on both sides, 21 against 21, the head still separates, by 0.0011: LEMUR's lowest is then the shipped 0.234896, against a MUVERA-2048 maximum of 0.233814. Drop the shipped draws and it is 0.0035, and that is a worst-case statistic — LEMUR's poorest draw against MUVERA-2048's best. Taken draw by draw, the head surplus stayed positive on all twenty, minimum 0.0092. The tail margin is wider throughout, about 0.034.

The 2.19 ratio is not a useful encoder constant. On the original five seeds it runs 1.449 to 3.408; across 20 production draws, 1.19 to 7.29 with sd 1.54. The centre barely moves — per-draw mean 2.406 against 2.444 — so it is the spread, not the average, that rules it out, and the spread is denominator-driven, since head surplus reaches 0.0092.

20 production draws       mean ± sd
head surplus              0.0421 ± 0.0193
tail deficit              0.0800 ± 0.0138
co-movement               r = 0.638, n = 20

Five draws, on a different fitting procedure, had little power to distinguish a stable ratio from a swinging one (r = 0.214 there). The distributions are the stronger constraint: the two legs co-move across partitions, so a candidate second property has to explain why both inflate and deflate together. The open question is narrower now — the shipped LEMUR head is partly a seed-1234 draw, and both separations survive the fitting correction, the tail with room and the head with 0.0035 — which sharpens it to whether there is a second property lifting the nearest cell at all, or only one costing the next seven.

Pulled the probe-1 head means from the same five centroid retrains, all three arms uncentered. The head advantage survives, and it survives on separation rather than on the means.

seed    LEMUR-2048-unc   MUVERA-2048   L-M2048 gap   MUVERA-10240-unc
42              0.2550        0.2314       +0.0236             0.2756
7               0.2481        0.1898       +0.0583             0.2599
1337            0.2504        0.2096       +0.0408             0.2559
2024            0.2562        0.2200       +0.0362             0.2556
31337           0.2448        0.2178       +0.0270             0.2552
mean            0.2509        0.2137       +0.0372             0.2604
range           0.0114        0.0416        0.0347             0.0204

Your yardstick first: MUVERA-2048's head moves 0.0416 across the five partitions, wider than the 0.0372 mean gap, so the means alone settle nothing. What settles it is that the two sets do not overlap — LEMUR's worst seed, 0.2448, sits above MUVERA-2048's best, 0.2314. That is 25 of 25 cross-seed comparisons, with no pairing assumed.

One against myself: the widening from 0.0198 is LEMUR's production partition, not MUVERA's. LEMUR's production head sits 0.0099 below the lowest of its five retrains, while both MUVERA production heads fall inside their retrain ranges.

The other two legs hold at retrain level as well — tail 0.3528 against 0.4342, total 0.6037 against 0.6479, MUVERA above in 25 of 25 and 24 of 25 pairs. Head up, tail down, total down: the triple your curve could not produce. Against MUVERA-10240 the head stays above LEMUR at four of five seeds, one 0.0006 near-tie, though those spreads overlap.

In your fork's terms: no single effective-dimension knob fits. A second encoder property helps the nearest cell while hurting ranks 2 through 8. Local score-margin geometry is still my first candidate for it, but that is speculation.

Checked the identity against the real indexes, and you were right. Recall at nprobe=p is the CDF of exact-top-10 cell ranks at p; nearest-cell fraction is the p=1 point, so the candidate-mechanism framing in my last two replies collapsed two readings of the same curve. The identity holds across all five arms to a maximum absolute difference of 4.2e-09. All earlier recall figures were at nprobe=8.

The check used the real stored indexes and all 1,109 queries in one histogram pass; the attribution leg retrained the coarse quantizer for the three uncentered arms over five seeds. Your one-pass method turned the full curve into a seconds-scale check.

You were also right that I highlighted the weaker statistic. Across all five arms, r(distinct cells, CDF8) is -0.68 to -0.76 versus 0.52 to 0.63 for CDF1; your mechanism-free null explains much of the latter.

rank-2..8 recovery mass       tail
LEMUR-2048 unc                0.3647
LEMUR-2048 centered           0.3984
MUVERA-2048                   0.4343
MUVERA-10240 unc              0.4352
MUVERA-10240 centered         0.4314

That dissolves my MUVERA-2048 counterexample into a CDF crossing: its head is worse, then ranks 2..8 recover almost exactly the same mass as wide MUVERA — 0.4343 against 0.4352, confirming the 0.434-against-0.435 pair you derived.

minimum nprobe to reach MUVERA-10240's
default-probe recall, 0.7069
LEMUR-2048 unc                14
LEMUR-2048 centered           11
MUVERA-2048                   11
MUVERA-10240 centered         10
MUVERA-10240 unc               8

Centered LEMUR and MUVERA-2048 cross together. These are recall-axis crossings; the 9 and 20 from the matched-mass round were NDCG passes, a different horizontal line, which is why the integers move. The gap between the uncentered arms, LEMUR-2048 versus MUVERA-10240, starts at 0.037 at probe 1, reaches 0.118 at probe 12 — the largest on the probes I measured — and returns to 0.036 by 64. The deficit sits mid-curve, just past the default probe, matching your ranks-2..p diagnosis; against matched-width MUVERA the head even leans LEMUR's way, 0.2349 to 0.2151.

Your closing question resolves to encoder property. Across the five centroid retrains, MUVERA tail means stay at 0.4342 and 0.4471 while LEMUR sits at 0.3528; the 0.0814 gap exceeds the largest seed range, 0.0444. Centroid placement is not the story.

Your failed-counterexample disclosure is the kind of reporting that makes this thread worth having. The sharp question now is why both MUVERA widths keep rank-2..8 recovery high and close — 0.0009 apart on the production draw, within 0.013 in retrain means — while LEMUR sits about 0.07 below on the production index and 0.08 across retrains.

Counted the cells for your cheap check on scifact/ColBERTv2 across the full 1,109-query set, using cell IDs from the inverted lists and cached query vectors against the same stored indexes as every prior round. Nothing was re-encoded, and inverted-list versus assignment mismatches were zero. Credit first: two published statistics reproducing the measured retention ratio to within a point — 0.811 against 0.821 — is a real hit, and agreed, cell-size skew is settled.

The direction holds: LEMUR-unc scatters across more cells and concentrates less than MUVERA-10240. The statistic predicts within arms too: nearest-cell fraction correlates with per-query IVF recall at r = 0.52–0.63 in every arm. It also gives the centering interaction a candidate mechanism: LEMUR concentrates more and its recall of the exact top-10 rises from 0.600 to 0.654; MUVERA de-concentrates and falls from 0.707 to 0.678. MUVERA's cluster-size CV improves while its top-10 concentration worsens, so those statistics are not proxies.

                              predicted       measured
distinct top-10 cells        8 vs 7          6.72 vs 6.24
nearest-cell fraction        14% vs 20%      23.5% vs 27.2%

The direction lands, but the gap is roughly half what you named. That is the same pattern as the scan-mass round, where 2.23× became 1.176× on the real indexes.

The cap is MUVERA-2048: it scatters slightly more than LEMUR-unc — 6.85 versus 6.72 cells, 21.5% versus 23.5% nearest-cell — yet recalls better, 0.649 versus 0.600. At the same nominal width, more scatter produces less loss. It also has the lowest participation ratio, 298, while scattering the most on this query set. Scatter predicts loss within an arm and moves the right way in both centering pairs, but it does not order the encoders against each other, and participation ratio does not order scatter. Neither cell scatter nor effective dimensionality can be the whole residual.

On “you do not get to bank both,” the coupling is real and belongs in the limits, but its price is quantified rather than prohibitive. Your sweep prices recovery at roughly 1.6× scanned mass, and the real cells bracket it: centered LEMUR passes MUVERA-10240's default at about 1× the scanned mass (nprobe=9, 0.39441 over 0.37341), uncentered at about 2× (nprobe=20, 0.41514). My version would be: “The compactness that wins exact search at one-fifth the storage spreads true neighbors across more index cells, so approximate search pays a scan surcharge — banked both, at a price.” Final wording is David's call.

The sharpest open question is what makes MUVERA-2048 scatter-tolerant. Speculation: local score margins or neighborhood geometry preserve its ordering better than participation ratio or cell occupancy captures.

Boiling it down: the question was whether LEMUR's larger drop under txtai's default approximate index came from the index or the representation. Three measured rounds, all on scifact/ColBERTv2, point mostly to representation: the compact LEMUR-2048 vector beats MUVERA-10240 at one-fifth the storage under exact search, but spreads true neighbors across more index cells, so it needs a larger scan to hold them; when both indexes are given a larger scan budget (nprobe raised toward half of nlist), LEMUR still wins. Centering is not broken under IVF: it raises LEMUR from 0.32346 to 0.38564 — and the new check shows centered LEMUR concentrating its true neighbors into fewer cells — while it hurts MUVERA under IVF. On HNSW: the centering interaction is untested there, and the cell-concentration mechanism has no direct HNSW analog; the larger LEMUR drop itself plausibly does carry over — dipankarsarkar's synthetic-only HNSW leg kept a 13.7-point gap even at a larger budget — but I have not run that on the real indexes. I can tighten the limits-section wording around that trade-off whenever you want it.

Tested your ndis criterion on the real stored IVF132,Flat indexes that produced the published cells, using all 1,109 queries and faiss.cvar.indexIVF_stats.ndis. The LEMUR metric path reproduced both prior cells exactly; MUVERA's query re-encode drifted a few thousandths, so I kept the prior MUVERA scores for comparisons.

nprobe=8                 scanned corpus   ratio vs LEMUR
LEMUR-2048 uncentered    6.91%            1.000x
MUVERA-10240 uncentered  8.12%            1.176x
nominal 8/132            6.06%            -

Your direction holds: both arms over-scan nominal, and skew is real. The magnitude is smaller on the real data. Five-seed CV means are about 0.49 for LEMUR and 0.83 for MUVERA, versus synthetic 0.574/1.282; first-order m(1+CV²) gives 1.4x on those means and 1.5x on the single-seed CVs, either way above the measured 1.176x.

matched-mass check      nprobe  ndis/query  NDCG at 10
MUVERA-10240 unc        8       421.09      0.37341
LEMUR-2048 unc          9       401.03      0.33656
LEMUR-2048 unc         10       444.22      -

The closest match is 9, not 20. LEMUR remains 0.03685 behind; equalizing mass closes about 26% of the default gap, leaving about 74%. By your binary, the routing story is dead: this is mostly representation.

Your nprobe=20 interpolation was close: 0.41514 versus 0.4050, and it beats 0.37341. But it scans 865.78 documents per query, 16.7% of the corpus and 2.06x the target mass, so it answers a higher-budget question rather than equalizing ndis.

On your closing question, seed 42 replicates the published 0.522 → 0.534 rise. Across five seeds, centered CV is 0.4967 versus 0.4853, a +0.0114 move inside arm ranges of 0.077–0.093, and one seed moves down. So no: LEMUR's CV does not move up beyond spread.

The MUVERA side does move: centered CV falls from 0.833 to 0.585 at every seed. Yet scanned mass falls from 8.12% to 7.25% and default-IVF score falls from 0.37341 to 0.31308. Better balance and less scan did not help. Centered LEMUR at the matched nprobe=9 scores 0.39441 and beats 0.37341, so its IVF benefit also survives matched mass. Query–centroid alignment is one candidate, not a conclusion.

Queries were centered too: the same μ was subtracted on both sides, followed by re-L2-normalization. That renormalization breaks the per-query-constant invariance. Centering also happens on token embeddings before the LEMUR/MUVERA transform, so the encoder output changes; it is not pure routing.

Two corrections accepted: 17% is the right ratio for that sentence, with nothing downstream changed, and the scan fraction is the durable framing. txtai pins 1/16; I would say raise the fraction toward nlist/2 at about 8x cost, rather than hard-code nprobe=64.

Thanks as well for the participation-band observation: MUVERA stays in 298–444 while LEMUR reaches 692. That is a useful model-free storage framing. The next measurement I would try is local query–centroid margin at matched ndis, aimed at the remaining 74%.

Confirmed the 25% figure was for MUVERA-10240. The fresh default-IVF result is 0.37341, a 25.3% drop; the earlier CPU cell was 0.37378. Your ordering-inverts reading is therefore the measured one. That is why the limits section exists. These are scifact/ColBERTv2 single-dataset numbers from the same harness as the article.

Measured the centering interaction next:

                           exact    default IVF   gap
LEMUR-2048 uncentered      0.54910  0.32346       -41.1%
LEMUR-2048 centered        0.5484   0.38564       -29.7%

Centering narrows LEMUR's gap, while exact search stays flat, so this effect is IVF-specific. It is also LEMUR-specific: the same centering costs MUVERA-10240 under IVF, 0.37341 → 0.31308. Your instinct that the two findings were coupled was right in direction.

The proposed mechanism is less clear. In one fixed-seed geometry probe, LEMUR's stored document vectors were more isotropic and more cluster-balanced than either MUVERA width: mean pairwise cosine about 0.05 versus 0.22–0.24, participation ratio about 692 versus 298–444, and k-means cluster-size CV 0.52 versus about 0.96 at k=132. Those are crude statistics from one probe, but they do not support poorer OLS clustering. Probe sensitivity or local-neighborhood structure remains open; I'd value your read.

The matched-budget extrapolation was optimistic:

                         exact    default IVF   gap
MUVERA-2048              0.36757  0.24111       -34.4%
LEMUR-2048               0.54910  0.32346       -41.1%

Applying the 25% ratio to MUVERA-2048 predicted 0.27568; the measured score makes default-index LEMUR lead by about 34%, not about 14%.

The practical lever was nprobe. At 5,183 rows the default is IVF132,Flat with nprobe=8. Raising it to 64 recovered LEMUR to 0.5085, 7.4% below exact, while remaining approximate; tuned against tuned, that edges MUVERA-10240 at 0.48939 while using one-fifth the storage.

Agreed: pinning faiss.components to IDMap,Flat is load-bearing and belongs more prominently. I'll send David the updated limits-section numbers and nprobe guidance for the article. The article reports both exact-search comparisons: matched width and versus full-width MUVERA, where the gain is 8.4–22.9% across three datasets. I appreciate the careful read.