code-daemon-reranker-v1

A cross-encoder reranker for code search. It ships with the UltraCode MCP server as TensorRT and OpenVINO engines and re-orders the top candidates of UltraCode's hybrid retriever (embeddings + BM25 + code graph).

Revision 2026-09-13 — the weights under this name were replaced. The previous revision (2026-08) was a listwise fine-tune on public CoIR pairs, and inside UltraCode's own hybrid search it scored below the untuned model it was fine-tuned from. This revision is trained on the reranking pools UltraCode itself produces and beats that base live, on ten repositories it never saw (below). Numbers published for the old revision do not describe these weights.

• ~117 M parameters — XLM-RoBERTa, 12 layers × 384 hidden, 250 k SentencePiece vocabulary.
• 2-input ONNX (input_ids, attention_mask; no token_type_ids) → one relevance logit. UltraCode reads sigmoid(logit) and blends it with the retriever's score.
• 256 tokens per (query, document) pair; the query is kept whole and the document's tail is cut.

How it was made

Warm-started from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 and fine-tuned on exactly what the reranker sees when it serves:

1. 173 open-source repositories were indexed by UltraCode — none of the ten evaluation repositories below.
2. Behavioural queries were written per code entity by an LLM that was forbidden to use the entity's own identifiers (a whole-word leak filter enforced it): "compare two hash hierarchies and return list of changed paths", not diffMerkleTrees. 86 780 were kept. The label is the entity's file, by construction.
3. For every query UltraCode's own search exported the candidate pool its retriever produced (20 candidates) and the document text it feeds the cross-encoder — including the "island" form, a file header with the matched entity's siblings and a >>> marker on the candidate, which most live documents take.
4. 128 579 training groups of one positive and up to 16 negatives from the pool's own uncertain band; listwise softmax cross-entropy, 2 epochs, learning rate 2e-5.
5. The checkpoint was selected on a held-out development set — the pools of the ten evaluation repositories, with UltraCode's ranking replayed using the new scores — and taken from the best of 17 evaluation rounds (step 5 000 of 8 036).

So the negatives are the retriever's own confusions, not a proxy retriever's, and the documents are the serving ones, not raw code snippets. That is the whole difference from the previous revision.

Evaluation — live, inside UltraCode's search

Ten held-out repositories, 6 141 natural-language → code queries, run through UltraCode's eval_search with both models compiled to the same TensorRT FP16 engine path and only the reranker changed (pool of 20 candidates, 900 characters of code per document):

	hit@1	hit@5	MRR@10	nDCG@10
mmarco-mMiniLMv2-L12-H384-v1 (untuned base)	0.1653	0.4498	0.2847	0.3308
this model	0.1671	0.4695	0.2978	0.3436
difference	+0.002	+0.020	+0.013	+0.013

Per repository:

repository	queries	nDCG@10 base	this model	Δ nDCG	Δ hit@5
ast-grep	631	0.4326	0.4625	+0.030	+0.035
jellyfin	629	0.2661	0.2993	+0.033	+0.051
yazi	571	0.2610	0.2771	+0.016	+0.023
typeorm	588	0.3966	0.4120	+0.016	+0.022
nest	615	0.4055	0.4197	+0.014	+0.021
re2	622	0.3403	0.3527	+0.012	+0.018
chunkhound	620	0.4106	0.4157	+0.005	+0.007
beads	617	0.0696	0.0743	+0.005	+0.008
tigerfs	634	0.2350	0.2373	+0.002	+0.016
serena	614	0.4912	0.4852	−0.006	−0.003

Better on 9 of 10 repositories. Every metric reproduced exactly across two independent rounds. On the same queries, a paired analysis of the offline replay gives Δ nDCG@10 +0.0147, 95 % bootstrap CI [+0.011, +0.018], significant in six repositories and significantly worse in none; the live pooled number (+0.0127) lands inside that interval.

Read it with these in mind:

• The queries are synthetic and single-positive at file level — the right answer is the file the query was written from, and a relevant neighbouring file scores zero. Absolute numbers are therefore low; the comparison between the two rows is the finding.
• beads scores low for both models because many of its labels point at files that no longer exist in the current tree.
• The gain sits in the middle of the list (hit@3 … hit@5, MRR); hit@1 barely moves.

Speed

The architecture is identical to the base, and so is the speed. Measured interleaved — the order of the two models alternates between rounds and repositories, and each keeps the minimum of two rounds — because a back-to-back comparison on a busy machine measures the machine: the same model moved between 47 and 62 ms on one repository across rounds.

	median per-repository p50	sum of p50 over the ten
base	53.8 ms	595 ms
this model	56.3 ms	598 ms

End-to-end query latency (retrieval + reranking 20 pairs), laptop RTX 5060, TensorRT FP16.

Intended use

Re-rank a candidate pool from a first-stage retriever for natural-language → code search. Feed (query, candidate) pairs, take the logit, sort descending. The score is a raw logit — compare it within a query, not against a fixed threshold. The model was trained on UltraCode's document shapes; plain (query, code snippet) pairs work, but that is not the distribution it was tuned for.

import onnxruntime as ort, numpy as np
from transformers import AutoTokenizer

tok  = AutoTokenizer.from_pretrained(".")
sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])

def rerank(query, docs, max_len=256):
    enc = tok([query] * len(docs), docs, padding=True, truncation="only_second",
              max_length=max_len, return_tensors="np", return_token_type_ids=False)
    logits = sess.run(None, {"input_ids":      enc["input_ids"].astype(np.int64),
                             "attention_mask": enc["attention_mask"].astype(np.int64)})[0]
    return sorted(zip(logits.reshape(-1).tolist(), docs), reverse=True)   # higher = more relevant

What's in this repo

• TensorRT code-daemon-reranker-v1_{win_x64,linux_x64}_trt11.0_sm_120.engine — NVIDIA RTX 50xx, FP16.
• OpenVINO 2026.4 code-daemon-reranker-v1_ov2026.4_{cpu,igpu}_fp16_b16_s256.{xml,bin} — Intel CPU / iGPU, FP16.
• Tokenizer — tokenizer.json + tokenizer_config.json (what AutoTokenizer reads) and sentencepiece.bpe.model (stock XLM-R SentencePiece; UltraCode converts its ids to the HuggingFace order itself).
• ONNX source — model.onnx + model.onnx.data, FP32.
• Raw weights — model.safetensors + config.json, the same FP32 weights in the XLMRobertaForSequenceClassification layout: AutoModelForSequenceClassification loads them with no missing or unexpected keys, and its logit matches the ONNX to 3e-6. The Apple (MLX) build is prepared from this pair.

Engines for other NVIDIA architectures and a TVM/Vulkan build of this revision are not published yet. FP16 everywhere: the mmarco-format graph under OpenVINO INT8 on the Intel iGPU hits a known access violation.

Apple Neural Engine (Core ML)

coreml_ane/embed.mlpackage/ is a Core ML multifunction package that runs this cross-encoder reranker on the Apple Neural Engine. It is a bundle, not a file — four entries (Manifest.json, shapes.json, Data/com.apple.CoreML/model.mlmodel, Data/com.apple.CoreML/weights/weight.bin) that must keep their relative paths.

One compiled function per shape, named b<batch>_s<seq>: b16 s64, b16 s128, b16 s256. Fixed shapes are not a simplification. ct.EnumeratedShapes converts and runs, and measures 233 emb/s against 2 873 on the same encoder, because the dynamic ops it injects push the graph off the Neural Engine. shapes.json lists what was compiled, so a caller can ask instead of assuming.

Weights are fp16. The package is built from the MLX safetensors beside it, so a model is ANE-ready exactly when it is MLX-ready — there is no second set of source weights. Load it with MLComputeUnits.cpuAndNeuralEngine: plain .all lets Core ML place the graph on the GPU instead, which measured 809 emb/s against the ANE's 2 873.

A shape the package does not carry is not an error — the caller is expected to fall back to the MLX graph, which takes any shape. That is what makes the fixed-shape package safe to ship alongside model_gpu_mlx*/ rather than instead of it.

License

Released under the MIT license (the warm-start base and the XLM-R backbone are MIT/Apache; the fine-tuned weights are released MIT).

Warm-start base: cross-encoder/mmarco-mMiniLMv2-L12-H384-v1 — mMARCO ← MS MARCO, whose terms are non-commercial research.

⚠️ The warm-start base derives from MS MARCO (non-commercial). Whether a fine-tuned model inherits dataset-use terms is legally unsettled; this is not legal advice. Retrain from a permissive base if strict compliance is required. The training queries of this revision were generated for this project over public repositories; no MS MARCO data was used in the fine-tune itself.

Attribution

Warm-started from cross-encoder/mmarco-mMiniLMv2-L12-H384-v1. Backbone: XLM-RoBERTa. Measured 2026-09-13; harness in the UltraCode repository (models/code-daemon-reranker/live_ab_rerank.py, compare_dev_per_repo.py).