base_model:
- nomic-ai/CodeRankEmbed
base_model_relation: quantized
license: mit
library_name: candle
tags:
- code
- code-retrieval
- sentence-transformers
- nomic-bert
- f16
- candle
pipeline_tag: feature-extraction
CodeRankEmbed-f16
An f16 (half-precision) cast of
nomic-ai/CodeRankEmbed
— the 137M NomicBert bi-encoder for code retrieval — in safetensors,
for GPU inference (e.g. candle on Apple-Silicon Metal) at roughly half
the memory of the f32 base.
This repo is weights only, identical architecture: every tensor is
the base model cast f32 → f16, tensor names/shapes unchanged. Use it
exactly like the base model (same config.json, tokenizer.json, CLS
pooling, and the required query instruction prefix).
Why
The base repo ships f32 safetensors (~547 MB). On the Metal GPU the
f16 weights halve the working set and matmul bandwidth with no change
to retrieval quality, so it is the form used by
embedding-search on Apple Silicon.
Validation (f16 vs f32, CodeSearchNet Python, N=300)
Same code/corpus, dtype the only difference:
| dtype | peak RSS | MRR@10 | Recall@1 |
|---|---|---|---|
| f32 (base) | 1116 MB | 0.9573 | 0.9367 |
| f16 (this) | 570 MB | 0.9573 | 0.9367 |
cosine(f16, f32)per-document: mean 0.999998, min 0.999996- top-1 retrieval agreement f16 vs f32: 1.0000
- MRR@10 / Recall@1 deltas: 0.0000
f16 is numerically a no-op for retrieval at about half the RAM. (The absolute MRR is high because the eval uses a small 300-doc distractor pool — it is an f16-vs-f32 parity check, not a full-CodeSearchNet reproduction of the base model's published score.)
Usage
The query must use the task instruction prefix (same as the base model); code/documents get no prefix:
Represent this query for searching relevant code: <your query>
CLS-pool the last hidden state and L2-normalize; cosine similarity for ranking.
Provenance & license
Produced by a pure dtype cast (CPU, candle) of
nomic-ai/CodeRankEmbed model.safetensors; config.json and
tokenizer.json copied unchanged. Inherits the base model's MIT
license. Credit and citation belong to the original authors — see the
base model card and
the CoRNStack paper (arXiv:2412.01007).