NightJar-CodeSearch-Embedding

NightJar-CodeSearch-Embedding is a 768-dimensional dense embedding model for natural-language-to-code retrieval and code-edit retrieval. It is based on Shuu12121/NightJar and fine-tuned with hard negatives and large in-batch negatives.

The model embeds a text query and a code document into the same vector space. Higher cosine similarity indicates a stronger match. It does not require query or document prefixes.

Model details

Property	Value
Architecture	ModernBERT Sentence Transformer
Parameters	Same transformer size as Shuu12121/NightJar
Embedding dimensions	768
Maximum sequence length	1,024 tokens
Pooling	CLS token
Similarity	Cosine similarity
Training objective	Cached Multiple Negatives Ranking Loss
License	Apache-2.0

Usage

Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Shuu12121/NightJar-CodeSearch-Embedding")

query = "parse a JSON string and return an error when it is invalid"
code = [
    "def parse_json(text):\n    return json.loads(text)",
    "def read_lines(path):\n    return Path(path).read_text().splitlines()",
]

query_embedding = model.encode(query, normalize_embeddings=True)
code_embeddings = model.encode(code, normalize_embeddings=True)
scores = code_embeddings @ query_embedding

for score, snippet in sorted(zip(scores, code), reverse=True):
    print(float(score), snippet)

No query:, passage:, or task-specific prefix is needed. Use the same model and encoding settings for queries and code. Normalized embeddings make the dot product equivalent to cosine similarity.

Transformers

Loading through SentenceTransformer is recommended because the repository's pooling configuration is part of the model. If the backbone is loaded directly with Transformers, use the final hidden state of the CLS token and L2-normalize the resulting vector.

Intended uses

• Semantic code search from natural-language queries
• Retrieval of code relevant to a requested edit
• Candidate generation for reranking or retrieval-augmented generation
• Code clustering, deduplication, and nearest-neighbor exploration

The model is a retriever, not a code generator or correctness verifier. A high similarity score does not guarantee that code is safe, correct, or appropriate for execution.

Supported programming languages

The code-search training mixture includes:

Bash, C, C++, C#, Dart, Go, Java, JavaScript, Kotlin, Lua, PHP, Python, Ruby, Rust, Scala, Swift, and TypeScript.

Performance may transfer to related languages, but languages not represented in training have not been systematically validated.

Training

The model was fine-tuned on 4,073,472 examples using knowledge-distilled hard negative datasets for code search and code-edit retrieval. Each example contained one positive and up to 15 explicit hard negatives; other positives in the 1,024-example logical batch also served as in-batch negatives.

The three training groups were sampled at a 1:8:8 batch ratio:

Training group	Weight	Purpose
Code-edit retrieval	1	Natural-language edit request to relevant code
Core-language code search	8	Natural-language query to code
Additional-language code search	8	Broader programming-language coverage

Each 17-batch weighted cycle was shuffled deterministically, rather than always presenting the groups in a fixed order. Languages within each group were balanced, and rows within each batch were shuffled.

Training data and decontamination

The training mixture uses the same decontaminated retrieval datasets as NightOwl-CodeEmbedding: code-search and code-comment pairs, together with commitpackft-derived code-edit pairs. All examples were constructed with one positive and 15 hard negatives mined by Qwen/Qwen3-Embedding-0.6B.

Before training, overlaps were removed between:

• The code-search/code-comment data and the CodeSearchNet test splits
• The commitpackft-derived code-edit data and the CodeEditSearchRetrieval benchmark evaluation examples

Although MTEB names the CodeEditSearchRetrieval evaluation split train, those evaluated examples were not included in this model's fine-tuning data.

Main hyperparameters

Hyperparameter	Value
Epochs	1
Logical batch size	1,024
Cached-loss mini-batch size	64
Learning rate	6e-5
Warmup ratio	0.0
Scheduler	Cosine
Weight decay	0.01
MNRL scale	100.0
Precision	bfloat16
Gradient accumulation	1
Hard negatives per example	15

Evaluation

MTEB CodeSearchNetRetrieval

The model was evaluated with MTEB 2.5.1 on the CodeSearchNetRetrieval test sets. The benchmark's main_score is nDCG@10. The reported average is an unweighted macro average over all six languages.

Go	Java	JavaScript	PHP	Python	Ruby	Average
0.9656	0.9278	0.8236	0.8915	0.9410	0.8663	0.9026

MTEB CodeEditSearchRetrieval

Code-edit retrieval was evaluated with MTEB 2.5.1. The task's main_score is nDCG@10. The macro average across the 13 language subsets is 0.7382.

Language	nDCG@10	Language	nDCG@10
Python	0.7711	JavaScript	0.7386
TypeScript	0.7759	Go	0.7716
Ruby	0.7723	Java	0.7365
PHP	0.7162	C	0.6687
C++	0.7105	Rust	0.6900
Swift	0.7470	Scala	0.7859
Shell	0.7119	Macro average	0.7382

CodeEditSearchRetrieval does not provide a standard test split in MTEB, so its official train split is used for evaluation. The evaluated examples were removed from the fine-tuning data and were not used to train this model. The score therefore measures in-domain retrieval on held-out benchmark examples; it is not training-set performance or a strictly zero-shot result.

In-training validation

The following results use cosine retrieval on 1,000 validation examples per CodeSearchNet language. Each query has one relevant document, so Accuracy@k and Recall@k are equivalent in this setup. The macro average is an unweighted mean over the six evaluated languages.

Language	Accuracy@1	Accuracy@10	nDCG@10	MRR@10
Go	0.864	0.978	0.9270	0.9101
Java	0.722	0.935	0.8389	0.8070
JavaScript	0.745	0.891	0.8187	0.7954
PHP	0.753	0.819	0.7921	0.7828
Python	0.857	0.983	0.9278	0.9092
Ruby	0.824	0.944	0.8896	0.8716
Macro average	0.7942	0.9250	0.8657	0.8460

These numbers come from an in-training validation setup and should not be treated as directly comparable to results produced with a different corpus, candidate pool, preprocessing pipeline, or benchmark implementation.

License

The model weights and original code in this repository are released under the Apache License 2.0.

The author's code-search training dataset was constructed from repositories licensed under MIT, Apache-2.0, BSD-2-Clause, BSD-3-Clause, Unlicense, CC0-1.0, and ISC.

The Apache-2.0 license for this model does not supersede any third-party rights or license obligations that may apply to source materials used during training. Users are responsible for complying with applicable licenses when redistributing or reusing original source code obtained independently of the model.

Limitations

• Evaluation above covers six CodeSearchNet languages, not every training language.
• Inputs longer than 1,024 tokens are truncated.
• The training data may contain public-code biases, duplicated patterns, or insecure implementations inherited from its sources.