File size: 1,260 Bytes
22d2b3c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | CodeRankEmbed-flash-attn
========================
This repository is a derivative of nomic-ai/CodeRankEmbed (MIT).
It contains:
- model weights, tokenizer, and configuration copied verbatim from
nomic-ai/CodeRankEmbed (commit 3c4b60807d71f79b43f3c4363786d9493691f8b1);
- a modified modeling_hf_nomic_bert.py that adds a native flash-attention varlen
path to NomicBertAttention.forward / NomicBertModel.forward. The remainder of
that file is the upstream nomic implementation unchanged.
Upstream lineage and attribution:
- CodeRankEmbed was trained by the CoRNStack team
(Suresh et al., "CoRNStack: High-Quality Contrastive Data for Text and Code
Retrieval", 2025; https://gangiswag.github.io/cornstack/).
- The model and its trust_remote_code modeling file are published by Nomic
(nomic-ai/CodeRankEmbed) under the MIT license.
- The BERT modeling implementation is based on Tri Dao's MLPerf BERT code
(https://github.com/mlcommons/training_results_v2.0), per the upstream file header.
License: MIT. The flash-attention modification added by this repository is also
released under MIT.
NO NEW TRAINING WAS PERFORMED. The model weights are identical to
nomic-ai/CodeRankEmbed; only the attention forward code differs.
|