CodeRankEmbed with native flash-attn varlen forward (derivative of nomic-ai/CodeRankEmbed; identical weights; flash-vs-fp32 parity cosine 0.99999, eager fallback bit-identical)
22d2b3c verified | CodeRankEmbed-flash-attn | |
| ======================== | |
| This repository is a derivative of nomic-ai/CodeRankEmbed (MIT). | |
| It contains: | |
| - model weights, tokenizer, and configuration copied verbatim from | |
| nomic-ai/CodeRankEmbed (commit 3c4b60807d71f79b43f3c4363786d9493691f8b1); | |
| - a modified modeling_hf_nomic_bert.py that adds a native flash-attention varlen | |
| path to NomicBertAttention.forward / NomicBertModel.forward. The remainder of | |
| that file is the upstream nomic implementation unchanged. | |
| Upstream lineage and attribution: | |
| - CodeRankEmbed was trained by the CoRNStack team | |
| (Suresh et al., "CoRNStack: High-Quality Contrastive Data for Text and Code | |
| Retrieval", 2025; https://gangiswag.github.io/cornstack/). | |
| - The model and its trust_remote_code modeling file are published by Nomic | |
| (nomic-ai/CodeRankEmbed) under the MIT license. | |
| - The BERT modeling implementation is based on Tri Dao's MLPerf BERT code | |
| (https://github.com/mlcommons/training_results_v2.0), per the upstream file header. | |
| License: MIT. The flash-attention modification added by this repository is also | |
| released under MIT. | |
| NO NEW TRAINING WAS PERFORMED. The model weights are identical to | |
| nomic-ai/CodeRankEmbed; only the attention forward code differs. | |