CodeRankEmbed-flash-attn
========================

This repository is a derivative of nomic-ai/CodeRankEmbed (MIT).

It contains:
  - model weights, tokenizer, and configuration copied verbatim from
    nomic-ai/CodeRankEmbed (commit 3c4b60807d71f79b43f3c4363786d9493691f8b1);
  - a modified modeling_hf_nomic_bert.py that adds a native flash-attention varlen
    path to NomicBertAttention.forward / NomicBertModel.forward. The remainder of
    that file is the upstream nomic implementation unchanged.

Upstream lineage and attribution:
  - CodeRankEmbed was trained by the CoRNStack team
    (Suresh et al., "CoRNStack: High-Quality Contrastive Data for Text and Code
    Retrieval", 2025; https://gangiswag.github.io/cornstack/).
  - The model and its trust_remote_code modeling file are published by Nomic
    (nomic-ai/CodeRankEmbed) under the MIT license.
  - The BERT modeling implementation is based on Tri Dao's MLPerf BERT code
    (https://github.com/mlcommons/training_results_v2.0), per the upstream file header.

License: MIT. The flash-attention modification added by this repository is also
released under MIT.

NO NEW TRAINING WAS PERFORMED. The model weights are identical to
nomic-ai/CodeRankEmbed; only the attention forward code differs.