Upload CodeEmbed model weights and configuration

README.md

@@ -0,0 +1,78 @@
+---
+language:
+  - en
+  - code
+tags:
+  - python
+  - code-search
+  - code-embeddings
+  - sentence-transformers
+  - unixcoder
+  - matryoshka
+  - onnx
+license: cc-by-nc-4.0
+pipeline_tag: feature-extraction
+library_name: sentence-transformers
+metrics:
+  - mrr
+  - ndcg
+  - recall
+---
+
+# 💻 code-embed
+
+CodeEmbed is a lightweight, high-performance code embedding model (≤130M parameters) developed by **[Divakant Pandey](https://github.com/divakantpandey)**. Specialized for semantic code search, code retrieval, and repository navigation with **8,192 token context** and **Matryoshka representation learning**.
+
+## 🚀 Key Features
+
+- **Author / Maintainer**: [Divakant Pandey](https://github.com/divakantpandey)
+- **Backbone**: UniXcoder bi-encoder architecture with a 2-layer MLP projection head and unit-sphere L2 normalization.
+- **3-Stage Curriculum**:
+  1. *SimCSE Unsupervised Contrastive Learning* (uniformity on hypersphere).
+  2. *Supervised Fine-Tuning* (MNRL with BM25 lexical hard negatives).
+  3. *Matryoshka Representation Learning (MRL)* (nested dimensions: 64, 128, 256, 384, 768, 1536).
+- **Fast CPU Inference**: ~20.8 ms per code snippet on standard laptop CPU.
+- **Production-Ready**: Exportable to ONNX with dynamic axes and dynamic INT8 quantization.
+
+## 📊 Benchmark Results
+
+| Benchmark | Metric | Value | Gate Status |
+| :--- | :--- | :---: | :---: |
+| **CodeSearchNet (Python)** | **MRR@10** | **0.7514** | **PASS** (≥ 0.65) |
+| **CoSQA (Code Search QA)** | **MRR@10** | **0.7509** | **PASS** (≥ 0.50) |
+| **Hypersphere Uniformity** | **Uniformity** | **-1.9758** | **PASS** (≤ -1.0) |
+| **Single-Snippet Latency** | **CPU Latency** | **20.8 ms** | **PASS** (< 100 ms) |
+
+## 💻 Quickstart Usage
+
+### With Sentence-Transformers:
+```python
+from sentence_transformers import SentenceTransformer
+
+# Automatically downloads from Hugging Face Hub
+model = SentenceTransformer("divakantpandey/code-embed")
+
+queries = ["binary search in sorted list", "read json file"]
+snippets = [
+    "def binary_search(arr, target): ...",
+    "import json\ndef load_json(path): return json.load(open(path))"
+]
+
+query_embeddings = model.encode(queries, normalize_embeddings=True)
+code_embeddings = model.encode(snippets, normalize_embeddings=True)
+
+# Compute cosine similarity
+similarities = query_embeddings @ code_embeddings.T
+print(similarities)
+```
+
+## 📜 License & Commercial Inquiries
+
+This model and its weights are released under the **Creative Commons Attribution-NonCommercial 4.0 International (CC-BY-NC 4.0)** license.
+
+- **Non-Commercial Use:** Free for academic research, education, benchmarking, and non-commercial projects with attribution.
+- **Attribution:** Please cite **Divakant Pandey** and link to [https://github.com/divakantpandey/code-embed](https://github.com/divakantpandey/code-embed).
+- **Commercial Licensing:** To use this model in commercial applications, proprietary software, enterprise APIs, or SaaS products, please contact:
+  - **Maintainer:** Divakant Pandey
+  - **Email:** `divakant01@gmail.com`
+  - **GitHub:** [@divakantpandey](https://github.com/divakantpandey)

backbone/config.json

@@ -0,0 +1,31 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "RobertaModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 8194,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.20.1",
+  "type_vocab_size": 10,
+  "use_cache": true,
+  "vocab_size": 51416
+}

backbone/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d013b306ee92fa36c45c0ced65e3b41742c9c14855ee2c1d9cae08bf0c3029e5
+size 525761392

code_embed_config.json

@@ -0,0 +1,9 @@
+{
+  "base_model_name_or_path": "microsoft/unixcoder-base",
+  "backbone_hidden_size": 768,
+  "embedding_dim": 1536,
+  "projection_hidden_dim": 1536,
+  "pooling": "cls",
+  "max_seq_length": 8192,
+  "dropout": 0.1
+}

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0d5b573befb66459c36aa5fdb109dcb46cc92b605be87607f9011195d4a54ea8
+size 537853608

onnx/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9d0f987502073dc232cb0d5aa8c5e08e923fc4703d9d6b582cfc087f96e9d13
+size 135275097

projection_head.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cbf1e13bb707299ddc99b72622c9807e48d44a58c72cbd399e61e081abfca2e4
+size 14170669

pycode_embed_config.json

@@ -0,0 +1,9 @@
+{
+  "base_model_name_or_path": "microsoft/unixcoder-base",
+  "backbone_hidden_size": 768,
+  "embedding_dim": 1536,
+  "projection_hidden_dim": 1536,
+  "pooling": "cls",
+  "max_seq_length": 8192,
+  "dropout": 0.1
+}

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "is_local": false,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}