Upload folder using huggingface_hub

README.md

@@ -0,0 +1,47 @@
+---
+license: other
+license_name: nvidia-open-model-license
+base_model: nvidia/nv-embedcode-7b-v1
+tags: [qhexrt, hexagon, npu, qualcomm, embedding, code, on-device, v81, snapdragon, 7b]
+library_name: qhexrt
+pipeline_tag: feature-extraction
+---
+
+# NV-EmbedCode-7B → Hexagon NPU (QHexRT, v81)
+
+On-device **code-retrieval embedding** bundle of **[nvidia/nv-embedcode-7b-v1](https://huggingface.co/nvidia/nv-embedcode-7b-v1)**
+for the **Qualcomm Hexagon v81 NPU** (SM8850), runnable with the [QHexRT](https://github.com/RunanywhereAI)
+C++ runtime. **No Python in the hot path.**
+
+`MistralBiDirectionalModel` (bidirectional Mistral-7B: hidden 4096, 32 layers, 32q/8kv, head_dim 128, plain
+rope theta 10000, **full attention**) + avg-pool + L2 → a **4096-d** embedding. The 7B encoder ships as **4
+chained W8 context parts** (`llama_embed_sharded`, the encoder-split engine; 1.6 GB each ≈ 6.4 GB, fits the
+~12 GB device).
+
+## Device validation (v81, SM8850) — device-vs-reference embedding cosine
+| input | cosine |
+|---|---|
+| "function to compute fibonacci numbers" | **0.9992** |
+| "def fib(n): ... fib(n-1)+fib(n-2)" | **0.9993** |
+| "SELECT * FROM users WHERE age > 30" | **0.9993** |
+| "binary search implementation in python" | **0.9993** |
+
+W8 7B vs the fp32 reference — **near bit-faithful** (≥ 0.9992). Code retrieval: cos(fib-query, fib-code)
+**0.6878** device vs **0.6872** reference, and **0.037** for an irrelevant SQL doc (correct ranking).
+~930 ms/embedding. (Reference computed via a host-rope replication validated bit-exact to HF's real
+`MistralDecoderLayer`, since the model's shipped custom forward is transformers-5.x-incompatible.)
+
+## What's inside (`v81/`)
+- `nvembedcode7b_enc_p{0..3}_w8.bin` — the 4 chained W8 encoder parts (graphs `nvembedcode7b_enc_p{k}_w8`).
+- `nvembedcode7b_embed_f16.bin` — token-embedding table (fp16, vocab 32000).
+- `tokenizer.json` — Mistral sentencepiece BPE tokenizer.
+- `nv-embedcode-7b.json` — the QHexRT manifest (host-op `llama_embed_sharded`, plain rope).
+
+## Run
+```bash
+adb push v81 /data/local/tmp/wq/nb14emb
+adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. QHX_EMB_QUERY_PREFIX= QHX_EMB_DOC_PREFIX= \
+  ./qhx_embed nb14emb/nv-embedcode-7b.json libQnnHtp.so libQnnSystem.so nb14emb \
+  'function to compute fibonacci numbers' 'def fib(n): return n if n<2 else fib(n-1)+fib(n-2)'"
+```
+Embeddings over **raw text** (no prefix). A QNN context binary is **arch/QAIRT-pinned** (v81, QAIRT 2.47, soc_model 87) — won't load on v79/v83.

v81/nv-embedcode-7b.json

@@ -0,0 +1,17 @@
+{
+  "schema_version": 1,
+  "_comment": "NV-EmbedCode-7B (MistralBiDirectional + avg-pool) on Hexagon v81. 7B encoder split into 4 chained W8 context parts via llama_embed_sharded. Host-rope (no Gather), PLAIN rope (Mistral theta 10000, factor 1), bidirectional. Encoder S=128, head_dim 128.",
+  "model": {"name": "nv-embedcode-7b", "family": "embedding", "dsp_arch": "v81", "tokenizer_pre": "default"},
+  "params": {"hidden": 4096, "vocab": 32000, "n_layers": 32, "head_dim": 128, "max_ctx": 128, "eos_token_id": 2},
+  "artifacts": {
+    "contexts": {
+      "enc_p0": {"bin": "nvembedcode7b_enc_p0_w8.bin"},
+      "enc_p1": {"bin": "nvembedcode7b_enc_p1_w8.bin"},
+      "enc_p2": {"bin": "nvembedcode7b_enc_p2_w8.bin"},
+      "enc_p3": {"bin": "nvembedcode7b_enc_p3_w8.bin"}
+    },
+    "embed": "nvembedcode7b_embed_f16.bin",
+    "tokenizer": "tokenizer.json"
+  },
+  "plan": {"steps": [{"host": "llama_embed_sharded", "params": {"encoder_parts": "nvembedcode7b_enc_p0_w8,nvembedcode7b_enc_p1_w8,nvembedcode7b_enc_p2_w8,nvembedcode7b_enc_p3_w8", "rope_factor": 1.0, "rope_theta": 10000.0, "neg": -50000}}]}
+}

v81/nvembedcode7b_embed_f16.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb2d0a34f6a78534d08505f9d7f32dd42499a5c160fb793c003fa0bb35cf88a4
+size 262144000

v81/nvembedcode7b_enc_p0_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96f74ba404cc6d7524cc897befd722a1a55b60f55fc4291fd9676eea3cd579a0
+size 1752801280

v81/nvembedcode7b_enc_p1_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8e5e0d1d386391e8201af40f5e459c3693a8c55460d96feec6480590641d5e9
+size 1752801280

v81/nvembedcode7b_enc_p2_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ed9dd339871801b4436d47f35a8ca03dffaa7495f0b66eab9c49a1a2d03babe7
+size 1752801280

v81/nvembedcode7b_enc_p3_w8.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1a4d5add718e9b6818ee33f90718abf7b901cd78b5242e68605e874c71b9920c
+size 1752813568