Sanitize public model card metadata
Browse files
README.md
CHANGED
|
@@ -1,47 +1,18 @@
|
|
| 1 |
---
|
| 2 |
-
license: other
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
-
#
|
| 11 |
|
| 12 |
-
|
| 13 |
-
for the **Qualcomm Hexagon v81 NPU** (SM8850), runnable with the [QHexRT](https://github.com/RunanywhereAI)
|
| 14 |
-
C++ runtime. **No Python in the hot path.**
|
| 15 |
|
| 16 |
-
|
| 17 |
-
rope theta 10000, **full attention**) + avg-pool + L2 → a **4096-d** embedding. The 7B encoder ships as **4
|
| 18 |
-
chained W8 context parts** (`llama_embed_sharded`, the encoder-split engine; 1.6 GB each ≈ 6.4 GB, fits the
|
| 19 |
-
~12 GB device).
|
| 20 |
|
| 21 |
-
|
| 22 |
-
| input | cosine |
|
| 23 |
-
|---|---|
|
| 24 |
-
| "function to compute fibonacci numbers" | **0.9992** |
|
| 25 |
-
| "def fib(n): ... fib(n-1)+fib(n-2)" | **0.9993** |
|
| 26 |
-
| "SELECT * FROM users WHERE age > 30" | **0.9993** |
|
| 27 |
-
| "binary search implementation in python" | **0.9993** |
|
| 28 |
-
|
| 29 |
-
W8 7B vs the fp32 reference — **near bit-faithful** (≥ 0.9992). Code retrieval: cos(fib-query, fib-code)
|
| 30 |
-
**0.6878** device vs **0.6872** reference, and **0.037** for an irrelevant SQL doc (correct ranking).
|
| 31 |
-
~930 ms/embedding. (Reference computed via a host-rope replication validated bit-exact to HF's real
|
| 32 |
-
`MistralDecoderLayer`, since the model's shipped custom forward is transformers-5.x-incompatible.)
|
| 33 |
-
|
| 34 |
-
## What's inside (`v81/`)
|
| 35 |
-
- `nvembedcode7b_enc_p{0..3}_w8.bin` — the 4 chained W8 encoder parts (graphs `nvembedcode7b_enc_p{k}_w8`).
|
| 36 |
-
- `nvembedcode7b_embed_f16.bin` — token-embedding table (fp16, vocab 32000).
|
| 37 |
-
- `tokenizer.json` — Mistral sentencepiece BPE tokenizer.
|
| 38 |
-
- `nv-embedcode-7b.json` — the QHexRT manifest (host-op `llama_embed_sharded`, plain rope).
|
| 39 |
-
|
| 40 |
-
## Run
|
| 41 |
-
```bash
|
| 42 |
-
adb push v81 /data/local/tmp/wq/nb14emb
|
| 43 |
-
adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. QHX_EMB_QUERY_PREFIX= QHX_EMB_DOC_PREFIX= \
|
| 44 |
-
./qhx_embed nb14emb/nv-embedcode-7b.json libQnnHtp.so libQnnSystem.so nb14emb \
|
| 45 |
-
'function to compute fibonacci numbers' 'def fib(n): return n if n<2 else fib(n-1)+fib(n-2)'"
|
| 46 |
-
```
|
| 47 |
-
Embeddings over **raw text** (no prefix). A QNN context binary is **arch/QAIRT-pinned** (v81, QAIRT 2.47, soc_model 87) — won't load on v79/v83.
|
|
|
|
| 1 |
---
|
| 2 |
+
license: "other"
|
| 3 |
+
tags:
|
| 4 |
+
- "hnpu"
|
| 5 |
+
- "hexagon"
|
| 6 |
+
- "npu"
|
| 7 |
+
- "embedding"
|
| 8 |
+
base_model: "nvidia/nv-embedcode-7b-v1"
|
| 9 |
+
pipeline_tag: "sentence-similarity"
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# nv embedcode 7b HNPU
|
| 13 |
|
| 14 |
+
Prebuilt HNPU artifacts for [nvidia/nv-embedcode-7b-v1](https://huggingface.co/nvidia/nv-embedcode-7b-v1), a public embedding model.
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
For model behavior, license, intended use, and limitations, see the [upstream model card](https://huggingface.co/nvidia/nv-embedcode-7b-v1).
|
|
|
|
|
|
|
|
|
|
| 17 |
|
| 18 |
+
Artifacts are architecture-pinned. Available artifact directories: `v81/`.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|