sanmonga22 commited on
Commit
9b99b5b
·
verified ·
1 Parent(s): 2734f08

Sanitize public model card metadata

Browse files
Files changed (1) hide show
  1. README.md +12 -41
README.md CHANGED
@@ -1,47 +1,18 @@
1
  ---
2
- license: other
3
- license_name: nvidia-open-model-license
4
- base_model: nvidia/nv-embedcode-7b-v1
5
- tags: [qhexrt, hexagon, npu, qualcomm, embedding, code, on-device, v81, snapdragon, 7b]
6
- library_name: qhexrt
7
- pipeline_tag: feature-extraction
 
 
8
  ---
9
 
10
- # NV-EmbedCode-7B → Hexagon NPU (QHexRT, v81)
11
 
12
- On-device **code-retrieval embedding** bundle of **[nvidia/nv-embedcode-7b-v1](https://huggingface.co/nvidia/nv-embedcode-7b-v1)**
13
- for the **Qualcomm Hexagon v81 NPU** (SM8850), runnable with the [QHexRT](https://github.com/RunanywhereAI)
14
- C++ runtime. **No Python in the hot path.**
15
 
16
- `MistralBiDirectionalModel` (bidirectional Mistral-7B: hidden 4096, 32 layers, 32q/8kv, head_dim 128, plain
17
- rope theta 10000, **full attention**) + avg-pool + L2 → a **4096-d** embedding. The 7B encoder ships as **4
18
- chained W8 context parts** (`llama_embed_sharded`, the encoder-split engine; 1.6 GB each ≈ 6.4 GB, fits the
19
- ~12 GB device).
20
 
21
- ## Device validation (v81, SM8850) — device-vs-reference embedding cosine
22
- | input | cosine |
23
- |---|---|
24
- | "function to compute fibonacci numbers" | **0.9992** |
25
- | "def fib(n): ... fib(n-1)+fib(n-2)" | **0.9993** |
26
- | "SELECT * FROM users WHERE age > 30" | **0.9993** |
27
- | "binary search implementation in python" | **0.9993** |
28
-
29
- W8 7B vs the fp32 reference — **near bit-faithful** (≥ 0.9992). Code retrieval: cos(fib-query, fib-code)
30
- **0.6878** device vs **0.6872** reference, and **0.037** for an irrelevant SQL doc (correct ranking).
31
- ~930 ms/embedding. (Reference computed via a host-rope replication validated bit-exact to HF's real
32
- `MistralDecoderLayer`, since the model's shipped custom forward is transformers-5.x-incompatible.)
33
-
34
- ## What's inside (`v81/`)
35
- - `nvembedcode7b_enc_p{0..3}_w8.bin` — the 4 chained W8 encoder parts (graphs `nvembedcode7b_enc_p{k}_w8`).
36
- - `nvembedcode7b_embed_f16.bin` — token-embedding table (fp16, vocab 32000).
37
- - `tokenizer.json` — Mistral sentencepiece BPE tokenizer.
38
- - `nv-embedcode-7b.json` — the QHexRT manifest (host-op `llama_embed_sharded`, plain rope).
39
-
40
- ## Run
41
- ```bash
42
- adb push v81 /data/local/tmp/wq/nb14emb
43
- adb shell "cd /data/local/tmp/wq && LD_LIBRARY_PATH=. QHX_EMB_QUERY_PREFIX= QHX_EMB_DOC_PREFIX= \
44
- ./qhx_embed nb14emb/nv-embedcode-7b.json libQnnHtp.so libQnnSystem.so nb14emb \
45
- 'function to compute fibonacci numbers' 'def fib(n): return n if n<2 else fib(n-1)+fib(n-2)'"
46
- ```
47
- Embeddings over **raw text** (no prefix). A QNN context binary is **arch/QAIRT-pinned** (v81, QAIRT 2.47, soc_model 87) — won't load on v79/v83.
 
1
  ---
2
+ license: "other"
3
+ tags:
4
+ - "hnpu"
5
+ - "hexagon"
6
+ - "npu"
7
+ - "embedding"
8
+ base_model: "nvidia/nv-embedcode-7b-v1"
9
+ pipeline_tag: "sentence-similarity"
10
  ---
11
 
12
+ # nv embedcode 7b HNPU
13
 
14
+ Prebuilt HNPU artifacts for [nvidia/nv-embedcode-7b-v1](https://huggingface.co/nvidia/nv-embedcode-7b-v1), a public embedding model.
 
 
15
 
16
+ For model behavior, license, intended use, and limitations, see the [upstream model card](https://huggingface.co/nvidia/nv-embedcode-7b-v1).
 
 
 
17
 
18
+ Artifacts are architecture-pinned. Available artifact directories: `v81/`.