raphaelsty commited on
Commit
562ebc3
·
verified ·
1 Parent(s): 15e2f71

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +46 -0
README.md ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: colbert-onnx
3
+ tags:
4
+ - colbert
5
+ - onnx
6
+ - sentence-transformers
7
+ - feature-extraction
8
+ license: mit
9
+ ---
10
+
11
+ # model-test
12
+
13
+ ONNX export of [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m) for fast CPU inference.
14
+
15
+ ## Model Details
16
+
17
+ - **Source Model**: [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m)
18
+ - **Embedding Dimension**: 64
19
+ - **Format**: ONNX (FP32 + INT8)
20
+
21
+ ## Files
22
+
23
+ | File | Description |
24
+ |------|-------------|
25
+ | `model.onnx` | FP32 ONNX model |
26
+ | `model_int8.onnx` | INT8 quantized model (faster) |
27
+ | `tokenizer.json` | Tokenizer configuration |
28
+ | `config_sentence_transformers.json` | Model configuration |
29
+
30
+ ## Usage with colbert-onnx (Rust)
31
+
32
+ ```rust
33
+ use colbert_onnx::Colbert;
34
+
35
+ let mut model = Colbert::from_pretrained("path/to/model")?;
36
+ let embeddings = model.encode_documents(&["Hello world"])?;
37
+ ```
38
+
39
+ ## Export Tool
40
+
41
+ This model was exported using [pylate-onnx-export](https://github.com/lightonai/next-plaid/tree/main/onnx/python):
42
+
43
+ ```bash
44
+ pip install "pylate-onnx-export @ git+https://github.com/lightonai/next-plaid.git#subdirectory=onnx/python"
45
+ pylate-onnx-export mixedbread-ai/mxbai-edge-colbert-v0-32m --push-to-hub raphaelsty/model-test
46
+ ```