| library_name: colbert-onnx | |
| tags: | |
| - colbert | |
| - onnx | |
| - sentence-transformers | |
| - feature-extraction | |
| license: mit | |
| # model-test | |
| ONNX export of [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m) for fast CPU inference. | |
| ## Model Details | |
| - **Source Model**: [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m) | |
| - **Embedding Dimension**: 64 | |
| - **Format**: ONNX (FP32 + INT8) | |
| ## Files | |
| | File | Description | | |
| |------|-------------| | |
| | `model.onnx` | FP32 ONNX model | | |
| | `model_int8.onnx` | INT8 quantized model (faster) | | |
| | `tokenizer.json` | Tokenizer configuration | | |
| | `config_sentence_transformers.json` | Model configuration | | |
| ## Usage with colbert-onnx (Rust) | |
| ```rust | |
| use colbert_onnx::Colbert; | |
| let mut model = Colbert::from_pretrained("path/to/model")?; | |
| let embeddings = model.encode_documents(&["Hello world"])?; | |
| ``` | |
| ## Export Tool | |
| This model was exported using [pylate-onnx-export](https://github.com/lightonai/next-plaid/tree/main/onnx/python): | |
| ```bash | |
| pip install "pylate-onnx-export @ git+https://github.com/lightonai/next-plaid.git#subdirectory=onnx/python" | |
| pylate-onnx-export mixedbread-ai/mxbai-edge-colbert-v0-32m --push-to-hub raphaelsty/model-test | |
| ``` | |