README.md

---
library_name: colbert-onnx
tags:
  - colbert
  - onnx
  - sentence-transformers
  - feature-extraction
license: mit
---

# model-test

ONNX export of [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m) for fast CPU inference.

## Model Details

- **Source Model**: [mixedbread-ai/mxbai-edge-colbert-v0-32m](https://huggingface.co/mixedbread-ai/mxbai-edge-colbert-v0-32m)
- **Embedding Dimension**: 64
- **Format**: ONNX (FP32 + INT8)

## Files

| File | Description |
|------|-------------|
| `model.onnx` | FP32 ONNX model |
| `model_int8.onnx` | INT8 quantized model (faster) |
| `tokenizer.json` | Tokenizer configuration |
| `config_sentence_transformers.json` | Model configuration |

## Usage with colbert-onnx (Rust)

```rust
use colbert_onnx::Colbert;

let mut model = Colbert::from_pretrained("path/to/model")?;
let embeddings = model.encode_documents(&["Hello world"])?;
```

## Export Tool

This model was exported using [pylate-onnx-export](https://github.com/lightonai/next-plaid/tree/main/onnx/python):

```bash
pip install "pylate-onnx-export @ git+https://github.com/lightonai/next-plaid.git#subdirectory=onnx/python"
pylate-onnx-export mixedbread-ai/mxbai-edge-colbert-v0-32m --push-to-hub raphaelsty/model-test
```