| --- |
| license: mit |
| tags: |
| - colbert |
| - onnx |
| - zero-pytorch |
| - modernbert |
| - text-embeddings |
| pipeline_tag: feature-extraction |
| library_name: generic |
| --- |
| |
| # lateon-onnx |
|
|
| This repository hosts the optimized, single-file **ONNX representation** of the ModernBERT-backed [lightonai/LateOn](https://huggingface.co/lightonai/LateOn) model. |
|
|
| It is designed to run completely PyTorch-free and dependency-free using the [intextus-embed](https://github.com/Intextus/intextus-embed) runtime library. |
|
|
| ## Model Metadata |
| - **Backbone**: ModernBERT-base (140M parameters) |
| - **Output Dimensions**: 128-dimensional late-interaction embeddings |
| - **ONNX File Size**: 580 MB (fully self-contained, merged weights) |
| - **Case Sensitivity**: Case-sensitive (requires `do_lower_case=False`) |
|
|
| ## Usage |
|
|
| Install the `intextus-embed` runtime: |
| ```bash |
| pip install intextus-embed |
| ``` |
|
|
| Load the model automatically and run inference (set `do_lower_case=False` because LateOn is case-sensitive): |
| ```python |
| from intextus import IntextusEncoder, compute_maxsim |
| |
| # Automatically downloads and caches the model from Hugging Face |
| model = IntextusEncoder("lateon", do_lower_case=False) |
| |
| # Encode queries and documents |
| query_embeddings = model.encode_queries("What is ultra-low latency?") |
| doc_embeddings = model.encode_docs("ONNX runtime bypasses the PyTorch layer completely.") |
| |
| # Compute the MaxSim similarity score via NumPy |
| score = compute_maxsim(query_embeddings[0], doc_embeddings[0]) |
| print(f"Relevance Score (MaxSim): {score:.4f}") |
| ``` |
|
|
