atekrugis/modernbert-intent-router-onnx-quantized

ModernBERT Intent Router - Quantized ONNX

Model Description

This model is an optimized version of atekrugis/intent-router-v11-modernbert. It uses a ModernBERT transformer body for generating embeddings and an XGBoost forest head for intent classification.

Quantization Details

• Format: ONNX
• Optimization Strategy: Dynamic Int8 Quantization
• Quantization Tool: onnxruntime.quantization

Results

• Original Size: 568.93 MB
• Quantized Size: 142.96 MB
• Size Reduction: 74.9%

Baseline Performance (Pre-Quantization)

• Math Intent: 99.7%
• Coding Intent: 99.9%
• Weather (Agentic) Intent: 99.4%

Usage

This model is designed for high-efficiency CPU inference using ONNX Runtime with significantly reduced memory footprint.