README.md

---
language:
- en
- hi
- multilingual
license: apache-2.0
library_name: transformers
pipeline_tag: text-classification
tags:
- query-classification
- intent-detection
- memory-scope
- modernbert
- onnx
- quantized
metrics:
- accuracy
- f1
model_name: Query Scope Classifier (ModernBERT-base)
---

# Multi-lingual Query Scope Classifier (`addyo07/query-scope-classifier`)

A production-grade, fast, multi-lingual single-pass sequence classifier fine-tuned from `answerdotai/ModernBERT-base` to categorize incoming user queries into 4 distinct scope categories across English, Devanagari Hindi, and Hinglish.

## 🏷️ 4-Class Taxonomy

1. **`ChitChat`** (Label `0`): Casual greetings, small talk, AI identity questions, emotional banter.
2. **`User`** (Label `1`): Personal facts, user preferences, memory updates, user profile instructions.
3. **`Domain`** (Label `2`, **Primary Default**): Code execution, math formulas, general domain task queries, technical instructions.
4. **`Temporal`** (Label `3`): Time-sensitive queries, schedules, dates, past session history, reminders.

---

## 📊 Performance & SLA Benchmarks

- **Base Architecture**: `answerdotai/ModernBERT-base` (149M parameters, RoPE, Unpadded FlashAttention-2).
- **Holdout Test Accuracy**: **96.18%** across 2,201 holdout samples.
- **Macro F1 Score**: **0.9619**
- **Calibrated Non-Default Precision**: **98.01%** at confidence threshold tau* = 0.81 (with automatic safe fallback to Domain when uncertain).
- **Quantized INT8 ONNX File Size**: **143.67 MB**

### Per-Class Recall Breakdown

| Scope Class | Recall | Precision | F1-Score |
|---|---|---|---|
| **ChitChat** | **98.00%** | **98.50%** | **0.9825** |
| **Temporal** | **97.28%** | **97.80%** | **0.9754** |
| **User** | **95.27%** | **97.73%** | **0.9648** |
| **Domain** (Default) | **94.18%** | **95.20%** | **0.9469** |

---

## 📁 Repository Structure

```
.gitattributes
README.md
model/
  onnx/
    config.json
    model_quantized.onnx         # 143.67 MB Dynamic INT8 ONNX model
  pytorch/
    config.json
    model.safetensors            # 571 MB PyTorch BFloat16 weights
    tokenizer.json
    tokenizer_config.json
scripts/                          # Full fine-tuning, dataset audit & quantization pipeline
```

---

## 💻 Python / PyTorch Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

MODEL_NAME = "addyo07/query-scope-classifier"
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, subfolder="model/pytorch")
model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME, subfolder="model/pytorch")

labels = ["ChitChat", "User", "Domain", "Temporal"]
query = "aaj sham ko mera schedule kya hai?"

inputs = tokenizer(query, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)
    pred_idx = torch.argmax(probs, dim=-1).item()

print(f"Predicted Scope: {labels[pred_idx]} (Confidence: {probs[0][pred_idx].item():.4f})")
```

---

## ⚡ ONNX Runtime Usage (Fast CPU Inference)

```python
import numpy as np
import onnxruntime as ort
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("addyo07/query-scope-classifier", subfolder="model/pytorch")
session = ort.InferenceSession("model/onnx/model_quantized.onnx", providers=["CPUExecutionProvider"])

query = "Remind me to submit the quarterly tax report tomorrow at 5pm"
inputs = tokenizer(query, return_tensors="np", max_length=64, truncation=True)

onnx_inputs = {
    "input_ids": inputs["input_ids"].astype(np.int64),
    "attention_mask": inputs["attention_mask"].astype(np.int64)
}
outputs = session.run(None, onnx_inputs)
logits = outputs[0][0]
probs = np.exp(logits) / np.sum(np.exp(logits))
pred_id = np.argmax(probs)

labels = ["ChitChat", "User", "Domain", "Temporal"]
print(f"Scope: {labels[pred_id]}, Confidence: {probs[pred_id]:.4f}")
```