README.md

---
language:
- en
- hi
license: mit
library_name: transformers
tags:
- distilbert
- onnx
- int8
- query-classification
- generic-semantic
- multilingual
- text-classification
datasets:
- addyo07/query-classification-dataset
metrics:
- accuracy
pipeline_tag: text-classification
model-index:
- name: distilbert-query-classifier
  results:
  - task:
      type: text-classification
      name: Generic vs Semantic Classification
    dataset:
      name: query-classification-dataset
      type: addyo07/query-classification-dataset
      split: test
    metrics:
    - type: accuracy
      value: 0.9839
      name: Accuracy
    - type: precision
      value: 0.9844
      name: Precision
    - type: recall
      value: 0.9834
      name: Recall
    - type: f1
      value: 0.9839
      name: F1
widget:
- text: "my name is John"
- text: "hello"
- text: "मेरा नाम रवि है"
- text: "नमस्ते"
- text: "I love spicy food"
- text: "stop"
---

# Query Sieve Classifier

**DistilBERT multilingual** fine-tuned to classify user queries as **GENERIC** or **SEMANTIC** — filtering chit-chat from durable knowledge worth storing.

## Model Description

- **Architecture**: `distilbert-base-multilingual-cased` (134M params)
- **Quantization**: INT8 dynamic (ONNX Runtime)
- **Input**: Short text queries in English or Hindi (≤10 words; longer sentences bypass to SEMANTIC)
- **Output**: Binary — GENERIC (0) or SEMANTIC (1)
- **Inference**: ONNX Runtime CPU, single-thread P99 = **16.87ms**

## Repository Structure

```
distilbert-query-classifier/
├── README.md                    # Model card (this file)
├── model/
│   ├── pytorch/                 # PyTorch safetensors (for transformers library)
│   │   ├── model.safetensors    # Full precision PyTorch weights
│   │   ├── config.json          # Model configuration
│   │   ├── tokenizer.json       # HuggingFace tokenizer
│   │   └── tokenizer_config.json
│   └── onnx/                    # ONNX INT8 quantized (for CPU inference)
│       └── model_quantized.onnx # INT8 quantized model (~130 MB)
├── scripts/                     # Python training pipeline
│   ├── config.py                # Constants and paths
│   ├── generate_dataset.py      # Synthetic data generation via Ollama
│   ├── train.py                 # Fine-tuning + ONNX export + latency benchmark
│   └── ...
```

## Usage

### Python (with transformers + ONNX Runtime)

```python
from transformers import AutoTokenizer
import onnxruntime as ort

# Load tokenizer from the pytorch folder
tokenizer = AutoTokenizer.from_pretrained(
    "addyo07/distilbert-query-classifier",
    subfolder="model/pytorch",
)

# Load ONNX model
session = ort.InferenceSession("model/onnx/model_quantized.onnx")

def classify(text: str) -> str:
    inputs = tokenizer(text, return_tensors="np", max_length=64, truncation=True, padding="max_length")
    logits = session.run(None, {
        "input_ids": inputs["input_ids"].astype(np.int64),
        "attention_mask": inputs["attention_mask"].astype(np.int64),
    })[0]
    return "SEMANTIC" if logits[0][1] > logits[0][0] else "GENERIC"
```

### Rust (with query-sieve crate)

```toml
[dependencies]
query-sieve = { git = "https://github.com/addy-47/query-sieve-rs" }
```

```rust
use query_sieve::GenericSemanticClassifier;

let classifier = GenericSemanticClassifier::load(
    "model/onnx/model_quantized.onnx",
    "model/pytorch/tokenizer.json",
)?;
let result = classifier.classify("my name is John")?;
```

### Download individual files

```bash
# ONNX model (for CPU inference)
wget https://huggingface.co/addyo07/distilbert-query-classifier/resolve/main/model/onnx/model_quantized.onnx

# PyTorch weights (for fine-tuning)
wget https://huggingface.co/addyo07/distilbert-query-classifier/resolve/main/model/pytorch/model.safetensors

# Tokenizer
wget https://huggingface.co/addyo07/distilbert-query-classifier/resolve/main/model/pytorch/tokenizer.json
```

## Performance

| Split | Accuracy |
|-------|----------|
| Test (15% holdout) | **98.39%** |

### Latency

| Mode | P50 | P99 |
|------|-----|-----|
| Multi-thread CPU | 8.39 ms | 11.81 ms |
| Single-thread CPU (intra_op_threads=1) | 14.81 ms | 16.87 ms |

## Training Data

Dataset: [addyo07/query-classification-dataset](https://huggingface.co/datasets/addyo07/query-classification-dataset)

12,044 synthetic examples generated by `llama3.1:8b`:

| Category | English | Hindi |
|----------|---------|-------|
| GENERIC | 3,003 | 3,019 |
| SEMANTIC | 3,017 | 3,005 |

The SEMANTIC category contains ~40% short standalone statements (3-7 words) to prevent the model from learning "semantic = long sentence."

## Training Scripts

The `scripts/` directory contains the full training pipeline:

1. `python scripts/generate_dataset.py --category en_semantic` — generate synthetic data via Ollama
2. `python scripts/train.py` — fine-tune DistilBERT + export ONNX INT8 + benchmark

## Rust Crate

The `query-sieve` Rust crate provides the inference runtime:

- **GitHub**: [addy-47/query-sieve-rs](https://github.com/addy-47/query-sieve-rs)
- ONNX Runtime (ort) with HuggingFace tokenizer
- Configurable single/multi-thread CPU
- >10 word bypass (long sentences auto-classify as SEMANTIC)

## License

MIT