| # 🔍 LLM Query Classifier — General vs Real-Time |
|
|
| A fine-tuned NLP model that classifies whether a user query requires a **static LLM response** or a **real-time data source**. Built on top of `MiniLM-L6` using Hugging Face Transformers. |
|
|
| --- |
|
|
| ## 💡 Why This Exists |
|
|
| Modern AI assistants face a core routing problem: |
|
|
| - *"What is the capital of France?"* → A static LLM can answer this perfectly. |
| - *"What is the price of Bitcoin right now?"* → A static LLM will hallucinate or give outdated info. |
|
|
| Without a classifier, you either always call an expensive real-time API (slow + costly), or you let the LLM guess (unreliable). This model solves that by routing queries intelligently **before** any expensive call is made. |
|
|
| --- |
|
|
| ## 🧠 Model & Approach |
|
|
| | Detail | Value | |
| |---|---| |
| | Base Model | `nreimers/MiniLM-L6-H384-uncased` | |
| | Task | Binary Text Classification | |
| | Framework | Hugging Face Transformers + PyTorch | |
| | Training Set | ~260 labeled queries | |
| | Validation Set | ~63 queries (20% split) | |
| | Classes | `0 = general`, `1 = realtime` | |
|
|
| --- |
|
|
| ## 📊 Dataset |
|
|
| 323 manually curated queries split across two classes: |
|
|
| - **General (0):** 145 examples — facts, definitions, history, science, explanations |
| - **Real-time (1):** 178 examples — current events, prices, weather, live scores, breaking news |
|
|
| **Sample queries:** |
|
|
| | Query | Label | |
| |---|---| |
| | "Explain the theory of relativity" | general | |
| | "Who wrote Pride and Prejudice?" | general | |
| | "What is the current price of Bitcoin?" | realtime | |
| | "Latest news on the Ukraine war" | realtime | |
| | "Who is the Prime Minister of the UK right now?" | realtime | |
|
|
| --- |
|
|
| ## 🚀 How to Use |
|
|
| ### 1. Install dependencies |
| ```bash |
| pip install transformers torch |
| ``` |
|
|
| ### 2. Load and run the classifier |
| ```python |
| from query_classifier import QueryClassifier |
| |
| classifier = QueryClassifier() |
| |
| query = "What is the weather in Islamabad today?" |
| category, confidence = classifier.classify(query) |
| |
| print(f"Category: {category}") # realtime |
| print(f"Confidence: {confidence:.2f}") # e.g. 0.97 |
| ``` |
|
|
| ### 3. Output |
| ``` |
| Category: realtime |
| Confidence: 0.97 |
| ``` |
|
|
| --- |
|
|
| ## 📁 Project Structure |
|
|
| ``` |
| query-classifier/ |
| ├── train_classifier.py # Fine-tuning script |
| ├── query_classifier.py # Inference class (plug-and-play) |
| ├── training_data.csv # Labeled dataset |
| ├── trained_model/ # Saved model weights (after training) |
| │ ├── config.json |
| │ ├── tokenizer_config.json |
| │ └── model.safetensors |
| └── README.md |
| ``` |
|
|
| --- |
|
|
| ## 🏋️ Training |
|
|
| ```bash |
| python train_classifier.py |
| ``` |
|
|
| Trains for 3 epochs with AdamW optimizer (lr=5e-5), saves the best checkpoint based on validation accuracy. |
|
|
| --- |
|
|
| ## 🔌 Integration Example |
|
|
| This classifier is designed to sit **in front of your LLM pipeline**: |
|
|
| ```python |
| category, confidence = classifier.classify(user_query) |
| |
| if category == "realtime": |
| response = call_search_api(user_query) # Tavily, Serper, etc. |
| else: |
| response = call_llm(user_query) # GPT-4, Claude, etc. |
| ``` |
|
|
| --- |
|
|
| ## 🛠️ Built With |
|
|
| - [Hugging Face Transformers](https://huggingface.co/transformers/) |
| - [PyTorch](https://pytorch.org/) |
| - [MiniLM](https://huggingface.co/nreimers/MiniLM-L6-H384-uncased) — lightweight and fast |
|
|
| --- |
|
|
| ## 👤 Author |
|
|
| Built by [Your Name] — NLP & LLM Fine-tuning specialist. |
| Open to freelance projects → [Your Upwork / LinkedIn link] |
