| --- |
| license: other |
| license_name: link-attribution |
| license_link: https://dejanmarketing.com/link-attribution/ |
| language: |
| - en |
| metrics: |
| - accuracy |
| - f1 |
| - precision |
| - recall |
| base_model: microsoft/deberta-v3-large |
| pipeline_tag: text-classification |
| tags: |
| - grounding |
| - retrieval |
| - LLM-enhancement |
| - DejanAI |
| --- |
| |
| [](https://dejan.ai/blog/grounding-classifier/) |
|
|
| # Prompt Grounding Classifier |
|
|
| This model predicts whether a prompt **requires grounding** in external sources like web search, databases, or RAG pipelines. |
|
|
| It was fine-tuned from [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) using binary labels: |
|
|
| - `1` = grounding required |
| - `0` = grounding not required |
|
|
| --- |
|
|
| ## 🚀 Use Case |
|
|
| This classifier acts as a **routing layer** in an LLM pipeline, helping decide: |
|
|
| - When to trigger retrieval |
| - When to let the model respond from internal knowledge |
| - How to optimize for latency and cost |
|
|
| --- |
|
|
| ## 📦 Training Details |
|
|
| - Model: DeBERTa v3 Large |
| - Fine-tuning: Full (no adapters) |
| - Dropout: 0.1 |
| - Scheduler: Cosine with warmup |
| - Batch size: 24 (accumulated) |
| - Evaluation: every 500 steps |
| - Metric used for best checkpoint: F1 |
|
|
| --- |
|
|
| ## 🧪 Example Predictions |
|
|
| | Prompt | Grounding | Confidence | |
| |---------------------------------------------------------|-----------|------------| |
| | What’s the exchange rate for USD to Yen right now? | 1 | 0.999 | |
| | Tell me a bedtime story about a robot and a dragon. | 0 | 0.9961 | |
| | Who is the current CEO of Microsoft? | 1 | 0.9986 | |
|
|
| --- |
|
|
| ## 🧠 How to Use |
|
|
| ```python |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| import torch.nn.functional as F |
| |
| model = AutoModelForSequenceClassification.from_pretrained("dejanseo/query-grounding") |
| tokenizer = AutoTokenizer.from_pretrained("dejanseo/query-grounding") |
| |
| prompt = "What time is the next train from Tokyo to Osaka?" |
| inputs = tokenizer(prompt, return_tensors="pt") |
| outputs = model(**inputs).logits |
| probs = F.softmax(outputs, dim=-1) |
| label = probs.argmax().item() |
| confidence = probs[0][label].item() |
| ``` |
|
|
| --- |
|
|
| ## 🧾 Dataset Origin |
|
|
| Prompts were collected using a Gemini 2.5 Pro + Google Search toolchain with grounding enabled. Each prompt's response was parsed to extract Gemini's grounding confidence, used as soft supervision for binary labeling: |
|
|
| - Label 1 if grounded confidence present |
| - Label 0 if response required no external evidence |
|
|
