README.md

metadata

license: other
license_name: link-attribution
license_link: https://dejanmarketing.com/link-attribution/
language:
  - en
metrics:
  - accuracy
  - f1
  - precision
  - recall
base_model: microsoft/deberta-v3-large
pipeline_tag: text-classification
tags:
  - grounding
  - retrieval
  - LLM-enhancement
  - DejanAI

Prompt Grounding Classifier

This model predicts whether a prompt requires grounding in external sources like web search, databases, or RAG pipelines.

It was fine-tuned from microsoft/deberta-v3-large using binary labels:

• 1 = grounding required
• 0 = grounding not required

---

🚀 Use Case

This classifier acts as a routing layer in an LLM pipeline, helping decide:

• When to trigger retrieval
• When to let the model respond from internal knowledge
• How to optimize for latency and cost

---

📦 Training Details

• Model: DeBERTa v3 Large
• Fine-tuning: Full (no adapters)
• Dropout: 0.1
• Scheduler: Cosine with warmup
• Batch size: 24 (accumulated)
• Evaluation: every 500 steps
• Metric used for best checkpoint: F1

---

🧪 Example Predictions

Prompt	Grounding	Confidence
What’s the exchange rate for USD to Yen right now?	1	0.999
Tell me a bedtime story about a robot and a dragon.	0	0.9961
Who is the current CEO of Microsoft?	1	0.9986

---

🧠 How to Use

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch.nn.functional as F

model = AutoModelForSequenceClassification.from_pretrained("dejanseo/query-grounding")
tokenizer = AutoTokenizer.from_pretrained("dejanseo/query-grounding")

prompt = "What time is the next train from Tokyo to Osaka?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model(**inputs).logits
probs = F.softmax(outputs, dim=-1)
label = probs.argmax().item()
confidence = probs[0][label].item()

---

🧾 Dataset Origin

Prompts were collected using a Gemini 2.5 Pro + Google Search toolchain with grounding enabled. Each prompt's response was parsed to extract Gemini's grounding confidence, used as soft supervision for binary labeling:

• Label 1 if grounded confidence present
• Label 0 if response required no external evidence