dejanseo commited on
Commit
e423891
·
verified ·
1 Parent(s): 281c498

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -5
README.md CHANGED
@@ -1,5 +1,70 @@
1
- ---
2
- license: other
3
- license_name: link-attribution
4
- license_link: https://dejanmarketing.com/link-attribution/
5
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: link-attribution
4
+ license_link: https://dejanmarketing.com/link-attribution/
5
+ language:
6
+ - en
7
+ metrics:
8
+ - accuracy
9
+ - f1
10
+ - precision
11
+ - recall
12
+ base_model: microsoft/deberta-v3-large
13
+ pipeline_tag: text-classification
14
+ tags:
15
+ - grounding
16
+ - retrieval
17
+ - LLM-enhancement
18
+ - DejanAI
19
+ ---
20
+
21
+ [![Dejan AI Logo](https://dejan.ai/wp-content/uploads/2024/02/dejan.png)](https://dejan.ai/blog/grounding-classifier/)
22
+
23
+ ## Prompt Grounding Classifier — DeBERTa v3 Large (Fine-Tuned)
24
+
25
+ This model predicts whether a natural language prompt **requires grounding** in external sources such as search, database, or retrieval-augmented generation (RAG).
26
+
27
+ It was fine-tuned from [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) using a binary label format (`1 = requires grounding`, `0 = self-contained`).
28
+
29
+ ### Why this matters
30
+
31
+ Routing decisions matter. This classifier acts as a gatekeeper for LLM pipelines by predicting whether a prompt should trigger external retrieval. It optimizes performance, reduces latency, and avoids unnecessary API calls.
32
+
33
+ ---
34
+
35
+ ## Model Details
36
+
37
+ - 🧠 **Architecture**: DeBERTa v3 Large
38
+ - ⚙️ **Training**: Full fine-tuning (no PEFT)
39
+ - 🧪 **Batch size**: 24 (with accumulation)
40
+ - 🔁 **Scheduler**: Cosine learning rate decay with warmup
41
+ - 📉 **Dropout adjusted**: 0.1 for attention and hidden layers
42
+ - 📦 **Final checkpoint size**: ~1.7 GB
43
+
44
+ ---
45
+
46
+ ## Example Predictions
47
+
48
+ | Prompt | Grounding | Confidence |
49
+ |------------------------------------------------------|-----------|------------|
50
+ | What’s the exchange rate for USD to Yen right now? | 1 | 0.999 |
51
+ | Tell me a bedtime story about a robot and a dragon. | 0 | 0.996 |
52
+ | Who is the current CEO of Microsoft? | 1 | 0.998 |
53
+
54
+ ---
55
+
56
+ ## How to Use
57
+
58
+ ```python
59
+ from transformers import AutoTokenizer, AutoModelForSequenceClassification
60
+ import torch.nn.functional as F
61
+
62
+ model = AutoModelForSequenceClassification.from_pretrained("dejan/deberta-grounding-classifier")
63
+ tokenizer = AutoTokenizer.from_pretrained("dejan/deberta-grounding-classifier")
64
+
65
+ prompt = "What time is the next train from Tokyo to Osaka?"
66
+ inputs = tokenizer(prompt, return_tensors="pt")
67
+ outputs = model(**inputs).logits
68
+ probs = F.softmax(outputs, dim=-1)
69
+ label = probs.argmax().item()
70
+ confidence = probs[0][label].item()