Upload ClaimLens Stage B RAG Verifier Model

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,119 @@
+---
+language:
+- en
+- hi
+- kn
+license: apache-2.0
+tags:
+- rag
+- hallucination-detection
+- xlm-roberta
+- nli
+- multilingual
+- fact-checking
+- text-classification
+pipeline_tag: text-classification
+widget:
+- text: "Antigravity is an AI development platform."
+  text_pair: "Antigravity is an AI development platform created by Deepmind engineers."
+- text: "The deadline is 30 June 2026."
+  text_pair: "Antigravity is an AI development platform created by Deepmind engineers."
+---
+
+# ClaimLens: Multilingual RAG Hallucination Verifier
+
+**ClaimLens** is a fine-tuned multilingual (`English`, `Hindi`, `Kannada`) claim-level hallucination verifier for Retrieval-Augmented Generation (RAG) pipelines based on `xlm-roberta-base`.
+
+It evaluates generated answers claim-by-claim against retrieved context passages and classifies each into one of three factual grounding states:
+- **`Correct` (Label 0):** Retrieved evidence entails the claim.
+- **`Incorrect` (Label 1):** Retrieved evidence contradicts the claim.
+- **`Unverified` (Label 2):** Evidence neither confirms nor denies the claim (unsupported / baseless).
+
+---
+
+## Model Architecture & Training Protocol
+
+* **Base Model:** `xlm-roberta-base`
+* **Task:** Sentence-Pair Classification (`<s> claim </s></s> evidence </s>`)
+* **Two-Stage Supervised Fine-Tuning:**
+  1. **Stage A (NLI Warm-up):** Trained on 60,000 NLI pairs (20k each for English, Hindi, and Kannada from `facebook/xnli` and `Divyanshu/indicxnli`).
+  2. **Stage B (RAG Fine-Tuning):** Fine-tuned on RAGTruth claim-level data (English) + machine-translated Hindi and Kannada subsets, using **Class-Weighted Cross-Entropy** to mitigate the ~90% majority class skew.
+
+---
+
+## Evaluation Results
+
+Evaluated against the RAG test sets across languages:
+
+| Metric | English (`en`) | Hindi (`hi`) | Kannada (`kn`) |
+| :--- | :---: | :---: | :---: |
+| **Macro-F1** | **0.4888** | **0.6086** | **0.5662** |
+| **Hallucination F1** | **0.3676** | **0.8162** | **0.8057** |
+| **Accuracy** | **76.61%** | **61.00%** | **56.67%** |
+| **IndicGLUE WNLI (Hindi)** | — | **94.37%** | — |
+
+*Significant performance improvement over Majority Class (0.1667) and TF-IDF baselines (0.4059 - 0.4270).*
+
+---
+
+## Quickstart & Usage
+
+### Using Hugging Face Transformers
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+model_id = "FaustDerLegende/RAGVERIFIER"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForSequenceClassification.from_pretrained(model_id)
+model.eval()
+
+claim = "Antigravity is an AI development platform."
+evidence = "Antigravity is an AI development platform created by Deepmind engineers."
+
+# Tokenize sentence-pair (claim, evidence)
+inputs = tokenizer(
+    claim,
+    evidence,
+    truncation="only_second",
+    max_length=384,
+    return_tensors="pt"
+)
+
+with torch.no_grad():
+    logits = model(**inputs).logits
+    probs = torch.softmax(logits, dim=-1)[0]
+
+id2label = model.config.id2label
+for i, prob in enumerate(probs):
+    print(f"{id2label[i]}: {prob.item():.4f}")
+
+predicted_class = id2label[torch.argmax(probs).item()]
+print(f"
+Final Verdict: {predicted_class}")
+```
+
+### Using Hugging Face Pipeline
+
+```python
+from transformers import pipeline
+
+classifier = pipeline(
+    "text-classification",
+    model="FaustDerLegende/RAGVERIFIER",
+    tokenizer="FaustDerLegende/RAGVERIFIER"
+)
+
+result = classifier({
+    "text": "Antigravity is an AI development platform.",
+    "text_pair": "Antigravity is an AI development platform created by Deepmind engineers."
+})
+print(result)
+```
+
+---
+
+## Citation & Acknowledgment
+
+Developed as part of the **ClaimLens** Multilingual Hallucination Detection project, adapting datasets from [RAGTruth (Niu et al.)](https://github.com/ParticleMedia/RAGTruth), [IndicXNLI (Aggarwal et al.)](https://huggingface.co/datasets/Divyanshu/indicxnli), and [IndicGLUE](https://huggingface.co/datasets/ai4bharat/indic_glue).

config.json

@@ -0,0 +1,41 @@
+{
+  "add_cross_attention": false,
+  "architectures": [
+    "XLMRobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "eos_token_id": 2,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "Correct",
+    "1": "Incorrect",
+    "2": "Unverified"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "is_decoder": false,
+  "label2id": {
+    "Correct": 0,
+    "Incorrect": 1,
+    "Unverified": 2
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "xlm-roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "tie_word_embeddings": true,
+  "transformers_version": "5.20.2",
+  "type_vocab_size": 1,
+  "use_cache": false,
+  "vocab_size": 250002
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5074cbdd38a89ed0d834f8c9c0305a5e621dd66f85d8fec3e7837dbae3e5965
+size 1112208084

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c554c8acbec4ce21df5b0a2adfc7c5214224315810a8e4307c895b1e683b1127
+size 17098182

tokenizer_config.json

@@ -0,0 +1,19 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "is_local": true,
+  "local_files_only": false,
+  "mask_token": "<mask>",
+  "max_length": 128,
+  "model_max_length": 512,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "stride": 0,
+  "tokenizer_class": "XLMRobertaTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "only_second",
+  "unk_token": "<unk>"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae0bda57a3f4afc6cb9894eb3b387e608582b42eafeee7c3c677f8ffe52cd7cb
+size 5265