README.md

---
library_name: transformers
base_model: huawei-noah/TinyBERT_General_4L_312D
language:
- en
license: mit
pipeline_tag: text-classification
task_ids:
- fact-checking
tags:
- edge-rag
- semantic-filtering
- hallucination-reduction
- cross-encoder
metrics:
- accuracy
- precision
- recall
- roc_auc
model-index:
- name: LF_BERT_v1
  results:
  - task:
      type: fact-checking
      name: Semantic Evidence Filtering
    dataset:
      name: Project Sentinel (HotpotQA-derived)
      type: hotpotqa/hotpot_qa
    metrics:
    - type: accuracy
      value: 0.8167
    - type: precision
      value: 0.5907
    - type: recall
      value: 0.8674
    - type: roc_auc
      value: 0.9064
---

# LF_BERT_v1

**LF_BERT_v1** is a lightweight **TinyBERT-based cross-encoder** fine-tuned for **semantic evidence filtering** in **Retrieval-Augmented Generation (RAG)** pipelines.

The model acts as a *semantic gatekeeper*, scoring `(query, candidate_sentence)` pairs to determine whether the sentence is **factually useful evidence** or a **semantic distractor**.  
It is designed for **CPU-only, edge, and offline deployments**, with millisecond-level inference latency.

This model is the core filtering component of **Project Sentinel**.

---

## Model Description

- **Architecture:** TinyBERT (4 layers, 312 hidden size)
- **Type:** Cross-encoder (joint encoding of query and sentence)
- **Task:** Binary fact-checking / evidence verification
- **Base Model:** `huawei-noah/TinyBERT_General_4L_312D`
- **Inference Latency:** ~5.3 ms (CPU)

### Input Format

```
[CLS] query [SEP] candidate_sentence [SEP]
```

- Maximum sequence length: 512 tokens

### Output

- Probability score ∈ [0,1] representing **factual utility**
- Typical deployment threshold: **0.85** (Strict Guard configuration)

---

## Intended Use

✔ Semantic filtering for RAG pipelines  
✔ Hallucination reduction  
✔ Early-exit decision systems  
✔ Edge / offline LLM deployments  

This model is especially suited for:
- Local document QA systems
- Privacy-sensitive environments
- Resource-constrained hardware (≤ 8 GB RAM)

---

## Limitations

- Trained on Wikipedia-based QA (HotpotQA)
- English-only
- Sentence-level relevance (not passage-level reasoning)
- Not a factual verifier for open-world claims

Performance may degrade on highly domain-specific or non-factual corpora.

---

## Training Data

The model was trained on a **binary dataset derived from HotpotQA (Distractor setting)**.

### Labels

- **1 – Supporting Fact:** Ground-truth evidence sentences
- **0 – Distractor:** Topically similar but factually insufficient sentences

### Dataset Statistics

| Split | Samples |
|------|--------|
| Train | 69,101 |
| Validation | 7,006 |

The dataset is intentionally **imbalanced**, reflecting real retrieval scenarios.

---

## Training Procedure

### Hyperparameters

- Learning rate: `1e-5`
- Batch size: `16`
- Epochs: `2`
- Optimizer: AdamW
- Scheduler: Linear
- Seed: `42`
- Loss: Weighted cross-entropy

### Training Results

| Epoch | Validation Loss | F1 | Accuracy | Precision | Recall | ROC-AUC |
|------|-----------------|----|----------|-----------|--------|--------|
| 1 | 0.4003 | 0.7119 | 0.8290 | 0.6146 | 0.8457 | 0.9038 |
| 2 | 0.4042 | 0.7028 | 0.8167 | 0.5907 | 0.8674 | 0.9064 |

---

## Thresholded Performance (Strict Guard)

- **Decision threshold:** 0.85
- **Hallucination rate:** 5.92%
- **Fact retention:** 60.34%
- **Average latency:** 5.30 ms (CPU)

This configuration prioritizes **trustworthiness over recall**.

---

## Citation

If you use this model, please cite:

```
@article{salih2026sentinel,
  title={Project Sentinel: Lightweight Semantic Filtering for Edge RAG},
  author={Salih, El Mehdi and Ait El Mouden, Khaoula and Akchouch, Abdelhakim},
  year={2026}
}
```

---

## Contact

**El Mehdi Salih**  
Mohammed V University – Rabat  
Email: elmehdi_salih@um5.ac.ma