RAG Evaluation Lab Baseline Model
Model Description
This repository contains a small, transparent prototype model for RAG systems often ship without a stable regression set or failure taxonomy.
The model combines per-label token weights with IDF-weighted evidence retrieval. It was generated for reproducible architecture demonstrations and does not call a hosted LLM.
Evaluation
- Held-out synthetic examples: 4
- Accuracy: 0.75
- Intended metrics: failure_class_accuracy, citation_coverage, release_gate_pass_rate
Intended Use
- Architecture prototyping
- CI and evaluation examples
- Local baseline comparisons
- Educational experimentation
Hugging Face Task Coverage
text-classificationquestion-answeringtext-rankingsummarization
Limitations and Risks
Synthetic cases validate the harness, not a production RAG system. Teams must add representative domain examples.
The dataset is synthetic and small. Do not use this model for consequential decisions without representative data, expert review, and production-grade evaluation.
Reproducibility
The linked GitHub repository includes train.py, the exact dataset split,
evaluation code, and the model JSON format.