Ram-090 Claude Opus 4.6 (1M context) commited on
Commit
a47bd99
·
1 Parent(s): 19e2a5e

Add HuggingFace Spaces metadata to README.md

Browse files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +29 -189
README.md CHANGED
@@ -1,200 +1,40 @@
1
- # Verification-Driven Hallucination Firewall (VDHF)
2
-
3
- A modular Python system that verifies RAG (Retrieval-Augmented Generation) outputs before delivering them to users, preventing AI hallucinations.
4
-
5
- ---
6
-
7
- ## 📋 Prerequisites
8
-
9
- ### Required
10
- 1. **Python 3.9+** - Download from https://www.python.org/downloads/
11
- - ⚠️ During installation, check **"Add Python to PATH"**
12
-
13
- ### Optional (for full LLM features)
14
- 2. **OpenAI API Key** - Get from https://platform.openai.com/api-keys
15
- - Without this, the system uses mock generation for testing
16
-
17
- ---
18
-
19
- ## 🚀 Quick Start
20
-
21
- ### Step 1: Install Dependencies
22
- ```powershell
23
- cd "c:\Users\HP\Desktop\Hallucination Firewall"
24
- pip install -r requirements.txt
25
- ```
26
-
27
- ### Step 2: (Optional) Configure API Key
28
- ```powershell
29
- copy .env.example .env
30
- # Edit .env and add your OpenAI API key
31
- ```
32
-
33
- ### Step 3: Run the System
34
- ```powershell
35
- python main.py
36
- ```
37
-
38
- ---
39
-
40
- ## 📂 Project Structure
41
-
42
- ```
43
- Hallucination Firewall/
44
-
45
- ├── config.py # Configuration (thresholds, models)
46
- ├── ingestion.py # Document loading (PDF, TXT, DOCX)
47
- ├── embeddings.py # Sentence-BERT embeddings + ChromaDB
48
- ├── retriever.py # Semantic search for evidence
49
- ├── generator.py # LLM response generation
50
- ├── claim_extractor.py # Extract factual claims from text
51
- ├── verifier.py # Verify claims using similarity + NLI
52
- ├── firewall.py # Decision engine (pass/block)
53
- ├── prompt_refiner.py # Regenerate safer responses
54
- ├── main.py # Main pipeline + interactive CLI
55
-
56
- ├── requirements.txt # Python dependencies
57
- ├── .env.example # API key template
58
-
59
- ├── sample_docs/
60
- │ └── sample.txt # Sample test documents
61
-
62
- └── tests/
63
- └── test_pipeline.py # Unit tests
64
- ```
65
-
66
- ---
67
-
68
- ## 🔧 How It Works
69
-
70
- ```
71
- ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
72
- │ User Query │───▶│ Retrieve │───▶│ Generate Answer │
73
- └─────────────┘ │ Evidence │ └────────┬────────┘
74
- └──────────────┘ │
75
-
76
- ┌─────────────┐ ┌──────────────┐ ┌─────────────────┐
77
- │ Output │◀───│ Firewall │◀───│ Extract Claims │
78
- │ (Verified) │ │ Decision │ │ & Verify │
79
- └─────────────┘ └──────────────┘ └─────────────────┘
80
-
81
- │ If fails threshold
82
-
83
- ┌──────────────┐
84
- │ Refine & │
85
- │ Regenerate │
86
- └──────────────┘
87
- ```
88
-
89
- ### Pipeline Steps:
90
- 1. **Ingest Documents** → Load PDFs/TXT files into vector database
91
- 2. **Retrieve Evidence** → Find relevant chunks for user query
92
- 3. **Generate Response** → LLM creates initial answer
93
- 4. **Extract Claims** → Break response into atomic factual statements
94
- 5. **Verify Claims** → Check each claim against evidence
95
- 6. **Firewall Decision** → Pass if ≥80% claims verified
96
- 7. **Regenerate** → If failed, refine prompt and try again
97
-
98
  ---
99
-
100
- ## 💻 Interactive Commands
101
-
102
- When running `main.py`, use these commands:
103
-
104
- | Command | Description |
105
- |---------|-------------|
106
- | `/ingest <path>` | Load documents from file or folder |
107
- | `/clear` | Clear all documents from memory |
108
- | `/count` | Show number of document chunks |
109
- | `/quit` | Exit the program |
110
-
111
- **Example session:**
112
- ```
113
- You: /ingest sample_docs
114
- Ingested sample_docs: 5 chunks total
115
-
116
- You: When was Python released?
117
- [Processing...]
118
- ✓ VERIFIED - Support Ratio: 100%
119
- Response: Python was first released in 1991 by Guido van Rossum.
120
- ```
121
-
122
- ---
123
-
124
- ## ⚙️ Configuration
125
-
126
- Edit `config.py` to customize:
127
-
128
- | Setting | Default | Description |
129
- |---------|---------|-------------|
130
- | `SIMILARITY_THRESHOLD` | 0.75 | Minimum similarity for claim-evidence match |
131
- | `FIREWALL_THRESHOLD` | 0.8 | Minimum % of claims that must be verified |
132
- | `TOP_K_RETRIEVAL` | 7 | Number of evidence chunks to retrieve |
133
- | `CHUNK_SIZE` | 1000 | Characters per document chunk |
134
- | `LLM_MODEL` | gpt-3.5-turbo | OpenAI model to use |
135
-
136
- ---
137
-
138
- ### Step 4: Run Interactive Querying
139
- To interactively query the system and see verification results:
140
- ```powershell
141
- python run.py --demo
142
- ```
143
-
144
- ---
145
-
146
- ## 🧪 Running Tests
147
-
148
-
149
- ```powershell
150
- python -m pytest tests/test_pipeline.py -v
151
- ```
152
-
153
  ---
154
 
155
- ## 📝 Example Output
156
-
157
- ```
158
- ============================================================
159
- VDHF Pipeline Result
160
- ============================================================
161
- Status: ✓ VERIFIED
162
- Support Ratio: 100.00%
163
- Claims: 2/2 supported
164
- Regeneration Attempts: 0
165
- ============================================================
166
- Response:
167
- Python was released in 1991. It was created by Guido van Rossum.
168
- ============================================================
169
- ```
170
 
171
- ---
172
 
173
- ## 🔑 What You Need to Provide
174
 
175
- | Item | Required? | How to Get |
176
- |------|-----------|------------|
177
- | Python 3.9+ | ✅ Yes | https://python.org/downloads |
178
- | Documents to verify against | ✅ Yes | Your PDFs, TXT files |
179
- | OpenAI API Key | ❌ Optional | https://platform.openai.com/api-keys |
180
 
181
- ---
 
 
 
 
 
182
 
183
- ## 📚 Module Details
184
 
185
- ### claim_extractor.py
186
- Extracts atomic factual claims from LLM responses:
187
- - Filters out opinions ("I think...")
188
- - Splits compound sentences ("X and Y" → "X", "Y")
189
- - Identifies verifiable statements
 
190
 
191
- ### verifier.py
192
- Two-step verification:
193
- 1. **Semantic Similarity**: Cosine similarity ≥ 0.75
194
- 2. **NLI Entailment**: Evidence must logically support claim
195
 
196
- ### firewall.py
197
- Decision logic:
198
- - Calculate `SupportRatio = supported_claims / total_claims`
199
- - If ratio ≥ 0.8: **PASS** (deliver to user)
200
- - If ratio < 0.8: **REGENERATE** (refine and retry)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Hallucination Firewall
3
+ emoji: 🛡️
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: docker
7
+ pinned: false
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
+ # Verification-Driven Hallucination Firewall (VDHF)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
+ A modular Python system that verifies RAG (Retrieval-Augmented Generation) outputs before delivering them to users, preventing AI hallucinations.
13
 
14
+ Upload documents (TXT, PDF, DOCX, Excel, CSV), ask questions, and get verified answers with every claim checked against your content.
15
 
16
+ ## How It Works
 
 
 
 
17
 
18
+ 1. **Upload Documents** - Upload any document to the system
19
+ 2. **Ask Questions** - Query your uploaded content
20
+ 3. **Claim Extraction** - Every factual claim in the response is identified
21
+ 4. **Verification** - Each claim is checked against your uploaded data
22
+ 5. **Firewall Decision** - Response is marked as Verified, Partially Verified, or Hallucinated
23
+ 6. **Regeneration** - If needed, a safer response is generated
24
 
25
+ ## Features
26
 
27
+ - Excel/CSV direct data analysis (no ML models needed)
28
+ - Student comparison and filter queries
29
+ - Claim verification against uploaded data
30
+ - Hallucination detection for non-existent records
31
+ - Groq LLM-powered analysis for complex questions
32
+ - Beautiful React frontend with tabular response rendering
33
 
34
+ ## Tech Stack
 
 
 
35
 
36
+ - **Backend**: FastAPI + Python
37
+ - **Frontend**: React + Vite + Tailwind CSS
38
+ - **ML**: Sentence-BERT, DeBERTa NLI
39
+ - **Vector DB**: ChromaDB
40
+ - **LLM**: Groq API