README.md

# 🔎 HF Space — Automated Requirement Citation Checker

This Space ingests a CSV with columns: **Prompt, Rubric, ID, Requirements, Important Evaluation Rules** and
verifies each atomic claim in the **Requirements** column against the public web.

It searches via **Google Custom Search (CSE)**, fetches top pages, and asks **both OpenAI** and **DeepSeek**
LLMs to decide if each claim is **SUPPORTED**, **CONTRADICTED**, or **UNVERIFIABLE**, returning citations and a concise reason.
A per-row conclusion is computed from all claims.

## Quick Start (Hugging Face Spaces)

1. Create a new Space → SDK: **Gradio** → Visibility: your choice.
2. Upload these files to the Space root:
   - `app.py`
   - `verifier.py`
   - `search.py`
   - `models.py`
   - `utils.py`
   - `requirements.txt`
   - `.env.template` (optional for local dev)
3. In the Space page → **Settings** → **Secrets**, add:
   - `OPENAI_API_KEY` — your OpenAI key (you mentioned `OAPI1`).
   - `DEEPSEEK_API_KEY` — your DeepSeek key (you mentioned `DAPI`).
   - `GOOGLE_API_KEY` — Google API key (you mentioned `API1`).
   - `GOOGLE_CSE_ID` — your CSE ID (you mentioned `CX1`).
   - (Optional) `OPENAI_MODEL` (default `gpt-4o-mini`)
   - (Optional) `DEEPSEEK_MODEL` (default `deepseek-chat`)
4. Deploy. Open the Space, upload your CSV, and click **Run Verification**.

## Output

- **Table view** summarizing Supported/Contradicted/Unverifiable counts and overall conclusion per row.
- **JSON** with full detail (per-claim statuses, reasons, citations, and both models' raw decisions).
- **Markdown report** with human-readable results and links.

## CSV Format

Columns required (case-sensitive):
- `Prompt`
- `Rubric`
- `ID`
- `Requirements`
- `Important Evaluation Rules`

Only **Requirements** is fact-checked; other columns are passed through for context if you want to extend prompts later.

## How it works

- `utils.split_atomic_claims` heuristically splits Requirements text into atomic claims (lists + sentences).
- `search.google_cse` queries Google CSE (multiple queries per claim), collects unique result links.
- `search.fetch_page` downloads pages and extracts plain text + titles for LLM review.
- `verifier.verify_claim` builds a single packed prompt with all sources and asks **both** OpenAI & DeepSeek to return structured JSON:
  ```json
  {
    "status": "SUPPORTED|CONTRADICTED|UNVERIFIABLE",
    "reason": "brief rationale",
    "citations": [{"url": "...", "title": "...", "quote": "short"}]
  }
  ```
- An ensemble chooses the final status; the other model’s output is preserved for transparency.
- `verifier.verify_requirement` aggregates claim-level results and computes a row-level conclusion:
  - Any CONTRADICTED → **CONTRADICTED**
  - Else if all supported → **SUPPORTED**
  - Else if mixed → **PARTIALLY SUPPORTED**
  - Else → **UNVERIFIABLE**

## Local Development

```bash
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.template .env  # fill keys
python app.py
```

## Notes & Tips

- Be conservative: the models are instructed to mark **UNVERIFIABLE** when evidence is insufficient.
- You can tune the number of search results and page length limits in `search.py`.
- For highly technical domains, consider whitelisting official domains in your CSE for higher precision.
- If you hit rate limits, Tenacity will retry with exponential backoff.

## License

MIT