Create README.md

README.md

@@ -4,76 +4,174 @@ emoji: 🏛️
 colorFrom: blue
 colorTo: indigo
 sdk: gradio
-sdk_version: 6.3.0
+sdk_version: 4.44.0
 app_file: app.py
-pinned: false
+pinned: true
 tags:
 - foia
-- public-records
 - government-transparency
+- public-records
 - journalism
 - open-data
+- legal-tech
+- oversight
+license: mit
+short_description: 'FOIA INTELLIGENCE SEARCH '
 ---
 
 # Federal FOIA Intelligence Search
 
 **Public Electronic Reading Rooms Only**
 
-This Hugging Face Space provides a federated search interface across
-**publicly available U.S. Government FOIA Electronic Reading Rooms**.
+This Hugging Face Space provides a federated search and analysis interface for
+**publicly released U.S. Government records made available under the Freedom of Information Act (FOIA)**.
+
+It is designed for **journalists, researchers, legal professionals, oversight bodies, and the public**.
 
 ---
 
 ## 🔍 What This Space Does
-- Searches **only public FOIA repositories**
-- Uses agency-published search endpoints
-- Enforces robots.txt and rate limits
-- Provides semantic clustering and citation export
+
+- Searches **only publicly accessible FOIA Electronic Reading Rooms**
+- Uses **official agency search endpoints or public landing pages**
+- Enforces **robots.txt, rate limiting, and safe defaults**
+- Provides **semantic search, clustering, and visualization**
+- Generates **court-ready citations and FOIA request packets**
 
 ---
 
 ## 🏛️ Live Public Sources
-- CIA FOIA Reading Room
-- FBI Vault
-- NSA FOIA Library
-- Department of Defense FOIA
-- National Reconnaissance Office (NRO) Releases
-- DOJ / DHS / State (metadata links)
+
+This Space currently supports live querying or indexed access to the following **public FOIA repositories**:
+
+- **CIA FOIA Electronic Reading Room**
+- **FBI Vault**
+- **NSA FOIA Library**
+- **Department of Defense FOIA Reading Room**
+- **National Reconnaissance Office (NRO) Declassified Releases**
+- **Department of Justice (DOJ) FOIA Library**
+- **Department of Homeland Security (DHS) FOIA Library**
+- **U.S. Department of State FOIA Reading Room**
+
+All access is:
+- Public
+- Unauthenticated
+- Non-privileged
+- Read-only
 
 ---
 
-## 📂 Hosted Public Collections
-Some historically named programs (e.g. **AATIP**, **SAP**, **Special Activities**)
-do **not** have independent FOIA search systems.
+## 📂 Hosted Public Collections (Clearly Labeled)
+
+Some historically named programs or collections (e.g. **AATIP**, **Special Access Programs**, **Special Activities**) do **not** operate independent FOIA portals.
+
+When these appear in the interface, they are:
 
-When included, they are:
-- Clearly labeled as **Hosted Public Releases**
+- Explicitly labeled as **Hosted Public Releases**
 - Linked to the **original publishing agency**
-- Restricted to already-released documents
+- Limited to **already-declassified, publicly released documents**
+- Included for **historical research and transparency**
+
+No restricted or classified systems are accessed.
+
+---
+
+## 📊 Analytics & Visualization Features
+
+- **Real-time agency coverage heatmap** (documents per agency)
+- **Latency & health indicators** per source
+- **Interactive semantic cluster graph** (Plotly)
+- **Timeline views** for document release dates
+- **Result deduplication and clustering**
+
+---
+
+## 🧠 Semantic Search
+
+- Uses **sentence-transformers + FAISS**
+- Supports:
+  - Semantic clustering
+  - Search-within-results
+  - Topic grouping
+- No model training on classified or private data
+
+---
+
+## 🧾 Legal & Journalistic Tools
+
+- **Court-ready Bluebook citation PDF export**
+- **Journalist ZIP export** (documents + index)
+- **FOIA request packet generator** (PDF / text)
+- **FOIA exemption (b-code) labeling** (heuristic, informational only)
+
+> ⚠️ FOIA requests are **generated only**.  
+> This Space does **not** submit requests on behalf of users.
 
 ---
 
 ## ⚖️ What This Space Does NOT Do
+
 ❌ No classified access  
 ❌ No scraping behind authentication  
+❌ No bypassing agency safeguards  
 ❌ No intelligence analysis or inference  
-❌ No automated FOIA submission  
+❌ No automated FOIA submissions  
+❌ No user tracking beyond standard HF analytics  
 
 ---
 
-## 🧾 Intended Users
-- Journalists
-- Researchers
-- Legal professionals
-- Oversight and compliance teams
+## 📜 Legal Basis
+
+All content accessed through:
+
+- **5 U.S.C. § 552 (Freedom of Information Act)**
+- Agency-maintained **Electronic Reading Rooms**
+- Public U.S. Government websites
+
+This Space operates entirely within:
+- U.S. law
+- Agency publication rules
+- Hugging Face platform policies
 
 ---
 
-## 📜 Legal Basis
-All content accessed via:
-- 5 U.S.C. § 552 (FOIA)
-- Agency electronic reading rooms
-- Public government websites
+## 🛡️ Safety & Governance
+
+- Robots.txt enforcement per adapter
+- Per-agency rate limits
+- Per-agency kill switches
+- Health monitoring & auto-disable
+- Adapter compliance tests (CI)
+
+---
+
+## 🎯 Intended Use
+
+This project is intended to support:
+
+- Investigative journalism
+- Academic and historical research
+- Legal review and litigation support
+- Government oversight
+- Public transparency initiatives
+
+---
+
+## 📬 Disclaimer
+
+This tool aggregates and analyzes **public information only**.  
+It does not provide legal advice, intelligence assessments, or official interpretations.
+
+Users are responsible for verifying primary sources.
+
+---
+
+## 📦 Open Source & Transparency
+
+All adapters, logic, and safety mechanisms are visible in the source code.
+No hidden data sources or privileged access exist.
+
+---
 
-This Space operates fully within
-Hugging Face content and safety policies.
+**Federal FOIA Intelligence Search**  
+*Making public records easier to find, understand, and cite.*