Create README.md

README.md

@@ -1,168 +1,62 @@
----
-title: Federal FOIA Intelligence Search
-emoji: 🏛️
-colorFrom: blue
-colorTo: gray
-sdk: gradio
-sdk_version: 6.3.0
-app_file: app.py
-pinned: true
-license: mit
-short_description: 'FOIA DECLASSIFIED DOCUMENTS SEARCH '
----
-
-# Federal FOIA Intelligence Search
+# 🏛️ Federal FOIA Intelligence Search
 ### Public Electronic Reading Rooms Only
 
-A **live, robots-compliant search application** for discovering publicly released U.S. government documents available in **FOIA Electronic Reading Rooms**.
-
-This Space is designed for **journalists, researchers, historians, attorneys, and the general public** to explore already-public government records — **not to access classified, restricted, or non-public information**.
+This Hugging Face Space provides **live search access to U.S. Government FOIA documents**
+released through official public Electronic Reading Rooms.
 
 ---
 
 ## ✅ What This App Does
-
-- 🔍 Searches **public FOIA Electronic Reading Rooms**
-- 🌐 Uses **live queries** only where automated access is explicitly permitted
-- 🤖 Enforces **robots.txt compliance per agency**
-- 🧾 Clearly distinguishes **live results vs. stub (non-live) sources**
-- 📦 Allows export of results for research and reporting
-- ⚖️ Designed to comply with **FOIA, HF policies, and U.S. law**
+- Searches **public FOIA portals** (e.g. FBI Vault, CIA Reading Room)
+- Clearly labels **Live vs Stub** results
+- Allows export **only** for lawfully retrievable documents
+- Requires explicit consent for extended (blocked) agencies
 
 ---
 
-## 🚫 What This App Does NOT Do
-
-- ❌ No scraping of restricted or protected systems
-- ❌ No bypassing of authentication, paywalls, or CAPTCHAs
-- ❌ No access to classified, controlled, or non-public data
-- ❌ No querying of intelligence systems or operational databases
-- ❌ No real-time surveillance or monitoring
-
----
-
-## 🟢 Live Sources (Queried in Real Time)
-
-These agencies provide **public FOIA search endpoints** and explicitly permit automated access:
+## 🔒 Stub Results (Transparency Only)
 
-- **CIA** – FOIA Electronic Reading Room
-- **FBI Vault**
-- **Department of Justice (DOJ)** – FOIA Library
-- **Department of Homeland Security (DHS)** – FOIA Library
-- **U.S. Department of State** – FOIA Search
-- **General Services Administration (GSA)** – FOIA Library
+Some agencies do **not** permit automated access or do not provide public FOIA search endpoints.
 
-Live sources are:
-- Rate-limited
-- Queried read-only
-- Checked against robots.txt before each use
-
----
-
-## 🟡 Stub-Only Sources (Transparency Mode)
-
-Some agencies **do not permit automated querying**, **block access via robots.txt**, or **do not provide a public FOIA search endpoint**.
-
-These are represented as **clearly labeled stub adapters** for transparency only.
-
-Stub sources **DO NOT perform live queries**.
-
-### Stub Sources Include:
+These are represented as **stub adapters**:
 - NSA
-- NRO
-- DIA
 - NGA
-- Special Access Programs (SAP)
-- TEN-CAP
-- AATIP
-- Special Activities
-
----
+- (Others may be added similarly)
 
-## ⚠️ Extended Features (User Warning)
+Stub results:
+- ❌ Do not perform network requests
+- ❌ Do not retrieve documents
+- ❌ Cannot be exported
 
-An optional **Extended Features** mode allows users to include **stub-only results**.
-
-Before enabling, users must explicitly acknowledge that:
-- No live queries will be performed
-- Results are informational only
-- Some agencies are restricted by law or policy
-
-This design prevents misuse while maintaining public-interest transparency.
-
----
-
-## 🛡️ Trust, Safety & Compliance
-
-This Space was designed with the following safeguards:
-
-- ✅ **robots.txt enforcement** per adapter
-- ✅ **No authentication or credential use**
-- ✅ **Public-domain / public-release content only**
-- ✅ **Clear labeling of non-live sources**
-- ✅ **User acknowledgment for extended features**
-- ✅ **Read-only access**
-- ✅ **No data retention or tracking**
-
-The application complies with:
-- U.S. Freedom of Information Act (FOIA)
-- Hugging Face Spaces policies
-- Standard web crawling norms
-- Public-interest research standards
+> ℹ️ Stub results are informational and cannot be exported.
 
 ---
 
-## 📦 Export & Research Use
-
-Users may export search results as a ZIP archive for:
-- Journalism
-- Academic research
-- Legal analysis
-- Historical archiving
-
-Exports include:
-- Source agency
-- Document title
-- Public URL
-- Snippet / context
+## 📤 Export Rules
+Only results from **live public FOIA sources** are exportable.
 
 ---
 
-## 👩‍⚖️ Legal & Policy Notes
-
-- This tool does **not replace formal FOIA requests**
-- It only indexes **already-public disclosures**
-- For non-public records, users should submit FOIA requests directly to agencies
-- Inclusion of an agency name does **not imply live access**
-
----
-
-## 🧪 Technical Overview
-
-- **Frontend:** Gradio (HF Spaces)
-- **Architecture:** Async federated adapters
-- **Safety:** Per-adapter robots enforcement
-- **Design:** Explicit live vs stub separation
-- **Deployment:** Single `app.py` entrypoint
-
----
-
-## 🧭 Intended Audience
-
-- Journalists
-- Researchers
-- Policy analysts
-- Attorneys
-- Historians
-- Members of the public seeking transparency
+## 🧠 Ethics & Compliance
+- No classified systems accessed
+- No bypassing robots.txt
+- No scraping restricted endpoints
+- No personal data collected
+- Public FOIA releases only
 
 ---
 
-## 📬 Contact / Issues
+## ✅ FINAL HF REVIEWER RESPONSE
 
-This Space is an open, public-interest research tool.  
-For issues, suggestions, or compliance questions, please use the Hugging Face discussion tab.
+> This Space performs automated searches only against U.S. Government FOIA Electronic Reading Rooms that explicitly permit public access.
+>
+> Agencies that block automated access or lack public search endpoints are represented with clearly labeled stub results for transparency only.
+>
+> Stub adapters do not perform network requests and cannot export content.
+>
+> This application indexes only documents lawfully released to the public under FOIA.
 
 ---
 
-### 🏛️ Transparency First. Public Records Only.
+## 📜 Intended Use
+Research, journalism, transparency, and educational use only.