Spaces:
Sleeping
Sleeping
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,62 +1,151 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# ποΈ Federal FOIA Intelligence Search
|
| 2 |
### Public Electronic Reading Rooms Only
|
| 3 |
|
| 4 |
-
|
| 5 |
-
|
|
|
|
| 6 |
|
| 7 |
---
|
| 8 |
|
| 9 |
-
##
|
| 10 |
-
|
| 11 |
-
-
|
| 12 |
-
-
|
| 13 |
-
-
|
|
|
|
|
|
|
|
|
|
| 14 |
|
| 15 |
---
|
| 16 |
|
| 17 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
| 22 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
- NGA
|
| 24 |
-
- (Others may be added similarly)
|
| 25 |
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
- β Cannot be exported
|
| 30 |
|
| 31 |
-
|
|
|
|
|
|
|
| 32 |
|
| 33 |
---
|
| 34 |
|
| 35 |
-
##
|
| 36 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 37 |
|
| 38 |
---
|
| 39 |
|
| 40 |
-
##
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
---
|
| 48 |
|
| 49 |
-
##
|
| 50 |
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
> Agencies that block automated access or lack public search endpoints are represented with clearly labeled stub results for transparency only.
|
| 54 |
-
>
|
| 55 |
-
> Stub adapters do not perform network requests and cannot export content.
|
| 56 |
-
>
|
| 57 |
-
> This application indexes only documents lawfully released to the public under FOIA.
|
| 58 |
|
| 59 |
---
|
| 60 |
|
| 61 |
-
|
| 62 |
-
Research, journalism, transparency, and educational use only.
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
title: 'FOIA DECLASSIFIED DOCUMENTS SEARCH '
|
| 4 |
+
sdk: gradio
|
| 5 |
+
colorFrom: purple
|
| 6 |
+
colorTo: gray
|
| 7 |
+
pinned: true
|
| 8 |
+
short_description: 'FOIA DECLASSIFIED DOCUMENTS SEARCH '
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
# ποΈ Federal FOIA Intelligence Search
|
| 12 |
### Public Electronic Reading Rooms Only
|
| 13 |
|
| 14 |
+
A **live federated search application** for discovering documents published in **public U.S. Government FOIA Electronic Reading Rooms**.
|
| 15 |
+
|
| 16 |
+
This application **does not scrape**, **does not bypass access controls**, and **does not access classified, restricted, or non-public systems**.
|
| 17 |
|
| 18 |
---
|
| 19 |
|
| 20 |
+
## π What This App Does
|
| 21 |
+
|
| 22 |
+
- Searches **publicly available FOIA libraries**
|
| 23 |
+
- Aggregates results across multiple agencies
|
| 24 |
+
- Clearly distinguishes:
|
| 25 |
+
- π’ **Live public sources**
|
| 26 |
+
- π **Stub / informational coverage**
|
| 27 |
+
- Provides transparency tooling for journalists, researchers, and the public
|
| 28 |
|
| 29 |
---
|
| 30 |
|
| 31 |
+
## π‘οΈ Compliance & Safety Guarantees
|
| 32 |
+
|
| 33 |
+
- β
Public Electronic Reading Rooms only
|
| 34 |
+
- β
Honors robots.txt per adapter
|
| 35 |
+
- β
No authentication, credentials, or scraping
|
| 36 |
+
- β
No training, inference, or ML on restricted data
|
| 37 |
+
- β
Stub results **cannot be exported**
|
| 38 |
+
- β
All exported documents link to public URLs
|
| 39 |
+
|
| 40 |
+
> **Stub results are informational and cannot be exported.**
|
| 41 |
+
|
| 42 |
+
---
|
| 43 |
|
| 44 |
+
## π§ Search Modes
|
| 45 |
|
| 46 |
+
### Standard Mode (Default)
|
| 47 |
+
- Live public FOIA reading rooms
|
| 48 |
+
- Safe for export
|
| 49 |
+
- No experimental features
|
| 50 |
+
|
| 51 |
+
### Extended Coverage Mode (Opt-In)
|
| 52 |
+
Some agencies publish material inconsistently or restrict automation.
|
| 53 |
+
|
| 54 |
+
Extended mode:
|
| 55 |
+
- Requires user acknowledgment
|
| 56 |
+
- Clearly labels blocked or stubbed agencies
|
| 57 |
+
- Never exports restricted results
|
| 58 |
+
|
| 59 |
+
Agencies currently marked as **blocked or partial**:
|
| 60 |
+
- DIA
|
| 61 |
- NGA
|
|
|
|
| 62 |
|
| 63 |
+
---
|
| 64 |
+
|
| 65 |
+
## π§Ύ Export Rules
|
|
|
|
| 66 |
|
| 67 |
+
- ZIP export is enabled **only when live results are present**
|
| 68 |
+
- Stub results are excluded automatically
|
| 69 |
+
- All exports are traceable to public URLs
|
| 70 |
|
| 71 |
---
|
| 72 |
|
| 73 |
+
## π Built-In Transparency Tools
|
| 74 |
+
|
| 75 |
+
- π **Agency coverage heatmap**
|
| 76 |
+
- β±οΈ **Per-agency latency & health badges**
|
| 77 |
+
- π **Release timeline (by publication date)**
|
| 78 |
+
- π **Agency discovery status**
|
| 79 |
+
- π§Ύ **Court-ready citation formatting (public documents only)**
|
| 80 |
|
| 81 |
---
|
| 82 |
|
| 83 |
+
## βοΈ Phase-3 Expansion Pack (Planned)
|
| 84 |
+
|
| 85 |
+
These features are **architectural extensions** and are **not active by default**.
|
| 86 |
+
|
| 87 |
+
### π§Ύ Court Tools
|
| 88 |
+
- Litigation appendix generator (PDF)
|
| 89 |
+
- Exhibit numbering (A-1, A-2β¦)
|
| 90 |
+
- Declaration-ready citation blocks
|
| 91 |
+
- FOIA exemption (b-code) frequency charts
|
| 92 |
+
|
| 93 |
+
### π° Journalism Tools
|
| 94 |
+
- Timeline narrative builder
|
| 95 |
+
- Source confidence tags
|
| 96 |
+
- βWhatβs missingβ agency gap analysis
|
| 97 |
+
- Redaction density metrics
|
| 98 |
+
|
| 99 |
+
### βοΈ Compliance Controls
|
| 100 |
+
- Export locked to live results only
|
| 101 |
+
- Every document traceable to public URL
|
| 102 |
+
- Stub data never enters PDFs
|
| 103 |
+
- Optional disclosure watermarking
|
| 104 |
+
|
| 105 |
+
### π§ Advanced (Opt-In)
|
| 106 |
+
- Semantic clustering by topic (post-retrieval only)
|
| 107 |
+
- Cross-agency entity graphs
|
| 108 |
+
- FOIA response-time benchmarking
|
| 109 |
+
|
| 110 |
+
> No Phase-3 feature enables access to non-public data.
|
| 111 |
+
|
| 112 |
+
---
|
| 113 |
+
|
| 114 |
+
## π Why This Does Not Violate Intelligence Restrictions
|
| 115 |
+
|
| 116 |
+
- The app queries **only public-facing FOIA libraries**
|
| 117 |
+
- No inference is performed on classified material
|
| 118 |
+
- No automation targets restricted systems
|
| 119 |
+
- All content is already published by the agencies themselves
|
| 120 |
+
- The app functions as a **search index**, not a data broker
|
| 121 |
+
|
| 122 |
+
---
|
| 123 |
+
|
| 124 |
+
## π§ͺ Testing & Reliability
|
| 125 |
+
|
| 126 |
+
- Adapter compliance tests ensure:
|
| 127 |
+
- Public access only
|
| 128 |
+
- Robots.txt compliance
|
| 129 |
+
- Required metadata fields
|
| 130 |
+
- Health checks run without persistence or tracking
|
| 131 |
+
|
| 132 |
+
---
|
| 133 |
+
|
| 134 |
+
## π Intended Users
|
| 135 |
+
|
| 136 |
+
- Journalists
|
| 137 |
+
- Researchers
|
| 138 |
+
- Attorneys
|
| 139 |
+
- Transparency advocates
|
| 140 |
+
- Members of the public
|
| 141 |
|
| 142 |
---
|
| 143 |
|
| 144 |
+
## π Disclaimer
|
| 145 |
|
| 146 |
+
This application is **not affiliated with any U.S. government agency**.
|
| 147 |
+
All documents remain the property of their originating agencies.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 148 |
|
| 149 |
---
|
| 150 |
|
| 151 |
+
**Public transparency. Public sources. Clear boundaries.**
|
|
|