Spaces:
Sleeping
Sleeping
| title: RICS Report Genius | |
| emoji: 🏠 | |
| colorFrom: blue | |
| colorTo: indigo | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: false | |
| license: mit | |
| short_description: RICS-style RAG report generator (FAISS + GPT-4o-mini). | |
| # Report Genius AI | |
| A production-quality **Retrieval-Augmented Generation (RAG)** system for producing | |
| RICS-style property survey reports. Each user's uploaded documents are indexed in a | |
| single pooled vector store with strict `tenant_id` isolation — your data never leaks | |
| into another user's results. | |
| --- | |
| ## Architecture | |
| ``` | |
| Upload (.docx/.pdf) | |
| │ | |
| ▼ | |
| Parser → Normaliser → Chunker | |
| │ | |
| ▼ | |
| Embeddings (OpenAI text-embedding-3-small) | |
| │ | |
| ▼ | |
| Vector Store (FAISS, on-disk) ← metadata: tenant_id, doc_id, chunk_id | |
| │ | |
| ▼ (query + tenant_id filter) | |
| Retriever (top-10) → Reranker (top-3) → ≤400 token context | |
| │ | |
| ▼ | |
| LLM Adapter (gpt-4o-mini, edit/adapt prompt) | |
| │ | |
| ▼ | |
| Post-processor ([VERIFY] enforcement) | |
| │ | |
| ▼ | |
| Section Cache (SHA-256 keyed) → Response JSON | |
| ``` | |
| --- | |
| ## Quick start | |
| ### 1. Prerequisites | |
| - Python 3.11+ | |
| - Docker & Docker Compose (for full stack) | |
| - An OpenAI API key | |
| ### 2. Clone and configure | |
| ```bash | |
| git clone https://github.com/My-Report-AI/Report-genius-ai.git | |
| cd Report-genius-ai | |
| cp .env.example .env | |
| # Edit .env and set OPENAI_API_KEY=sk-... | |
| ``` | |
| ### 3a. Run locally (without Docker) | |
| ```bash | |
| python -m venv .venv | |
| # Windows: | |
| .venv\Scripts\activate | |
| # macOS/Linux: | |
| source .venv/bin/activate | |
| pip install -e ".[dev]" | |
| # FAISS runs inside the API process (index under ~/.report_genius/faiss_index by default). | |
| uvicorn app.main:app --reload --port 8000 | |
| ``` | |
| ### 3b. Run with Docker Compose | |
| ```bash | |
| docker-compose up --build | |
| ``` | |
| API available at `http://localhost:8000`. | |
| Interactive docs at `http://localhost:8000/docs`. | |
| --- | |
| ## API endpoints | |
| | Method | Path | Description | | |
| |--------|------|-------------| | |
| | `POST` | `/upload` | Upload a `.docx` or `.pdf`; triggers ingestion | | |
| | `POST` | `/reports/{report_id}/generate` | Run retrieval + adapt pipeline | | |
| | `GET` | `/reports/{report_id}/sections` | Fetch generated sections | | |
| | `GET` | `/reports/{report_id}/status` | Poll ingestion / generation status | | |
| | `POST` | `/test/ingest` | **Dev only** — ingest a file from `/mnt/data/samples/` | | |
| ### Example: upload a file | |
| ```bash | |
| curl -X POST http://localhost:8000/upload \ | |
| -H "X-Tenant-ID: tenant_abc" \ | |
| -F "file=@/path/to/survey.docx" \ | |
| -F "tenant_id=tenant_abc" | |
| ``` | |
| ### Example: generate a section | |
| ```bash | |
| curl -X POST http://localhost:8000/reports/{report_id}/generate \ | |
| -H "X-Tenant-ID: tenant_abc" \ | |
| -H "Content-Type: application/json" \ | |
| -d '{ | |
| "template_id": "B1", | |
| "bullets": [ | |
| "Property is a semi-detached house built circa 1965", | |
| "Floor area approx 95 sqm", | |
| "Located in NW3 postcode" | |
| ] | |
| }' | |
| ``` | |
| ### Similar content, deduplication, and refreshing the knowledge library | |
| When surveyors add or revise inspection notes, the app can **surface similar material** already in the tenant’s workspace: | |
| 1. **Indexed uploads** (`.docx` / `.pdf` ingested into the FAISS index) — semantic (embedding) similarity. | |
| 2. **Other RICS sections’ draft notes** (sent from the browser) — lexical overlap (token Jaccard) to catch duplicate lines across sections. | |
| **API** | |
| ```bash | |
| curl -X POST http://localhost:8000/content/similar \ | |
| -H "X-Tenant-ID: tenant_abc" \ | |
| -H "Content-Type: application/json" \ | |
| -d '{ | |
| "text": "DPC minimum 150mm above ground per current guidance", | |
| "section_code": "E4", | |
| "peer_sections": { "I1": "…other section bullets…" }, | |
| "exclude_document_ids": [], | |
| "limit": 8, | |
| "min_relevance_percent": 28 | |
| }' | |
| ``` | |
| - **`exclude_document_ids`**: omit UUIDs of uploads you do not want in the match list (the UI passes the **current survey file** so hits focus on separate reference/guidance documents). | |
| - **`DELETE /documents/{document_id}`**: removes that file from disk, the database, and the vector index. **HTTP 409** if a report still references the file — start a new report or keep the file for traceability. | |
| **UI**: On the configure step, each section’s **Inspection Notes** toolbar has **Check similar**. The modal supports keeping notes, merging library or peer wording, replacing notes, or **removing an outdated upload** from the library. | |
| This is **RAG corpus hygiene**, not LLM fine-tuning: updating or removing indexed documents changes what retrieval sees on the next generation. | |
| --- | |
| ## Environment variables | |
| | Variable | Default | Description | | |
| |----------|---------|-------------| | |
| | `OPENAI_API_KEY` | *(required)* | OpenAI secret key | | |
| | `FAISS_INDEX_PATH` | `~/.report_genius/faiss_index` | Directory for the persisted FAISS index | | |
| | `DATABASE_URL` | `sqlite+aiosqlite:///./dev.db` | SQLAlchemy async DB URL | | |
| | `UPLOAD_DIR` | `/mnt/data/uploads` | Where uploaded files are stored | | |
| | `CACHE_DIR` | `/tmp/section_cache` | Section output cache directory | | |
| | `MAX_CONTEXT_TOKENS` | `400` | Max tokens fed to LLM as context | | |
| | `MAX_OUTPUT_TOKENS` | `300` | Max tokens in LLM response | | |
| | `RETRIEVAL_TOP_K` | `10` | Vector search top-k | | |
| | `RERANK_TOP_N` | `3` | Snippets kept after reranking | | |
| | `CHUNK_SIZE` | `500` | Target chunk size in tokens | | |
| | `CHUNK_OVERLAP` | `75` | Overlap between consecutive chunks | | |
| | `DEV_MODE` | `false` | Enables SQL echo and dev endpoint | | |
| | `TENANT_SECRET_KEY` | *(required in prod)* | Used to sign tenant tokens | | |
| --- | |
| ## Running tests | |
| ```bash | |
| # All tests with coverage | |
| pytest | |
| # Specific acceptance tests | |
| pytest app/tests/test_isolation.py -v # tenant isolation | |
| pytest app/tests/test_generator.py -v # non-invention + token limit | |
| pytest app/tests/test_cache.py -v # cache prevents duplicate LLM calls | |
| pytest app/tests/test_content_similar_api.py -v # similar-content API | |
| ``` | |
| Tests run in < 2 minutes (all LLM and embedding calls are mocked). | |
| --- | |
| ## Linting & type-checks | |
| ```bash | |
| black --check . | |
| ruff check . | |
| mypy app/ | |
| ``` | |
| --- | |
| ## Incremental milestones | |
| | Milestone | Description | | |
| |-----------|-------------| | |
| | M1 | Upload + DOCX/PDF parser → chunks saved to disk | | |
| | M2 | Chunker + FAISS index + mock embeddings | | |
| | M3 | Retriever + reranker + generator (mock OpenAI) | | |
| | M4 | Real OpenAI calls + section cache | | |
| | M5 | CI, security middleware, acceptance tests | | |
| --- | |
| ## Non-invention guarantee | |
| The LLM is always prompted with: | |
| > "Do not invent facts. Use only facts present in the bullets or retrieved_snippets. | |
| > If a fact is missing or unverifiable, mark it **[VERIFY]**." | |
| A post-processor additionally scans the output for any numeric fact or named entity | |
| not present in the source bullets or retrieved snippets, and automatically wraps it | |
| with `[VERIFY]`. These items are flagged for human review before the report is | |
| finalised. | |
| ### Inline section editing | |
| The results grid renders each generated section in a `.result-text` | |
| `contenteditable="plaintext-only"` block. Edits are persisted server-side via | |
| `PATCH /reports/{report_id}/sections/{section_code}` (body | |
| `{ "text": "..." }`, max 200 KB UTF-8, empty string allowed). Saves are | |
| debounced **700 ms** after the last keystroke, fire immediately on **blur**, | |
| and on **Cmd/Ctrl+S** (the browser's save-page dialog is suppressed). An | |
| in-flight request is aborted if the user resumes typing before it completes. | |
| The card header carries a small status pill that walks through: | |
| `✎ Editable → ⏳ Saving… → ✓ Saved` (or `⚠ Save failed` on a non-2xx response). | |
| When a save succeeds the backend stamps `meta.user_edited = true` inside the | |
| section's persisted `provenance` JSON; existing `provenance.sources`, `mode`, | |
| `confidence`, `ai_level`, `ai_percent`, `ai_transparency`, `style_profile`, | |
| `pipeline`, `fallback_used`, and `inspector` metadata are preserved. The | |
| client also updates `state.results[code].text` in place, so a subsequent | |
| **Re-Generate** of the same section sends the edited body as | |
| `draft_paragraph` rather than the original LLM output. | |
| --- | |
| ## Personalisation guarantees (per user / tenant) | |
| This system personalises output **per tenant** and **per report request**. | |
| ### 1) User isolation (who the model learns from) | |
| - Retrieval is always filtered by `tenant_id`. | |
| - Writing-style analysis samples only that tenant's indexed content. | |
| - Style profile cache is keyed by tenant, so one user's style never leaks to another. | |
| ### 2) Style personalisation (how it writes) | |
| For generation, the pipeline detects and applies a `WritingStyleProfile`: | |
| - tone | |
| - formality level | |
| - sentence complexity | |
| - vocabulary level | |
| - common phrases | |
| - structural patterns | |
| These fields are injected into prompts so sections match the user's own writing voice. | |
| ### 3) Preferences and choices (how much AI is allowed) | |
| The frontend AI scale (`ai_level` 1..5) controls behaviour: | |
| - **1 (RAG only)**: minimal rewrite, no notes-expansion, no style-profile injection. | |
| - **2..4 (light to strong)**: increasing rewrite strength and style matching. | |
| - **5 (full AI)**: strongest style adaptation and paraphrasing. | |
| `ai_level` is applied in all three modes: | |
| - `generate`: controls temperature + creativity hint + expansion behaviour. | |
| - `proofread`: controls edit intensity (conservative at low levels, stronger at high). | |
| - `enhance`: controls technical-expansion intensity and prose freedom. | |
| ### 4) Priorities (what the model should preserve first) | |
| Prompt and post-processing priorities are fixed in this order: | |
| 1. Preserve user facts (especially numbers/measurements/dates/addresses). | |
| 2. Use retrieved evidence from the same tenant. | |
| 3. Match user style/profile. | |
| 4. Mark uncertain/unsupported claims with `[VERIFY]`. | |
| This means user-provided facts and evidence are prioritised above stylistic flourish. | |
| ### 5) Current scope | |
| Current personalisation is driven by: | |
| - uploaded reference documents (style + evidence), | |
| - section bullets / extracted notes (user intent), | |
| - selected mode (`generate`, `proofread`, `enhance`), | |
| - selected AI scale (`ai_level`). | |
| There is currently no separate long-term "user preference profile" store beyond | |
| tenant style analysis and request-level controls above. | |
| --- | |
| ## Licence | |
| MIT — see `LICENSE`. | |