AI Agent commited on
Commit ·
b584c76
1
Parent(s): 8aa24f9
Update docs with feedback and resumability features
Browse files
NITDAA_ARCHITECTURE_DESIGN.md
CHANGED
|
@@ -7,14 +7,16 @@ Nitdaa is the mobile-friendly MVP sibling to the full HealthExpert enterprise pl
|
|
| 7 |
## 2. Core Differences from Base HealthExpert
|
| 8 |
- **UI/UX**: Transitioned from a 3-pane desktop layout to a single-pane vertical mobile layout.
|
| 9 |
- **Admin Constraints**: Admin controls (Kill Switch, DB Purge) are entirely hidden in the HTML to prevent public tampering.
|
| 10 |
-
- **Session Audit**: Uses `nitdaa_sessions.json` and `nitdaa_summary.json` for granular system telemetry rather than database-backed logs.
|
|
|
|
|
|
|
| 11 |
|
| 12 |
## 3. Data Flow
|
| 13 |
1. **Startup & Sync**: Flask initializes. `sync_he_data()` runs asynchronously to pull the latest dataset from `Sam-max1/he-data`. If differences are found, it gracefully purges vector/graph databases and rebuilds. Concurrently, it performs a 2-way remote synchronization of logs with `Sam-max1/mat_data`.
|
| 14 |
2. **Security & Connection**: Client connections face an immediate Math CAPTCHA. All API calls pass through `Flask-Limiter` and Strict HTTP Content Security Policies.
|
| 15 |
3. **Ingestion**: If new files are detected, `process_document_pipeline` splits them into 512-token chunks, vectorizes them into ChromaDB, and extracts graph entities locally using `spaCy`.
|
| 16 |
-
4. **Retrieval**: When a query hits `/api/query`,
|
| 17 |
-
5. **Synthesis**: A Cross-Encoder reranks the chunks, and the context is fed into
|
| 18 |
|
| 19 |
## 4. Resource Allocation
|
| 20 |
Nitdaa is tuned for the HuggingFace Spaces free tier:
|
|
|
|
| 7 |
## 2. Core Differences from Base HealthExpert
|
| 8 |
- **UI/UX**: Transitioned from a 3-pane desktop layout to a single-pane vertical mobile layout.
|
| 9 |
- **Admin Constraints**: Admin controls (Kill Switch, DB Purge) are entirely hidden in the HTML to prevent public tampering.
|
| 10 |
+
- **Session Audit**: Uses `nitdaa_sessions.json` and `nitdaa_summary.json` for granular system telemetry rather than database-backed logs. Features an integrated UI feedback loop that captures user sentiment (thumbs up/down) directly into the telemetry file.
|
| 11 |
+
- **Resumable Event Streaming**: The architecture uses an asynchronous Job ID system (`/api/query/start` and `/api/query/stream`) to survive mobile background disconnections.
|
| 12 |
+
- **Concurrent RAG Isolation**: Vector and Graph retrieval pipelines operate inside a Thread Pool with thread-local database connections, allowing multi-user concurrency without global UI locking.
|
| 13 |
|
| 14 |
## 3. Data Flow
|
| 15 |
1. **Startup & Sync**: Flask initializes. `sync_he_data()` runs asynchronously to pull the latest dataset from `Sam-max1/he-data`. If differences are found, it gracefully purges vector/graph databases and rebuilds. Concurrently, it performs a 2-way remote synchronization of logs with `Sam-max1/mat_data`.
|
| 16 |
2. **Security & Connection**: Client connections face an immediate Math CAPTCHA. All API calls pass through `Flask-Limiter` and Strict HTTP Content Security Policies.
|
| 17 |
3. **Ingestion**: If new files are detected, `process_document_pipeline` splits them into 512-token chunks, vectorizes them into ChromaDB, and extracts graph entities locally using `spaCy`.
|
| 18 |
+
4. **Retrieval (Concurrent)**: When a query hits `/api/query/start`, an asynchronous thread pool handles Tri-modal Hybrid Retrieval (Vector + BM25 + Graph) allowing multiple users to fetch context simultaneously.
|
| 19 |
+
5. **Synthesis & Feedback**: A Cross-Encoder reranks the chunks, and the context is fed into the local LLM generation queue. The frontend streams events using Server-Sent Events with offset recovery. Upon completion, users optionally submit ratings securely stored in `nitdaa_summary.json`.
|
| 20 |
|
| 21 |
## 4. Resource Allocation
|
| 22 |
Nitdaa is tuned for the HuggingFace Spaces free tier:
|
NITDAA_HEALTHEXPERT_USER_GUIDE.md
CHANGED
|
@@ -27,8 +27,10 @@ When a question is submitted:
|
|
| 27 |
- The system will display a **"Generating answer.. (1.2s)"** timer.
|
| 28 |
- Once context is retrieved from the Vector and Graph databases, the LLM will stream the Markdown-formatted answer back to the screen.
|
| 29 |
- Analysis may take upto 300 seconds due to slow shared resources. Please keep patience.
|
|
|
|
| 30 |
- Responses remain preserved in the scrollable window for the duration of your session.
|
| 31 |
|
| 32 |
## 5. Security & Session Handling
|
| 33 |
-
- **Multi-tenant Isolation**: Your conversation is strictly isolated from other users using an invisible `X-Session-Token`.
|
|
|
|
| 34 |
- **Session Expiration**: If the page is reloaded, or if you are inactive for an extended period, your session token resets, and previous context is cleared to ensure privacy.
|
|
|
|
| 27 |
- The system will display a **"Generating answer.. (1.2s)"** timer.
|
| 28 |
- Once context is retrieved from the Vector and Graph databases, the LLM will stream the Markdown-formatted answer back to the screen.
|
| 29 |
- Analysis may take upto 300 seconds due to slow shared resources. Please keep patience.
|
| 30 |
+
- **Providing Feedback**: Once an answer completes, you will see a Thumbs Up (👍) / Thumbs Down (👎) feedback interface along with an optional text field. Please submit your feedback to help us improve the system.
|
| 31 |
- Responses remain preserved in the scrollable window for the duration of your session.
|
| 32 |
|
| 33 |
## 5. Security & Session Handling
|
| 34 |
+
- **Multi-tenant Isolation**: Your conversation is strictly isolated from other users using an invisible `X-Session-Token`. Multiple users can query the system concurrently without interference.
|
| 35 |
+
- **Resumable Mobile Sessions**: If your mobile device goes to sleep or you navigate to another app, the system safely buffers your answer. When you return, the streaming will seamlessly resume without missing a word.
|
| 36 |
- **Session Expiration**: If the page is reloaded, or if you are inactive for an extended period, your session token resets, and previous context is cleared to ensure privacy.
|
NITDAA_HEALTHEXPERT_USER_GUIDE.pdf
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b2b3bd20ecbf623f2360d4b792d73d125b912172d2fa99a95d9eb0b36c3e0f7d
|
| 3 |
+
size 65046
|