ARCHITECTURE.md

Author RAG — Architecture Reference

---

🤖 AI AGENT MANDATORY PROTOCOL

THIS BLOCK IS FOR AI AGENTS (including Antigravity / Gemini). FOLLOW ON EVERY PROMPT.

On EVERY new request you must:

1. READ this file first before reading any source file or writing any code.
2. CHECK §9 Invariants — verify your planned change does not violate any rule.
3. CHECK §8 Where to Add Features — confirm you are editing the correct file for the task.
4. After completing any change that adds a file, route, model column, config value, or service — UPDATE this document to reflect the new state.
5. Push the updated ARCHITECTURE.md in the same commit as the code change. They must never go out of sync.

What counts as a required update:

Change made	Section to update
New file created	§4 Full File Tree
New route added	§4 (correct sub-section)
New DB column added	§4 Models table + §8
New config value added	§7 Configuration
New invariant established	§9 Invariants
New feature area added	§8 Where to Add Features
Architecture restructured	§3 Layer diagram + §4

NEVER do this:

• Edit source files without checking §9 first.
• Add a new file without adding it to §4.
• Define a new guard/detector outside guards.py.
• Add prompt text outside prompter.py.
• Skip updating this document after a structural change.

---

Purpose: This document is the single source of truth for any AI agent or human developer joining this codebase. Read this file first before opening any source file. It answers: What does each file do? How does a request flow? Where do I add X?

---

1. What This System Does

Author RAG is a multi-tenant SaaS platform. Each customer (an author) gets an AI-powered chatbot that lives on their website and answers questions about their books. The chatbot is a persuasive sales tool — it understands reader intent, retrieves precise passages from the book, and surfaces purchase links at the right moment.

Key numbers:

• Runtime: Python 3.11, FastAPI async
• Database: SQLite (dev) / PostgreSQL (prod) via SQLAlchemy async
• Vector store: ChromaDB (local persistent)
• LLM: OpenAI GPT-4o-mini (configurable)
• Embeddings: text-embedding-3-small

---

2. Top-Level Directory Layout

Author RAG/
├── app/                    ← All application code (see §4)
├── static/                 ← Compiled frontend (admin SPA, widget JS)
├── requirements.txt        ← Python dependencies
├── Dockerfile              ← HuggingFace Spaces deployment
├── ARCHITECTURE.md         ← THIS FILE
└── .env                    ← Secrets (never committed)

---

3. Layer Architecture

The codebase follows a strict 5-layer architecture. Data only flows downward. A layer may never import from a layer above it.

┌─────────────────────────────────────────────────────┐
│  Layer 1 — API / Routers                            │
│  app/api/, app/admin/routers/, app/superadmin/      │
│  • Accepts HTTP requests                            │
│  • Validates input (Pydantic schemas)               │
│  • Delegates to services — no business logic here   │
├─────────────────────────────────────────────────────┤
│  Layer 2 — Services / Pipeline                      │
│  app/services/                                      │
│  • All business logic lives here                    │
│  • The RAG pipeline is the core of this layer       │
│  • May call repositories and other services         │
├─────────────────────────────────────────────────────┤
│  Layer 3 — Repositories                             │
│  app/repositories/                                  │
│  • All database queries live here                   │
│  • Returns model objects — never raw SQL            │
│  • No business logic                                │
├─────────────────────────────────────────────────────┤
│  Layer 4 — Models                                   │
│  app/models/                                        │
│  • SQLAlchemy ORM table definitions only            │
│  • No methods with logic                            │
├─────────────────────────────────────────────────────┤
│  Layer 5 — Core / Infrastructure                    │
│  app/core/, app/config.py, app/dependencies.py      │
│  • DB engine, Redis, ChromaDB clients               │
│  • JWT / token crypto                               │
│  • No domain logic                                  │
└─────────────────────────────────────────────────────┘

---

4. Full File Tree with Descriptions

app/ — Root

File	Role
main.py	FastAPI app factory. Registers middleware, exception handlers, routers. Startup delegates to core/startup/.
config.py	All env-var settings via Pydantic BaseSettings. One source of truth for every config value.
dependencies.py	FastAPI Depends() providers: DB session, Redis, auth guards (get_current_author_scoped, get_subscription_author, get_current_superadmin).

---

app/core/ — Infrastructure

File	Role
core/startup/db.py	init_db() — creates all tables + runs incremental ALTER TABLE migrations on startup.
core/startup/seeder.py	seed_superadmin() — bootstraps the superadmin account from env vars on first run.
core/access/token_crypto.py	Creates and verifies signed subscription tokens (widget embed).
core/access/subscription.py	Subscription validation logic (expiry, budget, revocation).
core/access/jwt_blacklist.py	Redis-backed JWT revocation list.
core/access/totp.py	TOTP 2FA for SuperAdmin login.
core/chroma_client.py	Singleton ChromaDB client.
core/security.py	hash_password(), verify_password().

---

app/api/ — Public Widget API

These routes are called by the embedded widget on the author's website. All require X-Subscription-Token header (validated via get_subscription_author).

File	Routes	Role
api/chat.py	POST /api/chat/{slug}	Main chat endpoint — calls run_pipeline().
	POST /api/chat/{slug}/session/init	Creates a new visitor session.
	GET /api/chat/{slug}/session/history/{id}	Returns chat history.
	POST /api/chat/{slug}/session/farewell	Graceful session close + summarization.
	POST /api/chat/{slug}/session/rate	Star rating.
	POST /api/chat/{slug}/track-click	Records buy-link click.
	POST /api/chat/{slug}/feedback	👍/👎 message feedback.
	GET /api/chat/{slug}/events	SSE stream for ingestion progress.
api/ingest.py	POST /api/admin/{slug}/books/upload	Book PDF upload + async embedding pipeline.
api/widget.py	POST /api/widget/token	Returns a signed subscription token for the embed script.
api/schemas_router.py	/api/auth/*	Login, register, token refresh.

---

app/admin/ — Author Admin API

Prefix: /api/admin/{author_slug}/ Auth: Bearer JWT (get_current_author_scoped)

File	Routes	Role
admin/router.py	—	20-line aggregator. Mounts all 7 sub-routers.
admin/routers/dashboard.py	/sessions, /sessions/search, /sessions/{id}/transcript, /sessions/{id}/block, /sessions/{id}/messages/{mid}/annotate	Session management and message review.
admin/routers/books.py	/books, /books/{id}, /books/{id}/cover	Book list, delete, cover upload.
admin/routers/analytics.py	/analytics, /analytics/funnel, /analytics/intents, /analytics/visitors, /analytics/geo, /analytics/devices, /analytics/sessions/stats	Dashboard charts, funnel data, visitor analytics.
admin/routers/settings.py	/password, /widget-config, /profile, /personality, /notifications, /embed-token, /token-usage	All author account settings.
admin/routers/links.py	/smart-links/{book_id}	Buy/preview URL management.
admin/routers/qa.py	/qa, /qa/import, /qa/export, /qa/{id}	Custom Q&A training data (CRUD + CSV).
admin/routers/exports.py	/export/sessions, /export/analytics, /export/conversations	CSV data exports.

---

app/superadmin/ — SuperAdmin API

Prefix: /api/super/ Auth: Bearer JWT + TOTP (get_current_superadmin)

File	Routes	Role
superadmin/router.py	—	15-line aggregator. Mounts 3 sub-routers.
superadmin/routers/platform.py	/, /diag, /health, /audit, /announce, /backup	Platform diagnostics, audit log, announcements.
superadmin/routers/authors.py	/authors, /authors/{id}, /authors/{id}/suspend, /authors/{id}/grant, /authors/{id}/embed-token	Full author account lifecycle.
superadmin/routers/grants.py	/grants, /grants/{id}/revoke, /grants/{id}/bonus-tokens, /grants/{id}/extend, /grants/{id}/reset-tokens, /grant	Subscription token budget management.
superadmin/routers/_utils.py	—	Shared _err() error handler (no routes).

---

app/services/pipeline/ — The RAG Pipeline ⭐

This is the core of the product. Every visitor message flows through run_pipeline().

app/services/pipeline/
├── __init__.py       ← Public API: run_pipeline, PipelineResult, invalidate_book_cache
├── core.py           ← 12-step orchestrator (the conductor, ~200 lines)
├── generation.py     ← Steps 8–10: LLM call + faithfulness retry + safety scrub
├── handlers.py       ← Short-circuit response functions (greeting, catalog, piracy...)
├── helpers.py        ← Pure stateless utilities (call_llm, format_history, get_book_links...)
├── guards.py         ← Boolean detectors (is_greeting, is_full_story_request, ...) — SINGLE SOURCE OF TRUTH
├── cache.py          ← LRU answer cache (256 slots, MD5 key)
└── dedup.py          ← Chunk deduplication by Jaccard overlap

The 12-Step Pipeline (core.py)

Step 0   sanitize_user_input()         Strip dangerous/empty input
Step 1   check_boundary()              Jailbreak / piracy / off-topic guard
Step 1.5 check_custom_qa()            Short-circuit: exact Q&A match (no LLM)
Step 2   classify_intent()            3-tier: rules → cache → LLM-as-fallback
Step 3   Book resolution              Greeting / catalog / book-select short-circuits
         LRU cache lookup             Skip steps 4-12 on cache hit
Step 4   rewrite_query()              Generate 3 query variations for multi-pass retrieval
Step 5   retrieve_chunks()            ChromaDB vector search (top_k=15 default)
Step 6   rerank_chunks()              Cross-encoder re-ranking (top_n=5 default)
Step 6.5 deduplicate_chunks()         Remove near-duplicate overlapping windows
Step 7   build_context()              Token-aware context assembly (max 2000 tokens)
Step 8   MASTER_SYSTEM_PROMPT.format  Assemble full prompt with history + interest + context
         call_llm()                   OpenAI chat completions
Step 9   check_faithfulness()         NLI guardrail — retry with stricter prompt if hallucinating
Step 10  scrub_unsafe_response()      Output safety scrub + is_response_safe() check
Step 11  UpsellEngine.select_strategy() Choose upsell strategy based on intent + session state
Step 12  ResponseFormatter.format()   Assemble final JSON response with optional buy button
         cache_set()                  Store result for future identical questions

Where to look for what:

Task	File
Change the LLM model or temperature	config.py → OPENAI_CHAT_MODEL, RAG_TEMPERATURE
Edit the system prompt	services/prompter.py
Change what counts as a greeting	services/pipeline/guards.py → _GREETINGS
Change upsell logic	services/upsell_engine.py
Change faithfulness threshold	services/faithfulness.py
Change how queries are rewritten	services/rewriter.py
Change intent labels / rules	services/intent.py
Change chunking strategy	services/chunker.py
Change re-ranking	services/reranker.py
Change context token limit	config.py → RAG_CONTEXT_MAX_TOKENS

---

app/services/ — All Other Services

File	Role
rag_pipeline.py	Compatibility shim only. Imports from pipeline/ and re-exports.
intent.py	3-tier intent classifier: keyword rules → session cache → LLM fallback.
rewriter.py	Query rewriter — generates 3 variations using conversation context.
guardrails.py	Input sanitization, boundary checks, output safety, response scrubbing.
prompter.py	All prompt templates: MASTER_SYSTEM_PROMPT, boundary responses, upsell hooks.
faithfulness.py	NLI-based faithfulness scoring against retrieved context.
formatter.py	ResponseFormatter — assembles the final JSON {text, links, has_links} dict.
upsell_engine.py	UpsellEngine — selects upsell strategy, decides if link should show.
context_builder.py	Token-aware context string builder from ranked chunks.
vector_store.py	ChromaDB retrieval — multi-query search + score filtering.
reranker.py	Cross-encoder re-ranking of retrieved chunks.
embeddings.py	Generates and stores OpenAI embeddings into ChromaDB.
chunker.py	Splits book text into overlapping sliding-window chunks.
parser.py	Extracts clean text from PDF uploads.
auth_service.py	Register, login, token refresh logic.
superadmin_service.py	Business logic for author lifecycle (suspend, delete, grant).
token_budget.py	Token usage tracking and budget enforcement.
email_service.py	Transactional email sending.
rate_limiter.py	Per-IP / per-author rate limiting.
file_validator.py	PDF upload validation (size, MIME type, header).
session_store.py	Redis-backed session state persistence.
summarizer.py	End-of-session conversation summarizer.
notifications.py	In-app notification helpers.
analytics.py	Analytics event recording facade.

services/analytics_core/

| File | Role | |------||------| | tracker.py | Records AnalyticsEvent rows after each chat turn. | | aggregator.py | Pre-aggregates daily analytics for dashboard charts. | | geo.py | IP → country/region/city lookup (MaxMind GeoLite2). Exposes get_real_ip() (proxy-aware). | | visitor_tracker.py | record_visitor() — upserts Visitor table on every session/init. Deduplicates via localStorage UUID then fingerprint fallback. |

services/session_core/

File	Role
manager.py	SessionManager + SessionContext — the session state object passed through the pipeline.
context.py	Interest score and tag tracking across turns.
fingerprint.py	Visitor fingerprint generation (SHA-256, no PII).

---

app/models/ — Database Tables

File	Table	Key columns
user.py	users	id, email, role (author/superadmin), bot_name, chatbot_is_active
book.py	books	id, author_id, title, status, chroma_collection_id, chunk_count, buy_url
chat_session.py	chat_sessions, chat_messages	Session: visitor_fingerprint, visitor_uid, turn_count, rating, blocked. Message: role, intent, faithfulness_score, hallucination_detected
client_access.py	client_access	author_id, plan, token_budget, tokens_used, expires_at, is_revoked
custom_qa.py	custom_qa	author_id, question, answer, match_threshold, priority, match_count
analytics.py	analytics_events	author_id, session_id, intent, link_clicked, prompt_tokens, response_ms
link.py	links	author_id, book_id, purchase_url, preview_url
document.py	documents	book_id, filename, status, pages
visitor.py	visitors	author_id, visitor_uid (UUID, unique per author), fingerprint, first_seen, last_seen, page_views, country_code, region, city, device_type, browser, os

---

app/repositories/ — Database Access Layer

| File | Role | |------||------| | base.py | BaseRepository — shared get_by_id, save, delete methods. | | user_repo.py | UserRepository — lookup by email/slug, list all authors. | | book_repo.py | BookRepository — list_active_for_author(), list_for_author(). | | access_repo.py | AccessRepository — get_active_for_author() — used everywhere for budget checks. | | link_repo.py | LinkRepository — get_for_book(), upsert_for_book(). | | audit_repo.py | AuditRepository — append-only log(), list_recent(). | | document_repo.py | DocumentRepository — book document tracking. | | visitor_repo.py | VisitorRepository — get_by_uid(), get_by_fingerprint(), create(), touch(), distribution queries for analytics. |

---

app/schemas/ — Pydantic Request/Response Schemas

File	Role
chatbot.py	ChatRequest, ChatResponse, SessionInitResponse, FarewellRequest
admin.py	AnnotateRequest, FlagRequest, PasswordChangeRequest, ProfileUpdate, WidgetConfigUpdate
superadmin.py	CreateAuthorRequest, GrantAccessRequest, RevokeAccessRequest, AddBonusTokensRequest
auth.py	LoginRequest, RegisterRequest, TokenResponse

---

app/middleware/ — Request Middleware

File	Role
logging_middleware.py	Structured request/response logging with structlog.
rate_limit_middleware.py	Sliding-window rate limiting (configurable per route type).
security_headers.py	CSP, HSTS, X-Frame-Options headers.
metrics.py	Prometheus metrics collection (optional).

---

app/tasks/ — Background Tasks

File	Role
ingestion_task.py	Async book processing: parse → chunk → embed → store in ChromaDB.
analytics_task.py	Periodic analytics aggregation.
email_task.py	Queued email sending.
backup_task.py	SQLite + ChromaDB backup.
expiry_check_task.py	Subscription expiry warnings.
geo_update_task.py	Retroactive geo enrichment for sessions.
link_health_task.py	Checks buy links for 404s.
celery_app.py	Celery app instance (optional — tasks can run inline).

---

5. A Complete Request Trace

Scenario: A reader on an author's website asks "Is the ending happy?"

Browser (widget JS)
  │  POST /api/chat/{author_slug}
  │  Headers: X-Subscription-Token: <signed JWT>
  │  Body: { "message": "Is the ending happy?", "session_id": "..." }
  ▼
app/api/chat.py → chat()
  │  1. get_subscription_author()      Verify token, check budget, load author
  │  2. SessionManager.load()          Load session from Redis (history, book selection)
  │  3. run_pipeline(query, author, session_context, db)
  ▼
app/services/pipeline/core.py → run_pipeline()
  │  Step 0:  sanitize_user_input()       → "Is the ending happy?"
  │  Step 1:  check_boundary()            → no violation
  │  Step 1.5: check_custom_qa()          → no match
  │  Step 2:  classify_intent()           → intent="question", confidence=0.92
  │  Step 3:  BookRepository.list_active_for_author()  → [Book("The Last Signal")]
  │           is_greeting()              → False
  │           is_catalog_question()      → False
  │           LRU cache lookup           → miss
  │  Step 4:  rewrite_query()            → ["Is the ending happy?", "How does it end?", "resolution of the story"]
  │  Step 5:  retrieve_chunks()          → 15 chunks from ChromaDB
  │  Step 6:  rerank_chunks()            → top 5 chunks scored by cross-encoder
  │  Step 6.5: deduplicate_chunks()      → 4 unique chunks
  │  Step 7:  build_context()            → 1,840 tokens of context
  ▼
app/services/pipeline/generation.py → generate_response()
  │  Step 8:  MASTER_SYSTEM_PROMPT.format(...)
  │           call_llm()                 → "Without giving anything away, I can say..."
  │  Step 9:  check_faithfulness()       → faithful=True, score=0.91
  │  Step 10: is_response_safe()         → True
  │           return (response, 0.91, False, 312, 87)
  ▼
core.py (continues)
  │  Step 11: UpsellEngine.select_strategy()   → "SOFT_MENTION"
  │           should_include_link()            → True (turn 4, high interest)
  │           get_book_links()                 → purchase_url="https://amazon.com/..."
  │  Step 12: ResponseFormatter.format()       → {text, links:[{label:"Get the book", url:...}], has_links:True}
  │           cache_set()                      → stored for next identical query
  ▼
app/api/chat.py (continues)
  │  SessionManager.save()             Update session: turn_count++, history appended
  │  analytics.record_event()          Log intent, tokens, latency, link_shown
  │  token_budget.deduct()             Deduct prompt+completion tokens from grant
  ▼
HTTP 200 { "message": "Without giving anything away...", "links": [...] }
  ▼
Browser (widget JS)
  Renders response + "Get the book" button

---

6. Authentication Model

Three distinct auth tiers with separate dependencies:

Tier 1: Visitor (Widget)
  → Header: X-Subscription-Token (signed JWT, no login needed)
  → Dependency: get_subscription_author()
  → Checks: token signature, expiry, token budget, author active

Tier 2: Author (Admin)
  → Header: Authorization: Bearer <JWT>
  → Dependency: get_current_author_scoped()
  → Checks: JWT validity, author role, slug matches token

Tier 3: SuperAdmin
  → Header: Authorization: Bearer <JWT>
  → Dependency: get_current_superadmin()
  → Checks: JWT validity, superadmin role, TOTP verified

---

7. Key Configuration Values

All in app/config.py (loaded from .env):

Variable	Default	Effect
OPENAI_CHAT_MODEL	gpt-4o-mini	LLM used for response generation
OPENAI_EMBED_MODEL	text-embedding-3-small	Embedding model
RAG_RETRIEVAL_TOP_K	15	Chunks fetched from ChromaDB
RAG_RERANK_TOP_N	5	Chunks kept after re-ranking
RAG_RERANK_MIN_SCORE	0.3	Min cross-encoder score to keep
RAG_CONTEXT_MAX_TOKENS	2000	Max tokens in assembled context
RAG_MAX_RESPONSE_TOKENS	300	Max tokens in LLM response
RAG_TEMPERATURE	0.4	LLM temperature
RAG_BOOK_CONFIDENCE_THRESHOLD	0.7	Min score to route to a specific book
SUPERADMIN_EMAIL	—	Seeded on first startup
SUPERADMIN_PASSWORD	—	Seeded on first startup

---

8. Where to Add New Features

Task	Where to add it	Notes
New author admin route	app/admin/routers/<closest>.py	Or create a new file + mount in admin/router.py
New superadmin route	`app/superadmin/routers/<platform	authors
New intent label	app/services/intent.py → _RULE_MAP	Add keyword rules first; LLM only as fallback
New guard/detector	app/services/pipeline/guards.py	Never define is_greeting elsewhere
New short-circuit response	app/services/pipeline/handlers.py	Must return PipelineResult
New prompt template	app/services/prompter.py	Keep ALL prompt text in one file
New DB column	app/models/<model>.py + app/core/startup/db.py → _NEW_COLUMNS	No Alembic needed
New background task	app/tasks/<name>_task.py	Register in tasks/celery_app.py if async
New config value	app/config.py	Always typed, always has a default

---

9. Invariants — Rules That Must Never Be Broken

1. Services never import from routers. Data flows down only.
2. guards.py is the single source of truth for all boolean detectors. Never re-define is_greeting() elsewhere.
3. All prompt text lives in prompter.py. No inline f-strings with user-facing text in other files.
4. All pipeline results return PipelineResult. Never return a raw dict from run_pipeline().
5. Repositories never contain business logic. If it's a decision, it belongs in a service.
6. Never skip a pipeline step. Steps 0–12 run for every message; short-circuits return early from core.py, not by skipping steps.
7. Cache only cacheable intents. purchase_intent, complaint, greeting are never cached (time/person-sensitive).
8. Token usage must always be recorded. Every LLM call goes through call_llm() in helpers.py which returns token counts.
9. All new DB columns go into core/startup/db.py → _NEW_COLUMNS. This keeps migrations consistent across environments.
10. All exceptions have typed exception classes in app/exceptions/. Never raise HTTPException directly in service layer.
11. This document must be updated on every structural change. See §AI AGENT MANDATORY PROTOCOL above.

---

10. Change Log

AI agents: append an entry here after every significant change. Format: YYYY-MM-DD | What changed | Files affected.

Date	Change	Files Affected
2026-06-19	Initial project build — auth, chat API, admin panel, ingestion pipeline	All
2026-06-22	Intelligence overhaul — Python-first intent classifier, hybrid rewriter, LRU cache, chunk dedup, natural upsell	intent.py, rewriter.py, upsell_engine.py, rag_pipeline.py
2026-06-23	Modularity refactor Phase 1 — admin/router.py split into 7 sub-routers, rag_pipeline.py split into pipeline/ package	admin/router.py, admin/routers/*, services/pipeline/*, services/rag_pipeline.py
2026-06-23	Modularity refactor Phase 2 — superadmin/router.py split into 3 sub-routers, main.py startup extracted to core/startup/, pipeline/core.py generation extracted to pipeline/generation.py	superadmin/router.py, superadmin/routers/*, core/startup/*, services/pipeline/generation.py, main.py
2026-06-23	Added ARCHITECTURE.md with AI Agent Mandatory Protocol	ARCHITECTURE.md
2026-06-23	Production Improvements Phases 1–5: 8 bugs fixed, 7 features added. B1 fingerprint null crash, B2 silent truncation, B3 OpenAI client per-call, B4 no LLM retry, B5 piracy→jailbreak wrong intent, B6 history only 3 turns, B8 no budget warning, B9 sync ChromaDB in async loop. New: per-visitor rate limit, input cap, tenacity retry, intent rule expansion (+30% coverage), history fix, budget email warning, async health check.	config.py, middleware/rate_limit_middleware.py, api/chat.py, services/pipeline/core.py, services/pipeline/helpers.py, services/intent.py, dependencies.py, services/token_budget.py, requirements.txt
2026-06-23	Visitor Analytics System: Replaced session-based visitor counting with proper deduplication. New Visitor model (one row per browser per author), VisitorRepository, visitor_tracker.py service. session/init now accepts visitor_uid (localStorage UUID) in request body — HF-Spaces-safe (no cookies). 4 new analytics endpoints: /visitors, /geo, /devices, /sessions/stats. geo.py updated with region extraction and X-Real-IP header support.	models/visitor.py, repositories/visitor_repo.py, services/analytics_core/visitor_tracker.py, services/analytics_core/geo.py, api/chat.py, admin/routers/analytics.py, schemas/chatbot.py, models/chat_session.py, core/startup/db.py, models/__init__.py