docs/DESIGN.md

Cortex: Why FalkorDB for AI Engagements, and More (Design v0.2)

Status: DRAFT v0.3 for review · 2026-09-30 · Name: Cortex (confirmed) v0.2: Neo4j comparison dropped, PDF ingestion added, free LLM by default, PDF upload feature added. v0.3: role impersonation, tokenomics, PIN-protected keys, a consolidated "Limits & Restrictions" section, 4h idle reset, branding.

1. Goal

Show, with evidence people can test themselves, why FalkorDB is the right foundation for AI engagements, and what it adds beyond RAG.

• Main comparison: GraphRAG on FalkorDB vs plain vector RAG, the approach our environment already uses. Both run on the same PDFs, with the same questions and the same LLM. The only thing that differs is the graph.
• "And more": the same graph also supports things RAG can't do: fraud-ring detection, lineage and versioning, role-based guardrails, and many isolated graphs in one instance.
• The data: "Cortex Bank", a synthetic bank founded in 1926, with 12 departments.
• Audience: executives (the story and the impact) and engineers (how it works and whether it reproduces).
• Access: a public URL that people use on their own. No presenter or laptop needs to be available.
• Format: hybrid, a live walkthrough plus a recorded backup video. The video will be made once the build is nearly finished.
• Timeline: 2 days.

Out of scope for now: the Neo4j comparison (it can go in a later write-up if needed) and hosting on AWS (see §13).

2. Principles

1. Measure, don't claim. Every advantage shown comes with a result people can reproduce.
2. Fair comparison. Same PDFs, chunks, embeddings, LLM and prompts. Only the retrieval strategy differs.
3. Honest. Where plain RAG is just as good (for example, simple lookups of a single fact), we show that.
4. Free to run. A free LLM by default, free hosting, and local embeddings.
5. Reproducible. docker compose up on any laptop gives the same results.

3. How Content Gets In (hybrid)

Layer	Source	How it gets in	Size
Knowledge: policies, SOPs, guardrails, KPIs, audits, regulations	~60 synthetic PDFs (written by Claude, rendered as realistic bank documents)	LangChain pipeline: load → chunk → embed, and have the LLM extract entities and relationships → FalkorDB	~10–30k nodes
Operations: customers, accounts, transactions, devices, fraud rings	Structured data (banks keep this in databases, not PDFs)	Seeded Python generator → bulk load	~1M nodes / ~5M relationships
User uploads	Visitor's own PDF (≤5 MB, ≤20 pages)	Same pipeline, into a private sandbox graph for that session that is deleted after 24h	Small

PDF ─► LangChain loader ─► chunks ─┬─► embeddings (local model) ─► FalkorDB vector index
                                   └─► LLM entity/relation extraction (schema-guided) ─► FalkorDB graph
                                        (chunks linked to the entities they mention = evidence)

• Extraction is guided by a schema. The LLM may only produce the node and relationship types allowed in §4, and its output is validated before loading.
• The knowledge layer is pre-built. The 60 PDFs are ingested once at build time and the graph snapshot is bundled into the image, so start-up is fast and doesn't use any LLM quota. The pipeline can be re-run at any time.

4. Graph Model

Knowledge layer

Node	Examples / key properties
Document, Chunk	Source PDF, page, text, embedding (the evidence)
Department	Retail, Corporate & Commercial, Wealth, Treasury & Markets, Credit, Risk, Compliance (AML/KYC), Operations, IT & Cyber, HR, Legal, Internal Audit
Role, Person	Title, clearance level
SOP, Policy, Control	Version, effective dates, owner
Risk, Regulation	Basel III/IV, AML/KYC, GDPR, DORA, PCI-DSS, SOX, BCBS 239, EU AI Act
Guardrail	Separation of duties, maker-checker, AI-usage limits, data-access rules
KPI, KRI	Targets, thresholds, current values
System, AuditFinding, Entity (M&A history)	Legacy mainframe, acquired banks

Relationships: OWNS, IMPLEMENTS, MITIGATES, ENFORCES, SUPERSEDES, APPLIES_TO, CAN_ACCESS, MEASURES, RAISED_ON, MERGED_INTO, MENTIONS (Chunk→entity).

Operations layer: Customer, Account, Transaction, Device, IP, Address, Branch, Merchant, with fraud rings planted in the data and labelled with the right answers.

5. Synthetic Content (generated by Claude)

• ~60 PDFs across the 12 departments: policies, SOPs, guardrail standards, KPI frameworks, audit reports and regulatory summaries. They look realistic: letterhead, document IDs, version history tables, numbered sections and approval sign-offs.
• Deliberate cross-references between documents (for example, an SOP cites a policy, which cites a regulation). This is what makes questions that need several hops possible.
• Some legacy versions (such as a 1998 SOP replaced in 2019), so versioning and lineage can be shown.
• Source text is kept as Markdown in the repo, and the PDFs are rendered from it deterministically.

6. LLM & Embeddings

Use	Default (free)	Optional
Q&A and upload extraction	Google Gemini free tier (a Flash-class model), using the owner's key	Claude Sonnet 5.5 or Gemini, using the tester's own key
Embeddings	Small open-source model running locally on CPU	—

• Switching providers is a single setting (LangChain chat-model interface).
• Keys are protected by PINs (details in §11):
  • Owner's Gemini key: LLM features only unlock after entering an Access PIN that the owner shares with testers.
  • Tester's own key (BYOK): protected by a personal PIN the tester chooses.

7. Demo Scenarios

#	Scenario	What it proves
1	Regulatory impact: "An EU AI Act amendment lands. What's affected?"	Following 4–6 links through the graph: regulation → policies → controls → SOPs → teams → KPIs
2	Side by side: the same question answered by plain RAG and by GraphRAG, each showing its sources	Better answers on questions that need several hops or several documents
3	Role impersonation: the tester picks a role (see §7.1) and asks the same question; a compare mode shows two roles side by side	Access control and guardrails built into the graph
4	Upload your own PDF: see it turn into a graph and ask questions about it	Real ingestion, plus isolated graphs per session
5	Fraud rings across ~1M records	The "and more": graph analytics beyond RAG
6	Risk roll-up: one control → department KRI → a bank-wide score	Aggregating up a hierarchy
7	Version history: which SOP replaced which, and why	Lineage
8	Performance panel: response times for these queries on the 1M-node graph	Speed at a realistic scale

7.1 Role Impersonation

Testers choose who they are, and the same question gets different answers depending on what that role is allowed to see. Access is decided by graph paths (Role -[:CAN_ACCESS]-> Policy/System, Guardrail -[:APPLIES_TO]-> Role) and enforced before retrieval, so the LLM never sees content the role isn't allowed to see.

Role	Clearance	Sees
Branch Teller	L1	Retail SOPs, customer-facing policies
Relationship Manager (Wealth)	L2	Plus client suitability, wealth products
Credit Analyst	L2	Plus credit policy, limits, risk ratings
Compliance Officer (AML/KYC)	L3	Plus AML typologies, fraud-ring alerts, SAR procedures
Chief Risk Officer	L4	Everything, including bank-wide KRIs and board risk reports
Internal Auditor	L4 (read-only)	Everything, plus audit findings and control test results
External Contractor (IT)	L0	Only IT runbooks for their assigned systems

• A role badge is always visible. Every answer shows "Answered as: " and, where relevant, notes that N sources were withheld for this role (without revealing what they contain).
• Compare mode: one question, two roles, answers side by side, with the difference highlighted.
• This is a simulation for the demo: choosing a role is not authentication.

8. Evaluation (plain RAG vs GraphRAG)

• ~30 questions with known correct answers in 4 types: single fact, multi-hop, cross-document, aggregation/compliance.
• Two setups: (a) vector-only RAG (top-k chunks), (b) GraphRAG (vector entry points + graph traversal). Both use FalkorDB, so the only difference is whether the graph is used.
• Metrics:
  • answer accuracy (graded against the known answers)
  • completeness (were all affected items found?)
  • whether citations are correct
  • response time
  • tokenomics (see §8.1)
• Expected results, stated up front: similar on single-fact questions, and GraphRAG ahead on the other types. Results are published in the UI as they come out.

8.1 Tokenomics

Every answer shows its token economics, and the evaluation reports them in aggregate:

Metric	Meaning
Input / output tokens	What went into and came out of the LLM
Context efficiency	How many of the tokens sent were actually cited in the answer
Cost per answer	Tokens × list price of the selected model (shown as $0 on the free tier, with the equivalent paid cost next to it)
Cost per correct answer	Total cost ÷ correct answers. This is the headline number for executives
Tokens saved	GraphRAG vs vector RAG. Precise graph context typically means fewer, more relevant tokens (to be measured)

A running session total ("You've used X tokens ≈ $Y") appears in the header.

9. Architecture & Hosting

Build notes (v0.3 implementation):

• Hosting changed: Docker Spaces are paid on this account, so Cortex runs as a Gradio-SDK Space on ZeroGPU (free). app.py serves the FastAPI app on port 7860. FalkorDB runs embedded, using the redis-server and FalkorDB module shipped in the falkordblite package. The fallback, if the Space image can't run it, is the FalkorDB Cloud free tier.
• Deploy: a GitHub Action (.github/workflows/deploy-cortex-space.yml) pushes cortex/ to sk-aiu/cortex.
• Extraction: bank PDFs use pattern extraction (0 tokens; 295/295 relationships correct against the answer key). Uploads use LLM structured extraction.
• No LLM-written Cypher in v1: all Cypher is fixed and read-only, which is stricter than §11's rule.
• Operational graph size is chosen automatically from available RAM (≥12 GB: ~1M nodes; ≥6 GB: 50%; otherwise 20%).

      Hugging Face Docker Space (free, 16 GB RAM, 2 vCPU, port 7860)
 ┌─────────────────────────────────────────────────────────────┐
 │ supervisord                                                  │
 │  ├─ FalkorDB (knowledge graph + ops graph + sandbox graphs)  │
 │  └─ App: FastAPI + static web UI                             │
 │        ├─ LangChain pipelines (ingest, RAG, GraphRAG)        │
 │        ├─ Local embeddings (CPU)                             │
 │        └─ LLM gateway → Gemini (default) / Claude (BYO key)  │
 └─────────────────────────────────────────────────────────────┘

• Everything runs in one container, and the same image runs locally with docker compose up.
• Idle reset after 4 hours: if nobody uses the app for 4 hours, it resets itself. Sandbox graphs and chat sessions are cleared, memory is freed, and PIN sessions expire.
  • Note: on the free tier, Hugging Face decides when the container itself goes to sleep (about 48 hours idle), and that can't be changed without paid hardware. The 4-hour rule is enforced by our app.
  • Waking the container after it sleeps takes 1–3 minutes, and a "warming up" page is shown meanwhile.
• Removing Neo4j frees about 6 GB, which leaves plenty of room on 16 GB.

10. UI

• Executive view: landing page with key figures, one-click buttons for scenarios 1–8, a graph visualisation, and the "Ask Cortex" chat with side-by-side answers, source paths and a tokenomics strip for each answer.
• Role switcher in the header (§7.1), with compare mode.
• Engineer view: the raw Cypher behind every answer, the pipeline steps, evaluation results (CSV download), a link to the repo, and a settings panel for choosing the LLM or entering your own key.
• Stack: FastAPI backend and a lightweight static front end (Cytoscape.js from a CDN for the graph view).

11. Security, Cost, Limits & Restrictions

Every limit is shown to users where it applies: on the upload box, in the chat header, in the settings panel, and in a "Limits & fair use" page linked from the footer. Nothing is limited silently.

11.1 Key protection (PINs)

Key	Protection	Where the user sees it
Owner's Gemini key	Stored as a Space secret and never sent to the browser. LLM features need an Access PIN (6+ digits, shared by the owner, can be changed at any time). A valid PIN gives an 8-hour session	PIN prompt when the tester first asks a question
Tester's own key (BYOK)	Encrypted in the tester's browser with a personal PIN (WebCrypto AES-GCM, key derived with PBKDF2). Decrypted only when a request is sent, passed to the provider over HTTPS, never stored or logged on the server	Settings panel: "Your key is encrypted on this device. Use a key with a spending limit."
Failed PIN attempts	5 failures lead to a 15-minute lockout for that session or IP	Lockout message with a countdown

11.2 Limits & restrictions (the user-facing list)

Area	Limit / rule	Shown where
Sandbox graphs	Deleted 24 hours after upload, or at the 4-hour idle reset if that comes first	Countdown timer on the sandbox ("Wipes in 23h 12m")
Uploads	≤ 5 MB, ≤ 20 pages, PDF only, text-only extraction, max 3 uploads per session	On the upload box
Shared free LLM	Questions per session and per day (e.g. 20 / 200). BYOK has no app limit	Counter in the chat header ("12 of 20 left today")
Questions	Max 500 characters. Harmful or off-topic prompts are refused	Under the input box
LLM-generated queries	Read-only Cypher (write clauses blocked), 5-second timeout, 1,000-row limit	Engineer view, with the query
Role impersonation	A simulation, not authentication	Role badge tooltip
Data	100% synthetic. Never upload real customer or confidential data	Banner on the landing page and the upload box
Idle reset	After 4 hours without activity, sessions, sandboxes and PIN sessions are cleared	Footer note
Scripted scenarios	Answers are pre-computed: they cost nothing and work even if the LLM is down	"Cached answer" tag
Logging	Only questions and response times are logged. No keys, PINs or uploaded content	Limits & fair use page

11A. Name & Branding

• Name: Cortex. The name suggests the "intelligence layer" of the enterprise, and follows the IT convention of "Cortex" for intelligence, security and engineering platforms.
  • Caution: several commercial products already use "Cortex". That's fine for an internal demo, but check trademarks before any external or commercial use. The fictional bank is "Cortex Bank".
• Logo: an original mark. A "C" formed by an arc of connected graph nodes, with a bright node at its opening (knowledge → insight). Navy-to-orange gradient. Files: docs/brand/cortex-logo.svg / .png.
• Palette: navy #0B1B3A, blue #1E3A8A, orange #F97316, teal #0EA5A4 (for "success"), slate for text.

12. Repository Layout

cortex/
  docs/        DESIGN.md, DEMO_SCRIPT.md, EVAL.md, one-pager
  content/     source Markdown for the ~60 bank documents
  pdfs/        rendered PDFs (committed)
  generator/   pdf_render.py, ops_data.py (seeded, fraud injection)
  ingest/      LangChain pipeline: load, chunk, embed, extract, load to FalkorDB
  app/         FastAPI, RAG + GraphRAG chains, LLM gateway, static UI
  eval/        questions + known answers, runner, results
  deploy/      Dockerfile (HF Space), supervisord.conf, docker-compose.yml
  tests/

13. Future State (not now)

• AWS deployment (for example ECS, or EC2 with FalkorDB Cloud) once there's an AWS account and budget.
• Neo4j comparison as an optional write-up or benchmark appendix.
• Live external feed: daily AI and regulatory news ingested as Signal nodes that link to the policies they affect. This was the original "AI News" idea.

14. Two-Day Plan

When	Deliverable
Day 1 AM	Bank content (Markdown) → ~60 PDFs; ops data generator
Day 1 PM	LangChain ingestion into FalkorDB; graph snapshot; RAG and GraphRAG chains working
Day 2 AM	UI (executive and engineer views, role switcher, tokenomics), scenarios, uploads, PIN-protected keys, evaluation run
Day 2 PM	Hugging Face Space deploy, limits page, demo script, updated one-pager
After build	Recorded backup video, once everything is built and nearly ready

15. Risks

Risk	Mitigation
Free-tier LLM rate limits	Pre-built graph, cached scenario answers, BYO key
Imperfect LLM extraction	Schema-guided extraction, validation, spot checks against known answers
Space cold start	Warm-up page, fast snapshot load
Trust around BYO keys	Encrypted in the browser with a PIN, not logged, disclosure on the page, suggest limited keys
Access PIN leaked	Owner changes it in Space secrets; rate limits cap the damage
Hugging Face sleep can't be set to 4h on the free tier	App-level 4-hour idle reset (§9)
Licensing (FalkorDB is SSPL)	Flag to Legal before any production use

Open items: owner creates a free Gemini API key and a Hugging Face Space, and chooses the Access PIN (instructions will be provided).

16. Cradle of the Learning Portal

Cortex is where the owner's Learning Portal started (1 Oct 2026). After Cortex went live, the owner wanted to be able to explain it from scratch, at any depth, without reading long text. That became a separate product: a private, phone-first portal that turns every project into learning, in a loop: Build → Capture → Learn → Explore → Decide → Build.

Agreed brief (for the new repo; the portal's name = the repo's name, chosen by the owner):

Topic	Decision
Proof of concept	Cortex only; other projects join later
Device and privacy	iPhone first; private (private Hugging Face Space)
Formats	Audio episodes, 60-90 s narrated reels, swipe cards, 3-question checks, "explain it back"; text is the fallback
Voice	Kokoro-82M af_heart, same as the X-Ray film (docs/film/make_audio.py)
Structure	Concepts ↔ Projects ↔ Decisions, linked as a graph
Retention	Daily 5-minute spaced-repetition drill
Updates	Weekly scheduled run reads the repos (commits + DECISIONS.md) and refreshes learning packs; on demand too
Explore	Weekly tech radar (Adopt / Trial / Watch / Hold) for the technologies and techniques each project uses, with dated sources; the owner decides
Capture habit	Every project keeps a DECISIONS.md (decision, options, reason, lesson); see DECISIONS.md