| # Cortex: Why FalkorDB for AI Engagements, and More (Design v0.2) |
|
|
| > Status: **DRAFT v0.3 for review** · 2026-09-30 · Name: **Cortex** (confirmed) |
| > v0.2: Neo4j comparison dropped, PDF ingestion added, free LLM by default, PDF upload feature added. |
| > v0.3: role impersonation, tokenomics, PIN-protected keys, a consolidated "Limits & Restrictions" section, 4h idle reset, branding. |
|
|
| ## 1. Goal |
| Show, with evidence people can test themselves, why **FalkorDB** is the right foundation for AI |
| engagements, and what it adds beyond RAG. |
|
|
| - **Main comparison:** **GraphRAG on FalkorDB** vs **plain vector RAG**, the approach our environment already uses. |
| Both run on the same PDFs, with the same questions and the same LLM. The only thing that differs is the graph. |
| - **"And more":** the same graph also supports things RAG can't do: fraud-ring detection, lineage and versioning, |
| role-based guardrails, and many isolated graphs in one instance. |
| - **The data:** "Cortex Bank", a synthetic bank founded in 1926, with 12 departments. |
| - **Audience:** executives (the story and the impact) and engineers (how it works and whether it reproduces). |
| - **Access:** a public URL that people use on their own. No presenter or laptop needs to be available. |
| - **Format:** hybrid, a live walkthrough plus a recorded backup video. The video will be made once the build is nearly finished. |
| - **Timeline:** 2 days. |
|
|
| **Out of scope for now:** the Neo4j comparison (it can go in a later write-up if needed) and hosting on AWS (see §13). |
|
|
| ## 2. Principles |
| 1. **Measure, don't claim.** Every advantage shown comes with a result people can reproduce. |
| 2. **Fair comparison.** Same PDFs, chunks, embeddings, LLM and prompts. Only the retrieval strategy differs. |
| 3. **Honest.** Where plain RAG is just as good (for example, simple lookups of a single fact), we show that. |
| 4. **Free to run.** A free LLM by default, free hosting, and local embeddings. |
| 5. **Reproducible.** `docker compose up` on any laptop gives the same results. |
|
|
| ## 3. How Content Gets In (hybrid) |
|
|
| | Layer | Source | How it gets in | Size | |
| |---|---|---|---| |
| | **Knowledge**: policies, SOPs, guardrails, KPIs, audits, regulations | **~60 synthetic PDFs** (written by Claude, rendered as realistic bank documents) | **LangChain pipeline**: load → chunk → embed, and have the LLM extract entities and relationships → FalkorDB | ~10–30k nodes | |
| | **Operations**: customers, accounts, transactions, devices, fraud rings | Structured data (banks keep this in databases, not PDFs) | Seeded Python generator → bulk load | ~1M nodes / ~5M relationships | |
| | **User uploads** | Visitor's own PDF (≤5 MB, ≤20 pages) | Same pipeline, into a **private sandbox graph** for that session that is deleted after 24h | Small | |
|
|
| ``` |
| PDF ─► LangChain loader ─► chunks ─┬─► embeddings (local model) ─► FalkorDB vector index |
| └─► LLM entity/relation extraction (schema-guided) ─► FalkorDB graph |
| (chunks linked to the entities they mention = evidence) |
| ``` |
|
|
| - **Extraction is guided by a schema.** The LLM may only produce the node and relationship types allowed in §4, |
| and its output is validated before loading. |
| - **The knowledge layer is pre-built.** The 60 PDFs are ingested once at build time and the graph snapshot is |
| bundled into the image, so start-up is fast and doesn't use any LLM quota. The pipeline can be re-run at any time. |
|
|
| ## 4. Graph Model |
|
|
| **Knowledge layer** |
| | Node | Examples / key properties | |
| |---|---| |
| | `Document`, `Chunk` | Source PDF, page, text, embedding (the evidence) | |
| | `Department` | Retail, Corporate & Commercial, Wealth, Treasury & Markets, Credit, Risk, Compliance (AML/KYC), Operations, IT & Cyber, HR, Legal, Internal Audit | |
| | `Role`, `Person` | Title, clearance level | |
| | `SOP`, `Policy`, `Control` | Version, effective dates, owner | |
| | `Risk`, `Regulation` | Basel III/IV, AML/KYC, GDPR, DORA, PCI-DSS, SOX, BCBS 239, EU AI Act | |
| | `Guardrail` | Separation of duties, maker-checker, AI-usage limits, data-access rules | |
| | `KPI`, `KRI` | Targets, thresholds, current values | |
| | `System`, `AuditFinding`, `Entity` (M&A history) | Legacy mainframe, acquired banks | |
|
|
| Relationships: `OWNS`, `IMPLEMENTS`, `MITIGATES`, `ENFORCES`, `SUPERSEDES`, `APPLIES_TO`, `CAN_ACCESS`, |
| `MEASURES`, `RAISED_ON`, `MERGED_INTO`, `MENTIONS` (Chunk→entity). |
|
|
| **Operations layer:** `Customer`, `Account`, `Transaction`, `Device`, `IP`, `Address`, `Branch`, `Merchant`, |
| with fraud rings planted in the data and labelled with the right answers. |
|
|
| ## 5. Synthetic Content (generated by Claude) |
| - **~60 PDFs** across the 12 departments: policies, SOPs, guardrail standards, KPI frameworks, audit reports and |
| regulatory summaries. They look realistic: letterhead, document IDs, version history tables, numbered |
| sections and approval sign-offs. |
| - **Deliberate cross-references** between documents (for example, an SOP cites a policy, which cites a |
| regulation). This is what makes questions that need several hops possible. |
| - **Some legacy versions** (such as a 1998 SOP replaced in 2019), so versioning and lineage can be shown. |
| - Source text is kept as Markdown in the repo, and the PDFs are rendered from it deterministically. |
|
|
| ## 6. LLM & Embeddings |
| | Use | Default (free) | Optional | |
| |---|---|---| |
| | Q&A and upload extraction | **Google Gemini free tier** (a Flash-class model), using the owner's key | Claude Sonnet 5.5 or Gemini, using the **tester's own key** | |
| | Embeddings | Small open-source model running locally on CPU | — | |
|
|
| - Switching providers is a single setting (LangChain chat-model interface). |
| - **Keys are protected by PINs** (details in §11): |
| - **Owner's Gemini key:** LLM features only unlock after entering an **Access PIN** that the owner shares with testers. |
| - **Tester's own key (BYOK):** protected by a **personal PIN** the tester chooses. |
|
|
| ## 7. Demo Scenarios |
| | # | Scenario | What it proves | |
| |---|---|---| |
| | 1 | **Regulatory impact:** "An EU AI Act amendment lands. What's affected?" | Following 4–6 links through the graph: regulation → policies → controls → SOPs → teams → KPIs | |
| | 2 | **Side by side:** the same question answered by plain RAG and by GraphRAG, each showing its sources | Better answers on questions that need several hops or several documents | |
| | 3 | **Role impersonation:** the tester picks a role (see §7.1) and asks the same question; a **compare mode** shows two roles side by side | Access control and guardrails built into the graph | |
| | 4 | **Upload your own PDF:** see it turn into a graph and ask questions about it | Real ingestion, plus isolated graphs per session | |
| | 5 | **Fraud rings** across ~1M records | The "and more": graph analytics beyond RAG | |
| | 6 | **Risk roll-up:** one control → department KRI → a bank-wide score | Aggregating up a hierarchy | |
| | 7 | **Version history:** which SOP replaced which, and why | Lineage | |
| | 8 | **Performance panel:** response times for these queries on the 1M-node graph | Speed at a realistic scale | |
|
|
| ### 7.1 Role Impersonation |
| Testers choose who they are, and the same question gets different answers depending on what that role is allowed to see. |
| Access is decided by graph paths (`Role -[:CAN_ACCESS]-> Policy/System`, `Guardrail -[:APPLIES_TO]-> Role`) and |
| enforced **before** retrieval, so the LLM never sees content the role isn't allowed to see. |
|
|
| | Role | Clearance | Sees | |
| |---|---|---| |
| | Branch Teller | L1 | Retail SOPs, customer-facing policies | |
| | Relationship Manager (Wealth) | L2 | Plus client suitability, wealth products | |
| | Credit Analyst | L2 | Plus credit policy, limits, risk ratings | |
| | Compliance Officer (AML/KYC) | L3 | Plus AML typologies, fraud-ring alerts, SAR procedures | |
| | Chief Risk Officer | L4 | Everything, including bank-wide KRIs and board risk reports | |
| | Internal Auditor | L4 (read-only) | Everything, plus audit findings and control test results | |
| | External Contractor (IT) | L0 | Only IT runbooks for their assigned systems | |
|
|
| - A role badge is always visible. Every answer shows **"Answered as: <role>"** and, where relevant, notes that |
| N sources were withheld for this role (without revealing what they contain). |
| - **Compare mode:** one question, two roles, answers side by side, with the difference highlighted. |
| - This is a simulation for the demo: choosing a role is not authentication. |
|
|
| ## 8. Evaluation (plain RAG vs GraphRAG) |
| - **~30 questions with known correct answers** in 4 types: single fact, multi-hop, cross-document, aggregation/compliance. |
| - **Two setups:** (a) vector-only RAG (top-k chunks), (b) GraphRAG (vector entry points + graph traversal). |
| Both use FalkorDB, so the only difference is whether the graph is used. |
| - **Metrics:** |
| - answer accuracy (graded against the known answers) |
| - completeness (were all affected items found?) |
| - whether citations are correct |
| - response time |
| - **tokenomics** (see §8.1) |
| - **Expected results, stated up front:** similar on single-fact questions, and GraphRAG ahead on the other types. |
| Results are published in the UI as they come out. |
|
|
| ### 8.1 Tokenomics |
| Every answer shows its token economics, and the evaluation reports them in aggregate: |
| | Metric | Meaning | |
| |---|---| |
| | Input / output tokens | What went into and came out of the LLM | |
| | Context efficiency | How many of the tokens sent were actually cited in the answer | |
| | Cost per answer | Tokens × list price of the selected model (shown as $0 on the free tier, with the equivalent paid cost next to it) | |
| | **Cost per correct answer** | Total cost ÷ correct answers. This is the headline number for executives | |
| | Tokens saved | GraphRAG vs vector RAG. Precise graph context typically means fewer, more relevant tokens (to be measured) | |
|
|
| A running session total ("You've used X tokens ≈ $Y") appears in the header. |
|
|
| ## 9. Architecture & Hosting |
|
|
| > **Build notes (v0.3 implementation):** |
| > - **Hosting changed:** Docker Spaces are paid on this account, so Cortex runs as a **Gradio-SDK Space on ZeroGPU (free)**. |
| > `app.py` serves the FastAPI app on port 7860. FalkorDB runs **embedded**, using the redis-server and FalkorDB module |
| > shipped in the `falkordblite` package. The fallback, if the Space image can't run it, is the FalkorDB Cloud free tier. |
| > - **Deploy:** a GitHub Action (`.github/workflows/deploy-cortex-space.yml`) pushes `cortex/` to `sk-aiu/cortex`. |
| > - **Extraction:** bank PDFs use pattern extraction (0 tokens; 295/295 relationships correct against the answer key). |
| > Uploads use LLM structured extraction. |
| > - **No LLM-written Cypher in v1:** all Cypher is fixed and read-only, which is stricter than §11's rule. |
| > - **Operational graph size** is chosen automatically from available RAM (≥12 GB: ~1M nodes; ≥6 GB: 50%; otherwise 20%). |
|
|
| ``` |
| Hugging Face Docker Space (free, 16 GB RAM, 2 vCPU, port 7860) |
| ┌─────────────────────────────────────────────────────────────┐ |
| │ supervisord │ |
| │ ├─ FalkorDB (knowledge graph + ops graph + sandbox graphs) │ |
| │ └─ App: FastAPI + static web UI │ |
| │ ├─ LangChain pipelines (ingest, RAG, GraphRAG) │ |
| │ ├─ Local embeddings (CPU) │ |
| │ └─ LLM gateway → Gemini (default) / Claude (BYO key) │ |
| └─────────────────────────────────────────────────────────────┘ |
| ``` |
| - Everything runs in one container, and the same image runs locally with `docker compose up`. |
| - **Idle reset after 4 hours:** if nobody uses the app for 4 hours, it resets itself. Sandbox graphs and chat |
| sessions are cleared, memory is freed, and PIN sessions expire. |
| - Note: on the free tier, Hugging Face decides when the container itself goes to sleep (about 48 hours idle), |
| and that can't be changed without paid hardware. The 4-hour rule is enforced by our app. |
| - Waking the container after it sleeps takes 1–3 minutes, and a "warming up" page is shown meanwhile. |
| - **Removing Neo4j frees about 6 GB**, which leaves plenty of room on 16 GB. |
| |
| ## 10. UI |
| - **Executive view:** landing page with key figures, one-click buttons for scenarios 1–8, a graph visualisation, |
| and the "Ask Cortex" chat with side-by-side answers, source paths and a tokenomics strip for each answer. |
| - **Role switcher** in the header (§7.1), with compare mode. |
| - **Engineer view:** the raw Cypher behind every answer, the pipeline steps, evaluation results (CSV download), |
| a link to the repo, and a settings panel for choosing the LLM or entering your own key. |
| - **Stack:** FastAPI backend and a lightweight static front end (Cytoscape.js from a CDN for the graph view). |
|
|
| ## 11. Security, Cost, Limits & Restrictions |
| **Every limit is shown to users where it applies**: on the upload box, in the chat header, in the settings panel, |
| and in a "Limits & fair use" page linked from the footer. Nothing is limited silently. |
|
|
| ### 11.1 Key protection (PINs) |
| | Key | Protection | Where the user sees it | |
| |---|---|---| |
| | Owner's Gemini key | Stored as a Space secret and never sent to the browser. LLM features need an **Access PIN** (6+ digits, shared by the owner, can be changed at any time). A valid PIN gives an 8-hour session | PIN prompt when the tester first asks a question | |
| | Tester's own key (BYOK) | Encrypted **in the tester's browser** with a personal PIN (WebCrypto AES-GCM, key derived with PBKDF2). Decrypted only when a request is sent, passed to the provider over HTTPS, never stored or logged on the server | Settings panel: "Your key is encrypted on this device. Use a key with a spending limit." | |
| | Failed PIN attempts | 5 failures lead to a 15-minute lockout for that session or IP | Lockout message with a countdown | |
|
|
| ### 11.2 Limits & restrictions (the user-facing list) |
| | Area | Limit / rule | Shown where | |
| |---|---|---| |
| | Sandbox graphs | **Deleted 24 hours after upload**, or at the 4-hour idle reset if that comes first | Countdown timer on the sandbox ("Wipes in 23h 12m") | |
| | Uploads | ≤ 5 MB, ≤ 20 pages, PDF only, text-only extraction, max 3 uploads per session | On the upload box | |
| | Shared free LLM | Questions per session and per day (e.g. 20 / 200). BYOK has no app limit | Counter in the chat header ("12 of 20 left today") | |
| | Questions | Max 500 characters. Harmful or off-topic prompts are refused | Under the input box | |
| | LLM-generated queries | **Read-only** Cypher (write clauses blocked), 5-second timeout, 1,000-row limit | Engineer view, with the query | |
| | Role impersonation | A simulation, not authentication | Role badge tooltip | |
| | Data | 100% synthetic. Never upload real customer or confidential data | Banner on the landing page and the upload box | |
| | Idle reset | After 4 hours without activity, sessions, sandboxes and PIN sessions are cleared | Footer note | |
| | Scripted scenarios | Answers are **pre-computed**: they cost nothing and work even if the LLM is down | "Cached answer" tag | |
| | Logging | Only questions and response times are logged. No keys, PINs or uploaded content | Limits & fair use page | |
|
|
| ## 11A. Name & Branding |
| - **Name: Cortex.** The name suggests the "intelligence layer" of the enterprise, and follows the IT convention of |
| "Cortex" for intelligence, security and engineering platforms. |
| - **Caution:** several commercial products already use "Cortex". That's fine for an internal demo, but check |
| trademarks before any external or commercial use. The fictional bank is **"Cortex Bank"**. |
| - **Logo:** an original mark. A "C" formed by an arc of connected graph nodes, with a bright node at its opening |
| (knowledge → insight). Navy-to-orange gradient. Files: `docs/brand/cortex-logo.svg` / `.png`. |
| - **Palette:** navy `#0B1B3A`, blue `#1E3A8A`, orange `#F97316`, teal `#0EA5A4` (for "success"), slate for text. |
| |
| ## 12. Repository Layout |
| ``` |
| cortex/ |
| docs/ DESIGN.md, DEMO_SCRIPT.md, EVAL.md, one-pager |
| content/ source Markdown for the ~60 bank documents |
| pdfs/ rendered PDFs (committed) |
| generator/ pdf_render.py, ops_data.py (seeded, fraud injection) |
| ingest/ LangChain pipeline: load, chunk, embed, extract, load to FalkorDB |
| app/ FastAPI, RAG + GraphRAG chains, LLM gateway, static UI |
| eval/ questions + known answers, runner, results |
| deploy/ Dockerfile (HF Space), supervisord.conf, docker-compose.yml |
| tests/ |
| ``` |
|
|
| ## 13. Future State (not now) |
| - **AWS deployment** (for example ECS, or EC2 with FalkorDB Cloud) once there's an AWS account and budget. |
| - **Neo4j comparison** as an optional write-up or benchmark appendix. |
| - **Live external feed:** daily AI and regulatory news ingested as `Signal` nodes that link to the policies they |
| affect. This was the original "AI News" idea. |
|
|
| ## 14. Two-Day Plan |
| | When | Deliverable | |
| |---|---| |
| | Day 1 AM | Bank content (Markdown) → ~60 PDFs; ops data generator | |
| | Day 1 PM | LangChain ingestion into FalkorDB; graph snapshot; RAG and GraphRAG chains working | |
| | Day 2 AM | UI (executive and engineer views, role switcher, tokenomics), scenarios, uploads, PIN-protected keys, evaluation run | |
| | Day 2 PM | Hugging Face Space deploy, limits page, demo script, updated one-pager | |
| | After build | Recorded backup video, once everything is built and nearly ready | |
|
|
| ## 15. Risks |
| | Risk | Mitigation | |
| |---|---| |
| | Free-tier LLM rate limits | Pre-built graph, cached scenario answers, BYO key | |
| | Imperfect LLM extraction | Schema-guided extraction, validation, spot checks against known answers | |
| | Space cold start | Warm-up page, fast snapshot load | |
| | Trust around BYO keys | Encrypted in the browser with a PIN, not logged, disclosure on the page, suggest limited keys | |
| | Access PIN leaked | Owner changes it in Space secrets; rate limits cap the damage | |
| | Hugging Face sleep can't be set to 4h on the free tier | App-level 4-hour idle reset (§9) | |
| | Licensing (FalkorDB is SSPL) | Flag to Legal before any production use | |
|
|
| **Open items:** owner creates a free Gemini API key and a Hugging Face Space, and chooses the Access PIN (instructions will be provided). |
|
|
| ## 16. Cradle of the Learning Portal |
| Cortex is where the owner's **Learning Portal** started (1 Oct 2026). After Cortex went live, the owner wanted to be able to |
| explain it from scratch, at any depth, without reading long text. That became a separate product: |
| a private, phone-first portal that turns every project into learning, in a loop: **Build → Capture → Learn → Explore → Decide → Build**. |
|
|
| **Agreed brief (for the new repo; the portal's name = the repo's name, chosen by the owner):** |
| | Topic | Decision | |
| |---|---| |
| | Proof of concept | Cortex only; other projects join later | |
| | Device and privacy | iPhone first; private (private Hugging Face Space) | |
| | Formats | Audio episodes, 60-90 s narrated reels, swipe cards, 3-question checks, "explain it back"; text is the fallback | |
| | Voice | Kokoro-82M `af_heart`, same as the X-Ray film (`docs/film/make_audio.py`) | |
| | Structure | Concepts ↔ Projects ↔ Decisions, linked as a graph | |
| | Retention | Daily 5-minute spaced-repetition drill | |
| | Updates | Weekly scheduled run reads the repos (commits + `DECISIONS.md`) and refreshes learning packs; on demand too | |
| | Explore | Weekly tech radar (Adopt / Trial / Watch / Hold) for the technologies and techniques each project uses, with dated sources; the owner decides | |
| | Capture habit | Every project keeps a `DECISIONS.md` (decision, options, reason, lesson); see [DECISIONS.md](DECISIONS.md) | |
|
|
