# FALSIFY
### the AI that revises, not forgets
**Drop one contradicting fact. Watch dependent conclusions die, a losing hypothesis rise — permanently, across sessions.**
[](#-hackathon-track--theme)
[](#-hackathon-track--theme)
[](https://github.com/topoteretes/cognee)
[](https://www.python.org/)
[](LICENSE)
---
## The Problem: AI Remembers the *Wrong* Fact
Every memory layer bolted onto an LLM today is an **append-only pile of facts**. It can remember. It cannot *un-believe*.
When new evidence contradicts something the AI already "knows," today's systems do one of two bad things:
- **RAG / vector memory** keeps citing the stale fact forever — it has no notion that a belief can *die*.
- **Naive "delete the memory"** throws away the fact *and* every conclusion built on top of it, with no record of *why* — a lobotomy, not a revision.
This is **belief-level amnesia**. The AI doesn't just forget *where* it put the context (this hackathon's theme) — it confidently remembers a fact that has since been proven false, and every downstream conclusion inherits the lie.
> A forensic audit reveals the "March QA report" was **back-dated**. A human analyst instantly revises: *"Then the March timeline is dead — the January supplier email is now our best evidence."* Today's AI memory keeps answering **"March 2021."**
---
## The Solution: A Living Belief Graph
FALSIFY treats a research inquiry not as a chat log but as a **living belief graph** of typed, stateful nodes — `Hypothesis`, `Evidence`, `Conclusion` — wired together by dependency edges.
When a new fact **contradicts** an existing piece of evidence, FALSIFY performs **belief revision** on the graph itself:
1. **Refute** — the contradicted `Evidence` flips to `truth_state = REFUTED`.
2. **Propagate forward** — the refutation cascades *along `depends_on` edges* to every `Conclusion` that critically rested on it → `INVALIDATED`.
3. **Re-ignite** — the losing `Hypothesis` is demoted to `SUPERSEDED`; the strongest surviving rival is promoted as the new frontier.
4. **Surgically forget** — orphaned dead-ends (no surviving consumer) are hard-deleted from **both** the graph and the vector store. Provenance nodes (the refuted fact, the superseding fact) are **kept** so the graph always explains *why* it changed.
5. **Persist** — truth-state is written *on the node*, so the **next session's `recall()` skips the dead branches** — while a plain-RAG baseline still cites the stale fact.
The result is memory that **revises instead of forgets**: it changes its mind, keeps the receipts, and never loses the thread.
---
## Demo
*A **live** run (the LLM judges the contradiction): drop one back-dating fact → `E_qa` turns **red (refuted)** → Conclusion **K is forgotten** (deleted from graph + vector) → Hypothesis **A** dims to **amber (superseded)** → Hypothesis **B ignites** as the new frontier. Caption: **"AI revised, not forgot."***
The money shot is the **scoreboard** printed on every run:
```text
[SCOREBOARD]
FALSIFY recall : X knew by Jan 2021 (supplier email) [revised]
Plain-RAG : X knew by Mar 2021 (QA report) [STALE]
```
Same underlying store. Same query. FALSIFY revised its belief; the baseline did not.
---
## Key Features
| | Feature | What it does |
|---|---|---|
| 🧠 | **Belief graph, not a fact pile** | Nodes are *stateful beliefs* (`alive` / `refuted` / `superseded` / `invalidated`), not immutable rows. |
| ⚡ | **Forward refutation propagation** | One contradiction cascades through `depends_on` edges and invalidates every dependent conclusion in ~3s. |
| 🎯 | **Two-gate contradiction detection** | Cheap deterministic vector prefilter (`cosine < 0.35`) → skeptical LLM adjudication (`confidence ≥ 0.6`). No hallucinated refutations. |
| ✂️ | **Surgical forget** | Orphaned dead-ends are hard-deleted from graph **and** vector; provenance is retained. Not a lobotomy — a revision. |
| 🔁 | **Cross-session persistence** | Disbelief lives on the node. Restart the process and `recall()` still skips the dead branches. |
| 📊 | **Live A/B scoreboard** | FALSIFY (revised) vs. plain-RAG (stale) side-by-side, every run — the differentiator made visible. |
| 🕸️ | **Force-graph visualization** | Nodes colored by truth-state; forgotten nodes red-flash then ripple out of the sim. |
| 🔌 | **Zero external services** | Self-hosted Cognee defaults — LanceDB (vector) + Ladybug (graph) + SQLite. OpenAI-compatible; bring any endpoint. |
---
## How It Works
FALSIFY drives Cognee's v1.0 memory API (`remember` / `recall` / `improve` / `forget`) plus a set of **custom `memify` tasks** that operate directly on the graph engine's truth-state.
```mermaid
flowchart TD
subgraph S1["Session 1 — build the belief graph"]
Q["InvestigationQuestion
Did Company X know before the recall?"]
HA["Hypothesis A
knew via QA report, Mar 2021"]
HB["Hypothesis B
knew via supplier email, Jan 2021"]
HC["Hypothesis C
didn't know"]
Eqa["Evidence E_qa
March QA report"]
Eem["Evidence E_email
January supplier email"]
K["Conclusion K
X knew by March 2021"]
Q --- HA & HB & HC
Eqa -- supports --> HA
Eem -- supports --> HB
K -- "depends_on (critical)" --> Eqa
end
NF["🆕 New fact (Session 2)
Forensic audit: March QA report was back-dated"]
subgraph REV["Belief revision — custom memify tasks"]
direction TB
D["1. Detect contradiction
vector prefilter < 0.35 → LLM judge ≥ 0.6"]
R["2. E_qa → REFUTED"]
P["3. Forward BFS on depends_on
K → INVALIDATED"]
G["4. A → SUPERSEDED • B ignites (promoted)"]
F["5. Forget orphan K
delete from graph + vector
keep E_qa as refuted provenance"]
D --> R --> P --> G --> F
end
NF --> D
Eqa -.-> D
subgraph SCORE["Scoreboard"]
FA["FALSIFY recall → B (Jan 2021) ✅ revised"]
RA["Plain-RAG → March 2021 QA report ❌ stale"]
end
F --> FA
F --> RA
```
**The mechanism in one paragraph:** a `Conclusion --depends_on--> Evidence` edge is the propagation rail. Refutation seeds at an `Evidence` node; a conclusion stays justified only if it has a **grounded** critical support chain that bottoms out in a still-alive node. FALSIFY computes this as a **least-fixpoint** over the dependency graph, so one formulation correctly handles chains, **diamonds** (a conclusion survives while any critical alternative is grounded), non-critical dependencies, **and cycles** (a self-supporting loop with no grounded base collapses — and the fixpoint always terminates). Hypotheses are re-scored via their `supports` edges. Truth-state (`truth_alignment` + `truth_epoch`) is written on-node via `set_node_truth_state`, so it survives a restart and `recall()` filters on it. See [`falsify/tasks/propagate_refutation.py`](falsify/tasks/propagate_refutation.py) and [REQUIREMENTS.md](REQUIREMENTS.md) for the full algorithm, edge vocabulary, and edge-case handling.
---
## Install & Setup
### Prerequisites
- Python **3.10 – 3.14**
- An OpenAI **or any OpenAI-compatible** API key (OpenRouter, vLLM, LM Studio, Azure, …)
### 1. Clone & create an environment
```bash
git clone https://github.com/ArpitKumar8649/cognee-hackathon-project.git
cd cognee-hackathon-project
# uv (recommended)
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt
# …or plain pip
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```
### 2. Configure your key
```bash
cp .env.template .env
# then edit .env and set LLM_API_KEY
```
Minimal `.env` (OpenAI):
```bash
LLM_PROVIDER="openai"
LLM_API_KEY="sk-..."
LLM_MODEL="openai/gpt-5-mini"
```
Any OpenAI-compatible endpoint (OpenRouter shown):
```bash
LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="openrouter/google/gemini-2.0-flash-lite-preview-02-05:free"
LLM_ENDPOINT="https://openrouter.ai/api/v1"
```
> **Heads-up:** if you configure *only* the LLM or *only* embeddings, Cognee defaults the other to OpenAI. Either configure both or keep a valid OpenAI key handy. All databases default to **local, self-hosted** stores — no external services required.
---
## Usage
```bash
# Full run: build the belief graph, drop the contradicting fact,
# print the FALSIFY-vs-RAG scoreboard. Judges start here (~2 min).
python main.py
# Deterministic demo mode — pins the refuted evidence id so the
# cascade + forget run on real graph/vector APIs even if the LLM
# judge is flaky. This is the presentation safety net.
python main.py --demo
# Run the test suite — 11 tests, NO API key required
# (FakeGraph + mocked LLM): propagation, diamond, cycle-safety,
# surgical forget, and the two detector gates.
pytest -q
pytest -q tests/test_falsify.py # core belief-revision cascade
pytest -q tests/test_detect.py # two-gate contradiction detector
```
**No API key?** `main.py` prints a clear message explaining how to set `LLM_API_KEY` and exits with code **0** — it never crashes in front of a judge.
### Cross-session persistence
Truth-state is written *on the graph node*, so it survives a process restart. `main.py` re-reads the belief state fresh at the end of the run to prove the refuted branch never comes back. Run `python main.py --keep` to build on top of existing memory instead of pruning first.
### Visualization
Every run writes a self-contained interactive graph to **`output/graph.html`** — just open it in a browser (no server needed). Nodes are colored by truth-state: **alive = green**, **refuted = red (dashed)**, **invalidated = grey**, **superseded = amber**.
---
## Architecture
Full design — verified Cognee API surface, node/edge vocabulary, the exact forward-propagation algorithm, truth-state lifecycle, and every handled edge case — lives in the build contract:
- **[REQUIREMENTS.md](REQUIREMENTS.md)** — grep-verified Cognee API references, truth-state lifecycle, and edge-case matrix.
```text
main.py # entry point — seed + revise + scoreboard; graceful no-key exit 0
falsify/
models.py # DataPoint subclasses (Hypothesis/Evidence/Conclusion) + TruthState
edges.py # edge-name constants: DEPENDS_ON, SUPPORTS, REFUTES, SUPERSEDES
graph_ops.py # verified wrapper over Cognee's graph + vector engines
seed.py # demo corpus: the Company-X recall investigation
tasks/
detect_contradictions.py # two-gate detector (vector prefilter + skeptical-LLM judge)
propagate_refutation.py # grounded-fixpoint refutation cascade + hypothesis promotion
cascade_forget.py # surgical orphan delete (graph + vector), keeps provenance
falsify.py # orchestration: build_graph(), revise(new_fact), scoreboard()
utils.py # interactive HTML viz + BEFORE/AFTER console state
tests/
test_falsify.py # cascade / diamond / cycle-safety / surgical forget / promote
test_detect.py # detector: pinned demo path + Gate-1 filter + Gate-2 thresholds
conftest.py # FakeGraph fixture — key-free, DB-free in-memory engine stand-in
```
---
## How FALSIFY Maps to the Judging Criteria
| # | Criterion | How FALSIFY nails it |
|---|---|---|
| 1 | **Potential Impact** | Solves *belief-level amnesia* — AI confidently remembering facts that have been proven false. Every research, legal, medical, or intelligence copilot needs memory that can be *revised*, not just appended. |
| 2 | **Creativity / Originality** | Reframes graph nodes as **stateful beliefs** (alive / refuted / superseded / invalidated) and treats "changing your mind" as a first-class graph operation — not chat history, not RAG. |
| 3 | **Technical Excellence** | Custom `memify` extraction + enrichment tasks; deterministic-first **two-gate** contradiction detection; forward BFS propagation with cycle/diamond-safe `visited` sets; dual-store surgical delete; on-node persistent truth-state. |
| 4 | **Best Use of Cognee** | Drives the v1.0 memory API end-to-end — `remember(session_id)` → `recall()` → `improve()` → surgical `forget()` — plus custom `memify` tasks operating directly on `set_node_truth_state` / `get_neighborhood` / `delete_nodes`. |
| 5 | **UX / Presentation** | One-screen, 30-second beat: paste one fact → watch A collapse and B ignite → read the FALSIFY-vs-RAG scoreboard. Force-graph colored by belief state. |
| 6 | **Documentation & Reproducibility** | `python main.py` runs in ~2 min with zero external services; graceful no-key exit; full README + REQUIREMENTS + 11 key-free tests + demo script. |
---
## Hackathon Track & Theme
- **Event:** *The Hangover Part AI: Where's My Context?*
- **Track:** 🏆 **Best Use of Open Source** — built entirely on open-source Cognee with self-hosted, zero-dependency defaults (LanceDB + Ladybug + SQLite).
- **Category / Theme:** 🔬 **Research & Knowledge Copilot** — a research assistant whose memory revises its beliefs as new evidence arrives.
The theme asks *"Where's my context?"* FALSIFY's answer: the context isn't lost — it was **wrong**, and the AI should *revise* it, not blindly recall it.
---
## Future Work
- **Confidence-weighted partial refutation** — decay a Conclusion's confidence continuously instead of a binary alive/invalidated flip.
- **Multi-hop evidence provenance UI** — click any node to trace the full chain of *why it lives or died*.
- **Automated evidence ingestion** — stream documents in and let the two-gate detector surface contradictions proactively.
- **Human-in-the-loop review** — queue borderline LLM verdicts (0.4–0.6 confidence) for analyst confirmation before cascading.
- **Belief-diff export** — a git-style diff of the belief graph between any two epochs.
- **Neo4j / Postgres backends** — swap the graph engine for a distributed store with no code change (Cognee adapter interface).
---
## Acknowledgments
- **[Cognee](https://github.com/topoteretes/cognee)** — the open-source AI memory platform FALSIFY is built on. Its truth-state graph APIs, custom `memify` pipeline, and self-hosted defaults made belief revision possible without a single external service.
- **[WeMakeDevs](https://wemakedevs.org/)** — for hosting *The Hangover Part AI* hackathon and championing open-source builders.
---
**FALSIFY — the AI that revises, not forgets.**