---
title: FALSIFY β Belief-Revision Copilot
emoji: π§
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: false
license: mit
short_description: The AI that revises, not forgets β on Cognee.
---
# FALSIFY
### the AI that revises, not forgets
**Drop one contradicting fact. Watch dependent conclusions die, a losing hypothesis rise β permanently, across sessions.**
[](#-hackathon-track--theme)
[](#-hackathon-track--theme)
[](https://github.com/topoteretes/cognee)
[](https://www.python.org/)
[](LICENSE)
---
## The Problem: AI Remembers the *Wrong* Fact
Every memory layer bolted onto an LLM today is an **append-only pile of facts**. It can remember. It cannot *un-believe*.
When new evidence contradicts something the AI already "knows," today's systems do one of two bad things:
- **RAG / vector memory** keeps citing the stale fact forever β it has no notion that a belief can *die*.
- **Naive "delete the memory"** throws away the fact *and* every conclusion built on top of it, with no record of *why* β a lobotomy, not a revision.
This is **belief-level amnesia**. The AI doesn't just forget *where* it put the context (this hackathon's theme) β it confidently remembers a fact that has since been proven false, and every downstream conclusion inherits the lie.
> A forensic audit reveals the "March QA report" was **back-dated**. A human analyst instantly revises: *"Then the March timeline is dead β the January supplier email is now our best evidence."* Today's AI memory keeps answering **"March 2021."**
---
## The Solution: A Living Belief Graph
FALSIFY treats a research inquiry not as a chat log but as a **living belief graph** of typed, stateful nodes β `Hypothesis`, `Evidence`, `Conclusion` β wired together by dependency edges.
When a new fact **contradicts** an existing piece of evidence, FALSIFY performs **belief revision** on the graph itself:
1. **Refute** β the contradicted `Evidence` flips to `truth_state = REFUTED`.
2. **Propagate forward** β the refutation cascades *along `depends_on` edges* to every `Conclusion` that critically rested on it β `INVALIDATED`.
3. **Re-ignite** β the losing `Hypothesis` is demoted to `SUPERSEDED`; the strongest surviving rival is promoted as the new frontier.
4. **Surgically forget** β orphaned dead-ends (no surviving consumer) are hard-deleted from **both** the graph and the vector store. Provenance nodes (the refuted fact, the superseding fact) are **kept** so the graph always explains *why* it changed.
5. **Persist** β truth-state is written *on the node*, so the **next session's `recall()` skips the dead branches** β while a plain-RAG baseline still cites the stale fact.
The result is memory that **revises instead of forgets**: it changes its mind, keeps the receipts, and never loses the thread.
---
## Demo
*A **live** run (the LLM judges the contradiction): drop one back-dating fact β `E_qa` turns **red (refuted)** β Conclusion **K is forgotten** (deleted from graph + vector) β Hypothesis **A** dims to **amber (superseded)** β Hypothesis **B ignites** as the new frontier. Caption: **"AI revised, not forgot."***
### π Live Links
| | Link |
|---|---|
| π **Live Demo** | [huggingface.co/spaces/KumarArpit8649/Falsify](https://huggingface.co/spaces/KumarArpit8649/Falsify) |
| π¬ **Video Demo** | [YouTube β 3-minute walkthrough](https://youtu.be/YOUR_VIDEO_ID_HERE) |
| π» **Source Code** | [github.com/ArpitKumar8649/cognee-hackathon-project](https://github.com/ArpitKumar8649/cognee-hackathon-project) |
The money shot is the **scoreboard** printed on every run:
```text
[SCOREBOARD]
FALSIFY recall : X knew by Jan 2021 (supplier email) [revised]
Plain-RAG : X knew by Mar 2021 (QA report) [STALE]
```
Same underlying store. Same query. FALSIFY revised its belief; the baseline did not.
---
## Key Features
| | Feature | What it does |
|---|---|---|
| π§ | **Belief graph, not a fact pile** | Nodes are *stateful beliefs* (`alive` / `refuted` / `superseded` / `invalidated`), not immutable rows. |
| β‘ | **Forward refutation propagation** | One contradiction cascades through `depends_on` edges and invalidates every dependent conclusion in ~3s. |
| π― | **Two-gate contradiction detection** | Cheap deterministic vector prefilter (`cosine < 0.35`) β skeptical LLM adjudication (`confidence β₯ 0.6`). No hallucinated refutations. |
| βοΈ | **Surgical forget** | Orphaned dead-ends are hard-deleted from graph **and** vector; provenance is retained. Not a lobotomy β a revision. |
| π | **Cross-session persistence** | Disbelief lives on the node. Restart the process and `recall()` still skips the dead branches. |
| π | **Live A/B scoreboard** | FALSIFY (revised) vs. plain-RAG (stale) side-by-side, every run β the differentiator made visible. |
| πΈοΈ | **Force-graph visualization** | Nodes colored by truth-state; forgotten nodes red-flash then ripple out of the sim. |
| π | **Zero external services** | Self-hosted Cognee defaults β LanceDB (vector) + Ladybug (graph) + SQLite. OpenAI-compatible; bring any endpoint. |
---
## How It Works
FALSIFY drives Cognee's v1.0 memory API (`remember` / `recall` / `improve` / `forget`) plus a set of **custom `memify` tasks** that operate directly on the graph engine's truth-state.
```mermaid
flowchart TD
subgraph S1["Session 1 β build the belief graph"]
Q["InvestigationQuestion
Did Company X know before the recall?"]
HA["Hypothesis A
knew via QA report, Mar 2021"]
HB["Hypothesis B
knew via supplier email, Jan 2021"]
HC["Hypothesis C
didn't know"]
Eqa["Evidence E_qa
March QA report"]
Eem["Evidence E_email
January supplier email"]
K["Conclusion K
X knew by March 2021"]
Q --- HA & HB & HC
Eqa -- supports --> HA
Eem -- supports --> HB
K -- "depends_on (critical)" --> Eqa
end
NF["π New fact (Session 2)
Forensic audit: March QA report was back-dated"]
subgraph REV["Belief revision β custom memify tasks"]
direction TB
D["1. Detect contradiction
vector prefilter < 0.35 β LLM judge β₯ 0.6"]
R["2. E_qa β REFUTED"]
P["3. Forward BFS on depends_on
K β INVALIDATED"]
G["4. A β SUPERSEDED β’ B ignites (promoted)"]
F["5. Forget orphan K
delete from graph + vector
keep E_qa as refuted provenance"]
D --> R --> P --> G --> F
end
NF --> D
Eqa -.-> D
subgraph SCORE["Scoreboard"]
FA["FALSIFY recall β B (Jan 2021) β
revised"]
RA["Plain-RAG β March 2021 QA report β stale"]
end
F --> FA
F --> RA
```
**The mechanism in one paragraph:** a `Conclusion --depends_on--> Evidence` edge is the propagation rail. Refutation seeds at an `Evidence` node; a conclusion stays justified only if it has a **grounded** critical support chain that bottoms out in a still-alive node. FALSIFY computes this as a **least-fixpoint** over the dependency graph, so one formulation correctly handles chains, **diamonds** (a conclusion survives while any critical alternative is grounded), non-critical dependencies, **and cycles** (a self-supporting loop with no grounded base collapses β and the fixpoint always terminates). Hypotheses are re-scored via their `supports` edges. Truth-state (`truth_alignment` + `truth_epoch`) is written on-node via `set_node_truth_state`, so it survives a restart and `recall()` filters on it. See [`falsify/tasks/propagate_refutation.py`](falsify/tasks/propagate_refutation.py) and [REQUIREMENTS.md](REQUIREMENTS.md) for the full algorithm, edge vocabulary, and edge-case handling.
---
## Install & Setup
### Prerequisites
- Python **3.10 β 3.14**
- An OpenAI **or any OpenAI-compatible** API key (OpenRouter, vLLM, LM Studio, Azure, β¦)
### 1. Clone & create an environment
```bash
git clone https://github.com/ArpitKumar8649/cognee-hackathon-project.git
cd cognee-hackathon-project
# uv (recommended)
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt
# β¦or plain pip
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```
### 2. Configure your key
```bash
cp .env.template .env
# then edit .env and set LLM_API_KEY
```
Minimal `.env` (OpenAI):
```bash
LLM_PROVIDER="openai"
LLM_API_KEY="sk-..."
LLM_MODEL="openai/gpt-5-mini"
```
Any OpenAI-compatible endpoint (OpenRouter shown):
```bash
LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="openrouter/google/gemini-2.0-flash-lite-preview-02-05:free"
LLM_ENDPOINT="https://openrouter.ai/api/v1"
```
Alibaba DashScope (Qwen models):
```bash
LLM_PROVIDER="custom"
LLM_API_KEY="your_dashscope_api_key"
LLM_MODEL="openai/qwen-plus"
LLM_ENDPOINT="https://dashscope-intl.aliyuncs.com/compatible-mode/v1"
```
Cognee Cloud (optional β routes all memory ops to a hosted tenant):
```bash
COGNEE_CLOUD_URL="https://your-tenant.cognee.ai"
COGNEE_CLOUD_API_KEY="your_cognee_cloud_key"
```
> **Heads-up:** if you configure *only* the LLM or *only* embeddings, Cognee defaults the other to OpenAI. Either configure both or keep a valid OpenAI key handy. All databases default to **local, self-hosted** stores β no external services required.
---
## Usage
```bash
# Full run: build the belief graph, drop the contradicting fact,
# print the FALSIFY-vs-RAG scoreboard. Judges start here (~2 min).
python main.py
# Deterministic demo mode β pins the refuted evidence id so the
# cascade + forget run on real graph/vector APIs even if the LLM
# judge is flaky. This is the presentation safety net.
python main.py --demo
# Run the test suite β 11 tests, NO API key required
# (FakeGraph + mocked LLM): propagation, diamond, cycle-safety,
# surgical forget, and the two detector gates.
pytest -q
pytest -q tests/test_falsify.py # core belief-revision cascade
pytest -q tests/test_detect.py # two-gate contradiction detector
```
**No API key?** `main.py` prints a clear message explaining how to set `LLM_API_KEY` and exits with code **0** β it never crashes in front of a judge.
### Cross-session persistence
Truth-state is written *on the graph node*, so it survives a process restart. `main.py` re-reads the belief state fresh at the end of the run to prove the refuted branch never comes back. Run `python main.py --keep` to build on top of existing memory instead of pruning first.
### Visualization
Every run writes a self-contained interactive graph to **`output/graph.html`** β just open it in a browser (no server needed). Nodes are colored by truth-state: **alive = green**, **refuted = red (dashed)**, **invalidated = grey**, **superseded = amber**.
---
## Architecture
Full design β verified Cognee API surface, node/edge vocabulary, the exact forward-propagation algorithm, truth-state lifecycle, and every handled edge case β lives in the build contract:
- **[REQUIREMENTS.md](REQUIREMENTS.md)** β grep-verified Cognee API references, truth-state lifecycle, and edge-case matrix.
```text
main.py # entry point β seed + revise + scoreboard; graceful no-key exit 0
falsify/
models.py # DataPoint subclasses (Hypothesis/Evidence/Conclusion) + TruthState
edges.py # edge-name constants: DEPENDS_ON, SUPPORTS, REFUTES, SUPERSEDES
graph_ops.py # verified wrapper over Cognee's graph + vector engines
seed.py # demo corpus: the Company-X recall investigation
tasks/
detect_contradictions.py # two-gate detector (vector prefilter + skeptical-LLM judge)
propagate_refutation.py # grounded-fixpoint refutation cascade + hypothesis promotion
cascade_forget.py # surgical orphan delete (graph + vector), keeps provenance
falsify.py # orchestration: build_graph(), revise(new_fact), scoreboard()
utils.py # interactive HTML viz + BEFORE/AFTER console state
tests/
test_falsify.py # cascade / diamond / cycle-safety / surgical forget / promote
test_detect.py # detector: pinned demo path + Gate-1 filter + Gate-2 thresholds
conftest.py # FakeGraph fixture β key-free, DB-free in-memory engine stand-in
```
---
## How FALSIFY Maps to the Judging Criteria
| # | Criterion | How FALSIFY nails it |
|---|---|---|
| 1 | **Potential Impact** | Solves *belief-level amnesia* β AI confidently remembering facts that have been proven false. Every research, legal, medical, or intelligence copilot needs memory that can be *revised*, not just appended. |
| 2 | **Creativity / Originality** | Reframes graph nodes as **stateful beliefs** (alive / refuted / superseded / invalidated) and treats "changing your mind" as a first-class graph operation β not chat history, not RAG. |
| 3 | **Technical Excellence** | Custom `memify` extraction + enrichment tasks; deterministic-first **two-gate** contradiction detection; forward BFS propagation with cycle/diamond-safe `visited` sets; dual-store surgical delete; on-node persistent truth-state. |
| 4 | **Best Use of Cognee** | Drives the v1.0 memory API end-to-end β `remember(session_id)` β `recall()` β `improve()` β surgical `forget()` β plus custom `memify` tasks operating directly on `set_node_truth_state` / `get_neighborhood` / `delete_nodes`. |
| 5 | **UX / Presentation** | One-screen, 30-second beat: paste one fact β watch A collapse and B ignite β read the FALSIFY-vs-RAG scoreboard. Force-graph colored by belief state. |
| 6 | **Documentation & Reproducibility** | `python main.py` runs in ~2 min with zero external services; graceful no-key exit; full README + REQUIREMENTS + 11 key-free tests + demo script. |
---
## Hackathon Track & Theme
- **Event:** *The Hangover Part AI: Where's My Context?*
- **Track:** π **Best Use of Open Source** β built entirely on open-source Cognee with self-hosted, zero-dependency defaults (LanceDB + Ladybug + SQLite).
- **Category / Theme:** π¬ **Research & Knowledge Copilot** β a research assistant whose memory revises its beliefs as new evidence arrives.
The theme asks *"Where's my context?"* FALSIFY's answer: the context isn't lost β it was **wrong**, and the AI should *revise* it, not blindly recall it. Runs in ~2 minutes on fully self-hosted, zero-external-service Cognee (LanceDB + graph engine + SQLite), with an offline **keyless demo mode** β or bring your own API key (OpenAI, OpenRouter, DashScope, Azure, or any OpenAI-compatible endpoint) for live LLM-judged contradiction detection. Includes an 11-test suite that needs no API key.
---
## Future Work
- **Confidence-weighted partial refutation** β decay a Conclusion's confidence continuously instead of a binary alive/invalidated flip.
- **Multi-hop evidence provenance UI** β click any node to trace the full chain of *why it lives or died*.
- **Automated evidence ingestion** β stream documents in and let the two-gate detector surface contradictions proactively.
- **Human-in-the-loop review** β queue borderline LLM verdicts (0.4β0.6 confidence) for analyst confirmation before cascading.
- **Belief-diff export** β a git-style diff of the belief graph between any two epochs.
- **Neo4j / Postgres backends** β swap the graph engine for a distributed store with no code change (Cognee adapter interface).
---
## Acknowledgments
- **[Cognee](https://github.com/topoteretes/cognee)** β the open-source AI memory platform FALSIFY is built on. Its truth-state graph APIs, custom `memify` pipeline, and self-hosted defaults made belief revision possible without a single external service.
- **[WeMakeDevs](https://wemakedevs.org/)** β for hosting *The Hangover Part AI* hackathon and championing open-source builders.
---
**FALSIFY β the AI that revises, not forgets.**