README.md

metadata

title: FALSIFY — Belief-Revision Copilot
emoji: 🧠
colorFrom: indigo
colorTo: green
sdk: docker
app_port: 7860
pinned: false
license: mit
short_description: The AI that revises, not forgets — on Cognee.

FALSIFY

the AI that revises, not forgets

Drop one contradicting fact. Watch dependent conclusions die, a losing hypothesis rise — permanently, across sessions.

---

The Problem: AI Remembers the Wrong Fact

Every memory layer bolted onto an LLM today is an append-only pile of facts. It can remember. It cannot un-believe.

When new evidence contradicts something the AI already "knows," today's systems do one of two bad things:

• RAG / vector memory keeps citing the stale fact forever — it has no notion that a belief can die.
• Naive "delete the memory" throws away the fact and every conclusion built on top of it, with no record of why — a lobotomy, not a revision.

This is belief-level amnesia. The AI doesn't just forget where it put the context (this hackathon's theme) — it confidently remembers a fact that has since been proven false, and every downstream conclusion inherits the lie.

A forensic audit reveals the "March QA report" was back-dated. A human analyst instantly revises: "Then the March timeline is dead — the January supplier email is now our best evidence." Today's AI memory keeps answering "March 2021."

---

The Solution: A Living Belief Graph

FALSIFY treats a research inquiry not as a chat log but as a living belief graph of typed, stateful nodes — Hypothesis, Evidence, Conclusion — wired together by dependency edges.

When a new fact contradicts an existing piece of evidence, FALSIFY performs belief revision on the graph itself:

1. Refute — the contradicted Evidence flips to truth_state = REFUTED.
2. Propagate forward — the refutation cascades along depends_on edges to every Conclusion that critically rested on it → INVALIDATED.
3. Re-ignite — the losing Hypothesis is demoted to SUPERSEDED; the strongest surviving rival is promoted as the new frontier.
4. Surgically forget — orphaned dead-ends (no surviving consumer) are hard-deleted from both the graph and the vector store. Provenance nodes (the refuted fact, the superseding fact) are kept so the graph always explains why it changed.
5. Persist — truth-state is written on the node, so the next session's recall() skips the dead branches — while a plain-RAG baseline still cites the stale fact.

The result is memory that revises instead of forgets: it changes its mind, keeps the receipts, and never loses the thread.

---

Demo

A live run (the LLM judges the contradiction): drop one back-dating fact → E_qa turns red (refuted) → Conclusion K is forgotten (deleted from graph + vector) → Hypothesis A dims to amber (superseded) → Hypothesis B ignites as the new frontier. Caption: "AI revised, not forgot."

🔗 Live Links

	Link
🌐 Live Demo	huggingface.co/spaces/KumarArpit8649/Falsify
🎬 Video Demo	YouTube — 3-minute walkthrough
💻 Source Code	github.com/ArpitKumar8649/cognee-hackathon-project

The money shot is the scoreboard printed on every run:

[SCOREBOARD]
   FALSIFY recall : X knew by Jan 2021 (supplier email)   [revised]
   Plain-RAG      : X knew by Mar 2021 (QA report)         [STALE]

Same underlying store. Same query. FALSIFY revised its belief; the baseline did not.

---

Key Features

	Feature	What it does
🧠	Belief graph, not a fact pile	Nodes are stateful beliefs (alive / refuted / superseded / invalidated), not immutable rows.
⚡	Forward refutation propagation	One contradiction cascades through depends_on edges and invalidates every dependent conclusion in ~3s.
🎯	Two-gate contradiction detection	Cheap deterministic vector prefilter (cosine < 0.35) → skeptical LLM adjudication (confidence ≥ 0.6). No hallucinated refutations.
✂️	Surgical forget	Orphaned dead-ends are hard-deleted from graph and vector; provenance is retained. Not a lobotomy — a revision.
🔁	Cross-session persistence	Disbelief lives on the node. Restart the process and recall() still skips the dead branches.
📊	Live A/B scoreboard	FALSIFY (revised) vs. plain-RAG (stale) side-by-side, every run — the differentiator made visible.
🕸️	Force-graph visualization	Nodes colored by truth-state; forgotten nodes red-flash then ripple out of the sim.
🔌	Zero external services	Self-hosted Cognee defaults — LanceDB (vector) + Ladybug (graph) + SQLite. OpenAI-compatible; bring any endpoint.

---

How It Works

FALSIFY drives Cognee's v1.0 memory API (remember / recall / improve / forget) plus a set of custom memify tasks that operate directly on the graph engine's truth-state.

flowchart TD
    subgraph S1["Session 1 — build the belief graph"]
        Q["InvestigationQuestion<br/>Did Company X know before the recall?"]
        HA["Hypothesis A<br/>knew via QA report, Mar 2021"]
        HB["Hypothesis B<br/>knew via supplier email, Jan 2021"]
        HC["Hypothesis C<br/>didn't know"]
        Eqa["Evidence E_qa<br/>March QA report"]
        Eem["Evidence E_email<br/>January supplier email"]
        K["Conclusion K<br/>X knew by March 2021"]

        Q --- HA & HB & HC
        Eqa -- supports --> HA
        Eem -- supports --> HB
        K -- "depends_on (critical)" --> Eqa
    end

    NF["🆕 New fact (Session 2)<br/>Forensic audit: March QA report was back-dated"]

    subgraph REV["Belief revision — custom memify tasks"]
        direction TB
        D["1. Detect contradiction<br/>vector prefilter < 0.35  →  LLM judge ≥ 0.6"]
        R["2. E_qa → REFUTED"]
        P["3. Forward BFS on depends_on<br/>K → INVALIDATED"]
        G["4. A → SUPERSEDED &nbsp;•&nbsp; B ignites (promoted)"]
        F["5. Forget orphan K<br/>delete from graph + vector<br/>keep E_qa as refuted provenance"]
        D --> R --> P --> G --> F
    end

    NF --> D
    Eqa -.-> D

    subgraph SCORE["Scoreboard"]
        FA["FALSIFY recall → B (Jan 2021)  ✅ revised"]
        RA["Plain-RAG → March 2021 QA report  ❌ stale"]
    end

    F --> FA
    F --> RA

The mechanism in one paragraph: a Conclusion --depends_on--> Evidence edge is the propagation rail. Refutation seeds at an Evidence node; a conclusion stays justified only if it has a grounded critical support chain that bottoms out in a still-alive node. FALSIFY computes this as a least-fixpoint over the dependency graph, so one formulation correctly handles chains, diamonds (a conclusion survives while any critical alternative is grounded), non-critical dependencies, and cycles (a self-supporting loop with no grounded base collapses — and the fixpoint always terminates). Hypotheses are re-scored via their supports edges. Truth-state (truth_alignment + truth_epoch) is written on-node via set_node_truth_state, so it survives a restart and recall() filters on it. See falsify/tasks/propagate_refutation.py and REQUIREMENTS.md for the full algorithm, edge vocabulary, and edge-case handling.

---

Install & Setup

Prerequisites

• Python 3.10 – 3.14
• An OpenAI or any OpenAI-compatible API key (OpenRouter, vLLM, LM Studio, Azure, …)

1. Clone & create an environment

git clone https://github.com/ArpitKumar8649/cognee-hackathon-project.git
cd cognee-hackathon-project

# uv (recommended)
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt

# …or plain pip
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

2. Configure your key

cp .env.template .env
# then edit .env and set LLM_API_KEY

Minimal .env (OpenAI):

LLM_PROVIDER="openai"
LLM_API_KEY="sk-..."
LLM_MODEL="openai/gpt-5-mini"

Any OpenAI-compatible endpoint (OpenRouter shown):

LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="openrouter/google/gemini-2.0-flash-lite-preview-02-05:free"
LLM_ENDPOINT="https://openrouter.ai/api/v1"

Alibaba DashScope (Qwen models):

LLM_PROVIDER="custom"
LLM_API_KEY="your_dashscope_api_key"
LLM_MODEL="openai/qwen-plus"
LLM_ENDPOINT="https://dashscope-intl.aliyuncs.com/compatible-mode/v1"

Cognee Cloud (optional — routes all memory ops to a hosted tenant):

COGNEE_CLOUD_URL="https://your-tenant.cognee.ai"
COGNEE_CLOUD_API_KEY="your_cognee_cloud_key"

Heads-up: if you configure only the LLM or only embeddings, Cognee defaults the other to OpenAI. Either configure both or keep a valid OpenAI key handy. All databases default to local, self-hosted stores — no external services required.

---

Usage

# Full run: build the belief graph, drop the contradicting fact,
# print the FALSIFY-vs-RAG scoreboard. Judges start here (~2 min).
python main.py

# Deterministic demo mode — pins the refuted evidence id so the
# cascade + forget run on real graph/vector APIs even if the LLM
# judge is flaky. This is the presentation safety net.
python main.py --demo

# Run the test suite — 11 tests, NO API key required
# (FakeGraph + mocked LLM): propagation, diamond, cycle-safety,
# surgical forget, and the two detector gates.
pytest -q
pytest -q tests/test_falsify.py     # core belief-revision cascade
pytest -q tests/test_detect.py      # two-gate contradiction detector

No API key? main.py prints a clear message explaining how to set LLM_API_KEY and exits with code 0 — it never crashes in front of a judge.

Cross-session persistence

Truth-state is written on the graph node, so it survives a process restart. main.py re-reads the belief state fresh at the end of the run to prove the refuted branch never comes back. Run python main.py --keep to build on top of existing memory instead of pruning first.

Visualization

Every run writes a self-contained interactive graph to output/graph.html — just open it in a browser (no server needed). Nodes are colored by truth-state: alive = green, refuted = red (dashed), invalidated = grey, superseded = amber.

---

Architecture

Full design — verified Cognee API surface, node/edge vocabulary, the exact forward-propagation algorithm, truth-state lifecycle, and every handled edge case — lives in the build contract:

• REQUIREMENTS.md — grep-verified Cognee API references, truth-state lifecycle, and edge-case matrix.

main.py                       # entry point — seed + revise + scoreboard; graceful no-key exit 0
falsify/
  models.py                   # DataPoint subclasses (Hypothesis/Evidence/Conclusion) + TruthState
  edges.py                    # edge-name constants: DEPENDS_ON, SUPPORTS, REFUTES, SUPERSEDES
  graph_ops.py                # verified wrapper over Cognee's graph + vector engines
  seed.py                     # demo corpus: the Company-X recall investigation
  tasks/
    detect_contradictions.py  # two-gate detector (vector prefilter + skeptical-LLM judge)
    propagate_refutation.py   # grounded-fixpoint refutation cascade + hypothesis promotion
    cascade_forget.py         # surgical orphan delete (graph + vector), keeps provenance
  falsify.py                  # orchestration: build_graph(), revise(new_fact), scoreboard()
  utils.py                    # interactive HTML viz + BEFORE/AFTER console state
tests/
  test_falsify.py             # cascade / diamond / cycle-safety / surgical forget / promote
  test_detect.py              # detector: pinned demo path + Gate-1 filter + Gate-2 thresholds
  conftest.py                 # FakeGraph fixture — key-free, DB-free in-memory engine stand-in

---

How FALSIFY Maps to the Judging Criteria

#	Criterion	How FALSIFY nails it
1	Potential Impact	Solves belief-level amnesia — AI confidently remembering facts that have been proven false. Every research, legal, medical, or intelligence copilot needs memory that can be revised, not just appended.
2	Creativity / Originality	Reframes graph nodes as stateful beliefs (alive / refuted / superseded / invalidated) and treats "changing your mind" as a first-class graph operation — not chat history, not RAG.
3	Technical Excellence	Custom memify extraction + enrichment tasks; deterministic-first two-gate contradiction detection; forward BFS propagation with cycle/diamond-safe visited sets; dual-store surgical delete; on-node persistent truth-state.
4	Best Use of Cognee	Drives the v1.0 memory API end-to-end — remember(session_id) → recall() → improve() → surgical forget() — plus custom memify tasks operating directly on set_node_truth_state / get_neighborhood / delete_nodes.
5	UX / Presentation	One-screen, 30-second beat: paste one fact → watch A collapse and B ignite → read the FALSIFY-vs-RAG scoreboard. Force-graph colored by belief state.
6	Documentation & Reproducibility	python main.py runs in ~2 min with zero external services; graceful no-key exit; full README + REQUIREMENTS + 11 key-free tests + demo script.

---

Hackathon Track & Theme

• Event: The Hangover Part AI: Where's My Context?
• Track: 🏆 Best Use of Open Source — built entirely on open-source Cognee with self-hosted, zero-dependency defaults (LanceDB + Ladybug + SQLite).
• Category / Theme: 🔬 Research & Knowledge Copilot — a research assistant whose memory revises its beliefs as new evidence arrives.

The theme asks "Where's my context?" FALSIFY's answer: the context isn't lost — it was wrong, and the AI should revise it, not blindly recall it. Runs in ~2 minutes on fully self-hosted, zero-external-service Cognee (LanceDB + graph engine + SQLite), with an offline keyless demo mode — or bring your own API key (OpenAI, OpenRouter, DashScope, Azure, or any OpenAI-compatible endpoint) for live LLM-judged contradiction detection. Includes an 11-test suite that needs no API key.

---

Future Work

• Confidence-weighted partial refutation — decay a Conclusion's confidence continuously instead of a binary alive/invalidated flip.
• Multi-hop evidence provenance UI — click any node to trace the full chain of why it lives or died.
• Automated evidence ingestion — stream documents in and let the two-gate detector surface contradictions proactively.
• Human-in-the-loop review — queue borderline LLM verdicts (0.4–0.6 confidence) for analyst confirmation before cascading.
• Belief-diff export — a git-style diff of the belief graph between any two epochs.
• Neo4j / Postgres backends — swap the graph engine for a distributed store with no code change (Cognee adapter interface).

---

Acknowledgments

• Cognee — the open-source AI memory platform FALSIFY is built on. Its truth-state graph APIs, custom memify pipeline, and self-hosted defaults made belief revision possible without a single external service.
• WeMakeDevs — for hosting The Hangover Part AI hackathon and championing open-source builders.

---

FALSIFY — the AI that revises, not forgets.