cognee-hackathon-project-main/README.md

<div align="center">

# FALSIFY

### the AI that revises, not forgets

**Drop one contradicting fact. Watch dependent conclusions die, a losing hypothesis rise — permanently, across sessions.**

[![Track: Best Use of Open Source](https://img.shields.io/badge/Track-Best_Use_of_Open_Source-blue)](#-hackathon-track--theme)
[![Theme: Research & Knowledge Copilot](https://img.shields.io/badge/Theme-Research_&_Knowledge_Copilot-8A2BE2)](#-hackathon-track--theme)
[![Built on Cognee](https://img.shields.io/badge/Built_on-Cognee-00C48C)](https://github.com/topoteretes/cognee)
[![Python 3.10+](https://img.shields.io/badge/Python-3.10%2B-3776AB)](https://www.python.org/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow)](LICENSE)

</div>

---

## The Problem: AI Remembers the *Wrong* Fact

Every memory layer bolted onto an LLM today is an **append-only pile of facts**. It can remember. It cannot *un-believe*.

When new evidence contradicts something the AI already "knows," today's systems do one of two bad things:

- **RAG / vector memory** keeps citing the stale fact forever — it has no notion that a belief can *die*.
- **Naive "delete the memory"** throws away the fact *and* every conclusion built on top of it, with no record of *why* — a lobotomy, not a revision.

This is **belief-level amnesia**. The AI doesn't just forget *where* it put the context (this hackathon's theme) — it confidently remembers a fact that has since been proven false, and every downstream conclusion inherits the lie.

> A forensic audit reveals the "March QA report" was **back-dated**. A human analyst instantly revises: *"Then the March timeline is dead — the January supplier email is now our best evidence."* Today's AI memory keeps answering **"March 2021."**

---

## The Solution: A Living Belief Graph

FALSIFY treats a research inquiry not as a chat log but as a **living belief graph** of typed, stateful nodes — `Hypothesis`, `Evidence`, `Conclusion` — wired together by dependency edges.

When a new fact **contradicts** an existing piece of evidence, FALSIFY performs **belief revision** on the graph itself:

1. **Refute** — the contradicted `Evidence` flips to `truth_state = REFUTED`.
2. **Propagate forward** — the refutation cascades *along `depends_on` edges* to every `Conclusion` that critically rested on it → `INVALIDATED`.
3. **Re-ignite** — the losing `Hypothesis` is demoted to `SUPERSEDED`; the strongest surviving rival is promoted as the new frontier.
4. **Surgically forget** — orphaned dead-ends (no surviving consumer) are hard-deleted from **both** the graph and the vector store. Provenance nodes (the refuted fact, the superseding fact) are **kept** so the graph always explains *why* it changed.
5. **Persist** — truth-state is written *on the node*, so the **next session's `recall()` skips the dead branches** — while a plain-RAG baseline still cites the stale fact.

The result is memory that **revises instead of forgets**: it changes its mind, keeps the receipts, and never loses the thread.

---

## Demo

<div align="center">

![FALSIFY demo — a refutation cascade collapsing Hypothesis A and igniting Hypothesis B](output/falsify-demo.gif)

*A **live** run (the LLM judges the contradiction): drop one back-dating fact → `E_qa` turns **red (refuted)** → Conclusion **K is forgotten** (deleted from graph + vector) → Hypothesis **A** dims to **amber (superseded)** → Hypothesis **B ignites** as the new frontier. Caption: **"AI revised, not forgot."***

</div>

The money shot is the **scoreboard** printed on every run:

```text
[SCOREBOARD]
   FALSIFY recall : X knew by Jan 2021 (supplier email)   [revised]
   Plain-RAG      : X knew by Mar 2021 (QA report)         [STALE]
```

Same underlying store. Same query. FALSIFY revised its belief; the baseline did not.

---

## Key Features

| | Feature | What it does |
|---|---|---|
| 🧠 | **Belief graph, not a fact pile** | Nodes are *stateful beliefs* (`alive` / `refuted` / `superseded` / `invalidated`), not immutable rows. |
| ⚡ | **Forward refutation propagation** | One contradiction cascades through `depends_on` edges and invalidates every dependent conclusion in ~3s. |
| 🎯 | **Two-gate contradiction detection** | Cheap deterministic vector prefilter (`cosine < 0.35`) → skeptical LLM adjudication (`confidence ≥ 0.6`). No hallucinated refutations. |
| ✂️ | **Surgical forget** | Orphaned dead-ends are hard-deleted from graph **and** vector; provenance is retained. Not a lobotomy — a revision. |
| 🔁 | **Cross-session persistence** | Disbelief lives on the node. Restart the process and `recall()` still skips the dead branches. |
| 📊 | **Live A/B scoreboard** | FALSIFY (revised) vs. plain-RAG (stale) side-by-side, every run — the differentiator made visible. |
| 🕸️ | **Force-graph visualization** | Nodes colored by truth-state; forgotten nodes red-flash then ripple out of the sim. |
| 🔌 | **Zero external services** | Self-hosted Cognee defaults — LanceDB (vector) + Ladybug (graph) + SQLite. OpenAI-compatible; bring any endpoint. |

---

## How It Works

FALSIFY drives Cognee's v1.0 memory API (`remember` / `recall` / `improve` / `forget`) plus a set of **custom `memify` tasks** that operate directly on the graph engine's truth-state.

```mermaid
flowchart TD
    subgraph S1["Session 1 — build the belief graph"]
        Q["InvestigationQuestion<br/>Did Company X know before the recall?"]
        HA["Hypothesis A<br/>knew via QA report, Mar 2021"]
        HB["Hypothesis B<br/>knew via supplier email, Jan 2021"]
        HC["Hypothesis C<br/>didn't know"]
        Eqa["Evidence E_qa<br/>March QA report"]
        Eem["Evidence E_email<br/>January supplier email"]
        K["Conclusion K<br/>X knew by March 2021"]

        Q --- HA & HB & HC
        Eqa -- supports --> HA
        Eem -- supports --> HB
        K -- "depends_on (critical)" --> Eqa
    end

    NF["🆕 New fact (Session 2)<br/>Forensic audit: March QA report was back-dated"]

    subgraph REV["Belief revision — custom memify tasks"]
        direction TB
        D["1. Detect contradiction<br/>vector prefilter < 0.35  →  LLM judge ≥ 0.6"]
        R["2. E_qa → REFUTED"]
        P["3. Forward BFS on depends_on<br/>K → INVALIDATED"]
        G["4. A → SUPERSEDED &nbsp;•&nbsp; B ignites (promoted)"]
        F["5. Forget orphan K<br/>delete from graph + vector<br/>keep E_qa as refuted provenance"]
        D --> R --> P --> G --> F
    end

    NF --> D
    Eqa -.-> D

    subgraph SCORE["Scoreboard"]
        FA["FALSIFY recall → B (Jan 2021)  ✅ revised"]
        RA["Plain-RAG → March 2021 QA report  ❌ stale"]
    end

    F --> FA
    F --> RA
```

**The mechanism in one paragraph:** a `Conclusion --depends_on--> Evidence` edge is the propagation rail. Refutation seeds at an `Evidence` node; a conclusion stays justified only if it has a **grounded** critical support chain that bottoms out in a still-alive node. FALSIFY computes this as a **least-fixpoint** over the dependency graph, so one formulation correctly handles chains, **diamonds** (a conclusion survives while any critical alternative is grounded), non-critical dependencies, **and cycles** (a self-supporting loop with no grounded base collapses — and the fixpoint always terminates). Hypotheses are re-scored via their `supports` edges. Truth-state (`truth_alignment` + `truth_epoch`) is written on-node via `set_node_truth_state`, so it survives a restart and `recall()` filters on it. See [`falsify/tasks/propagate_refutation.py`](falsify/tasks/propagate_refutation.py) and [REQUIREMENTS.md](REQUIREMENTS.md) for the full algorithm, edge vocabulary, and edge-case handling.

---

## Install & Setup

### Prerequisites
- Python **3.10 – 3.14**
- An OpenAI **or any OpenAI-compatible** API key (OpenRouter, vLLM, LM Studio, Azure, …)

### 1. Clone & create an environment

```bash
git clone https://github.com/ArpitKumar8649/cognee-hackathon-project.git
cd cognee-hackathon-project

# uv (recommended)
uv venv && source .venv/bin/activate
uv pip install -r requirements.txt

# …or plain pip
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
```

### 2. Configure your key

```bash
cp .env.template .env
# then edit .env and set LLM_API_KEY
```

Minimal `.env` (OpenAI):

```bash
LLM_PROVIDER="openai"
LLM_API_KEY="sk-..."
LLM_MODEL="openai/gpt-5-mini"
```

Any OpenAI-compatible endpoint (OpenRouter shown):

```bash
LLM_PROVIDER="custom"
LLM_API_KEY="your_api_key"
LLM_MODEL="openrouter/google/gemini-2.0-flash-lite-preview-02-05:free"
LLM_ENDPOINT="https://openrouter.ai/api/v1"
```

> **Heads-up:** if you configure *only* the LLM or *only* embeddings, Cognee defaults the other to OpenAI. Either configure both or keep a valid OpenAI key handy. All databases default to **local, self-hosted** stores — no external services required.

---

## Usage

```bash
# Full run: build the belief graph, drop the contradicting fact,
# print the FALSIFY-vs-RAG scoreboard. Judges start here (~2 min).
python main.py

# Deterministic demo mode — pins the refuted evidence id so the
# cascade + forget run on real graph/vector APIs even if the LLM
# judge is flaky. This is the presentation safety net.
python main.py --demo

# Run the test suite — 11 tests, NO API key required
# (FakeGraph + mocked LLM): propagation, diamond, cycle-safety,
# surgical forget, and the two detector gates.
pytest -q
pytest -q tests/test_falsify.py     # core belief-revision cascade
pytest -q tests/test_detect.py      # two-gate contradiction detector
```

**No API key?** `main.py` prints a clear message explaining how to set `LLM_API_KEY` and exits with code **0** — it never crashes in front of a judge.

### Cross-session persistence

Truth-state is written *on the graph node*, so it survives a process restart. `main.py` re-reads the belief state fresh at the end of the run to prove the refuted branch never comes back. Run `python main.py --keep` to build on top of existing memory instead of pruning first.

### Visualization

Every run writes a self-contained interactive graph to **`output/graph.html`** — just open it in a browser (no server needed). Nodes are colored by truth-state: **alive = green**, **refuted = red (dashed)**, **invalidated = grey**, **superseded = amber**.

---

## Architecture

Full design — verified Cognee API surface, node/edge vocabulary, the exact forward-propagation algorithm, truth-state lifecycle, and every handled edge case — lives in the build contract:

- **[REQUIREMENTS.md](REQUIREMENTS.md)** — grep-verified Cognee API references, truth-state lifecycle, and edge-case matrix.

```text
main.py                       # entry point — seed + revise + scoreboard; graceful no-key exit 0
falsify/
  models.py                   # DataPoint subclasses (Hypothesis/Evidence/Conclusion) + TruthState
  edges.py                    # edge-name constants: DEPENDS_ON, SUPPORTS, REFUTES, SUPERSEDES
  graph_ops.py                # verified wrapper over Cognee's graph + vector engines
  seed.py                     # demo corpus: the Company-X recall investigation
  tasks/
    detect_contradictions.py  # two-gate detector (vector prefilter + skeptical-LLM judge)
    propagate_refutation.py   # grounded-fixpoint refutation cascade + hypothesis promotion
    cascade_forget.py         # surgical orphan delete (graph + vector), keeps provenance
  falsify.py                  # orchestration: build_graph(), revise(new_fact), scoreboard()
  utils.py                    # interactive HTML viz + BEFORE/AFTER console state
tests/
  test_falsify.py             # cascade / diamond / cycle-safety / surgical forget / promote
  test_detect.py              # detector: pinned demo path + Gate-1 filter + Gate-2 thresholds
  conftest.py                 # FakeGraph fixture — key-free, DB-free in-memory engine stand-in
```

---

## How FALSIFY Maps to the Judging Criteria

| # | Criterion | How FALSIFY nails it |
|---|---|---|
| 1 | **Potential Impact** | Solves *belief-level amnesia* — AI confidently remembering facts that have been proven false. Every research, legal, medical, or intelligence copilot needs memory that can be *revised*, not just appended. |
| 2 | **Creativity / Originality** | Reframes graph nodes as **stateful beliefs** (alive / refuted / superseded / invalidated) and treats "changing your mind" as a first-class graph operation — not chat history, not RAG. |
| 3 | **Technical Excellence** | Custom `memify` extraction + enrichment tasks; deterministic-first **two-gate** contradiction detection; forward BFS propagation with cycle/diamond-safe `visited` sets; dual-store surgical delete; on-node persistent truth-state. |
| 4 | **Best Use of Cognee** | Drives the v1.0 memory API end-to-end — `remember(session_id)` → `recall()` → `improve()` → surgical `forget()` — plus custom `memify` tasks operating directly on `set_node_truth_state` / `get_neighborhood` / `delete_nodes`. |
| 5 | **UX / Presentation** | One-screen, 30-second beat: paste one fact → watch A collapse and B ignite → read the FALSIFY-vs-RAG scoreboard. Force-graph colored by belief state. |
| 6 | **Documentation & Reproducibility** | `python main.py` runs in ~2 min with zero external services; graceful no-key exit; full README + REQUIREMENTS + 11 key-free tests + demo script. |

---

## Hackathon Track & Theme

- **Event:** *The Hangover Part AI: Where's My Context?*
- **Track:** 🏆 **Best Use of Open Source** — built entirely on open-source Cognee with self-hosted, zero-dependency defaults (LanceDB + Ladybug + SQLite).
- **Category / Theme:** 🔬 **Research & Knowledge Copilot** — a research assistant whose memory revises its beliefs as new evidence arrives.

The theme asks *"Where's my context?"* FALSIFY's answer: the context isn't lost — it was **wrong**, and the AI should *revise* it, not blindly recall it.

---

## Future Work

- **Confidence-weighted partial refutation** — decay a Conclusion's confidence continuously instead of a binary alive/invalidated flip.
- **Multi-hop evidence provenance UI** — click any node to trace the full chain of *why it lives or died*.
- **Automated evidence ingestion** — stream documents in and let the two-gate detector surface contradictions proactively.
- **Human-in-the-loop review** — queue borderline LLM verdicts (0.4–0.6 confidence) for analyst confirmation before cascading.
- **Belief-diff export** — a git-style diff of the belief graph between any two epochs.
- **Neo4j / Postgres backends** — swap the graph engine for a distributed store with no code change (Cognee adapter interface).

---

## Acknowledgments

- **[Cognee](https://github.com/topoteretes/cognee)** — the open-source AI memory platform FALSIFY is built on. Its truth-state graph APIs, custom `memify` pipeline, and self-hosted defaults made belief revision possible without a single external service.
- **[WeMakeDevs](https://wemakedevs.org/)** — for hosting *The Hangover Part AI* hackathon and championing open-source builders.

---

<div align="center">

**FALSIFY — the AI that revises, not forgets.**

</div>