README.md

---
title: ThoughtDAG
emoji: 📊
colorFrom: purple
colorTo: blue
sdk: static
pinned: false
short_description: Editable LLM context and intervention benchmarks
---

# ThoughtDAG

**Visible, editable context for LLM conversations — and reproducible experiments on context intervention.**

ThoughtDAG is a local-first, human-in-the-loop research workbench where the graph is not decoration: **wires determine the context sent to the model**. The same representation is used to study what happens when erroneous information enters a conversation, propagates downstream, and is later pruned or recomputed.

[Website](https://chenxiachan.github.io/thoughtdag/) · [GitHub](https://github.com/chenxiachan/thoughtdag)

## Context Intervention Benchmark

The benchmark represents each task as a portable ThoughtDAG graph and evaluates five paired conditions:

1. **clean** — the correct context chain;
2. **polluted** — an incorrect claim enters the chain;
3. **source prune** — the original error is removed while downstream descendants remain;
4. **subgraph prune** — the source and contaminated descendants are removed;
5. **recompute descendants** — the source is removed and dependent nodes are regenerated in order.

### Context Repair Pilot v1

`4 models` · `9 task families` · `3 propagation depths` · `540 conditions`

Across 72 cases where clean context was answered correctly and polluted context produced a wrong answer:

| Intervention | Repaired cases |
|---|---:|
| Remove the erroneous source only | 68 / 72 |
| Recompute descendants in dependency order | 71 / 72 |
| Remove the contaminated subgraph | 72 / 72 |

This pilot studies **context intervention**, not hidden model memory and not general model quality. Tasks are synthetic and objectively scored; each endpoint was sampled once at temperature zero with provider-default reasoning behavior.

[Read the case study](https://chenxiachan.github.io/thoughtdag/stories/context-repair/) · [Full methodology and results](https://chenxiachan.github.io/thoughtdag/research/context-repair-pilot-v1/) · [Browse the benchmark](https://github.com/chenxiachan/thoughtdag/tree/main/benchmark)

## What will be published here

- **Dataset:** portable graph cases, compiled requests, traces, scores, and provenance.
- **Space:** an interactive explorer for models, pollution families, propagation depth, and repair strategies.
- **Updates:** new model runs and new intervention families, versioned rather than folded into an unqualified leaderboard.

ThoughtDAG is MIT licensed. Benchmark limitations, model provenance, and scoring rules are published alongside every result.