ThoughtDAG

Visible, editable context for LLM conversations — and reproducible experiments on context intervention.

ThoughtDAG is a local-first, human-in-the-loop research workbench where the graph is not decoration: wires determine the context sent to the model. The same representation is used to study what happens when erroneous information enters a conversation, propagates downstream, and is later pruned or recomputed.

Website · GitHub

Context Intervention Benchmark

The benchmark represents each task as a portable ThoughtDAG graph and evaluates five paired conditions:

1. clean — the correct context chain;
2. polluted — an incorrect claim enters the chain;
3. source prune — the original error is removed while downstream descendants remain;
4. subgraph prune — the source and contaminated descendants are removed;
5. recompute descendants — the source is removed and dependent nodes are regenerated in order.

Context Repair Pilot v1

4 models · 9 task families · 3 propagation depths · 540 conditions

Across 72 cases where clean context was answered correctly and polluted context produced a wrong answer:

This pilot studies context intervention, not hidden model memory and not general model quality. Tasks are synthetic and objectively scored; each endpoint was sampled once at temperature zero with provider-default reasoning behavior.

Read the case study · Full methodology and results · Browse the benchmark

What will be published here

• Dataset: portable graph cases, compiled requests, traces, scores, and provenance.
• Space: an interactive explorer for models, pollution families, propagation depth, and repair strategies.
• Updates: new model runs and new intervention families, versioned rather than folded into an unqualified leaderboard.

ThoughtDAG is MIT licensed. Benchmark limitations, model provenance, and scoring rules are published alongside every result.