Blame for README.md

---

title: traceforge
emoji: 🧭
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 6.30.0
app_file: app.py
pinned: false
license: other
short_description: Turn real agent traces into ideal-world references
---


# traceforge

Turn a real agent trace into its **ideal-world reference**: the same work,
with the dead ends removed, gated for structure and grounding.

This Space is the portable face of the traceforge pipeline. The pipeline
itself is unchanged — triage labels every message, a windowed rewriter
produces the ideal path, and a judge admits only traces that pass the six
pillars. The UI hides that machinery behind three actions.

## Submit a trace

Upload a JSON/JSONL trace (traceforge native, OpenAI chat, or a bare message
list) or paste it. The app computes a **content hash** of the seed and
queues it.

**One seed, one output.** The hash covers the conversation content — message
text, tool calls and the set of tools the agent had — and excludes session
ids, timestamps, cwd and tool-call ids. Submitting the same trace twice
returns the existing output instead of creating a second one. That is what
makes the output dataset a set, not a log.

## Sponsor a dataset

Upload a JSONL of traces. Every row is hashed; new seeds are queued and
attributed to the sponsor name you give, known seeds are skipped. Processed
rows join the public output dataset. The dashboard shows the sponsor
leaderboard.

## Dashboard

Public statistics over this Space's corpus and, when bundled, the reference
corpus: seeds, outputs, queue, failure rate, task mix, languages, runtime
distribution, and **time sensitivity**.

## Compare seed → output

A diff of the seed against its ideal rewrite, one row per seed message, with
the reason it changed: **removed** (a dead end the ideal path routes around),
**rewritten** (same work, better shape — inline word diff), **added** (a turn
the writer introduced), **unchanged** (collapsed). The reason is the triage
verdict for that message, so the rewrite is auditable instead of opaque.

## Model endpoint

The Space has no on-device model. It talks to an OpenAI-compatible endpoint
configured with the `TRACEFORGE_BASE_URL`, `TRACEFORGE_LLM_KEY` and
`TRACEFORGE_MODEL` secrets. Without them the app still starts and the
dashboard still works; the Submit tab says the pipeline is not configured.

The rewrite is gated structurally, for grounding, and for identity, then a
rewrite identical to the seed is rejected as a no-op.

A seed that already reads as ideal comes back as **no change** rather than
an output or a failure — the pipeline had nothing to edit.

## Time sensitivity

A trace whose correct answer is a fact that changes over time is a bad
training target when the agent had no way to look that fact up. Train on it
and the model learns to *assert* the fact from its weights — "the current
president is X" — which is wrong the moment X changes.

The same question is a good target when the agent had a retrieval tool. The
ideal path calls the tool and answers from its result, so what the model
imitates is the procedure (search, read, answer), not the fact. The
procedure does not decay.

| class | meaning |
|---|---|
| `time_sensitive` | temporal fact, no retrieval tool — the answer decays |
| `time_anchored` | temporal fact, but a retrieval tool can fetch it |
| `not_time_sensitive` | no time-dependent fact in the task |

`time_anchored` carries a sub-flag: whether the tool was actually **called**.
A temporal task with a search tool available but never used is anchored in
principle — the ideal rewrite should make the agent call it — but the raw
trace answered from weights, so it needs a rewrite rather than a pass.

The detector is deliberately conservative: a false `time_sensitive` throws
away a good trace, so a pattern must name the **fact** (an officeholder, a
price, a released version, the weather), not just a temporal word. "The
current branch" and "the latest fix round" are local artifacts of the
session, not decaying facts.

## Configuration

Set these as Space secrets / variables:

| variable | meaning |
|---|---|
| `TRACEFORGE_BASE_URL` | OpenAI-compatible endpoint (`…/v1`) |
| `TRACEFORGE_LLM_KEY` | bearer token |
| `TRACEFORGE_MODEL` | writer/judge model id |
| `TRACEFORGE_TRIAGE_MODEL` | cheap labeler (default: `TRACEFORGE_MODEL`) |
| `TRACEFORGE_JUDGE_MODEL` | judge override (default: `TRACEFORGE_MODEL`) |
| `TRACEFORGE_DATA_DIR` | store root (default `./data/space`) |
| `TRACEFORGE_SPACE_WORKERS` | parallel pipeline workers (default 1) |

Without an endpoint the app still starts and the dashboard still works; the
Submit tab reports that the pipeline is not configured instead of hanging.

## Storage

The store is content-addressed on disk:

```

data/space/index.jsonl          append-only journal (last row per hash wins)

data/space/seeds/<hash>.json    the seed exactly as submitted

data/space/outputs/<hash>.json  the improved trace

```

A Hugging Face Space's disk is ephemeral unless persistent storage is
attached. For a durable public corpus, mount persistent storage at
`/data` and set `TRACEFORGE_DATA_DIR=/data/space`, or sync the store to a
dataset repo.

## Local use

```bash

pip install -e ".[webui]"

python app.py                       # http://127.0.0.1:7860

traceforge space serve --port 7860

traceforge space hash trace.json    # print the seed hash

traceforge space stats              # store census

```

## License

Private research corpus. All rights reserved.