Spaces:
Running
Running
| title: traceforge | |
| emoji: 🧭 | |
| colorFrom: indigo | |
| colorTo: green | |
| sdk: gradio | |
| sdk_version: 6.30.0 | |
| app_file: app.py | |
| pinned: false | |
| license: other | |
| short_description: Turn real agent traces into ideal-world references | |
| # traceforge | |
| Turn a real agent trace into its **ideal-world reference**: the same work, | |
| with the dead ends removed, gated for structure and grounding. | |
| This Space is the portable face of the traceforge pipeline. The pipeline | |
| itself is unchanged — triage labels every message, a windowed rewriter | |
| produces the ideal path, and a judge admits only traces that pass the six | |
| pillars. The UI hides that machinery behind three actions. | |
| ## Submit a trace | |
| Upload a JSON/JSONL trace (traceforge native, OpenAI chat, or a bare message | |
| list) or paste it. The app computes a **content hash** of the seed and | |
| queues it. | |
| **One seed, one output.** The hash covers the conversation content — message | |
| text, tool calls and the set of tools the agent had — and excludes session | |
| ids, timestamps, cwd and tool-call ids. Submitting the same trace twice | |
| returns the existing output instead of creating a second one. That is what | |
| makes the output dataset a set, not a log. | |
| ## Sponsor a dataset | |
| Upload a JSONL of traces. Every row is hashed; new seeds are queued and | |
| attributed to the sponsor name you give, known seeds are skipped. Processed | |
| rows join the public output dataset. The dashboard shows the sponsor | |
| leaderboard. | |
| ## Dashboard | |
| Public statistics over this Space's corpus and, when bundled, the reference | |
| corpus: seeds, outputs, queue, failure rate, task mix, languages, runtime | |
| distribution, and **time sensitivity**. | |
| ## Compare seed → output | |
| A diff of the seed against its ideal rewrite, one row per seed message, with | |
| the reason it changed: **removed** (a dead end the ideal path routes around), | |
| **rewritten** (same work, better shape — inline word diff), **added** (a turn | |
| the writer introduced), **unchanged** (collapsed). The reason is the triage | |
| verdict for that message, so the rewrite is auditable instead of opaque. | |
| ## Model endpoint | |
| The Space has no on-device model. It talks to an OpenAI-compatible endpoint | |
| configured with the `TRACEFORGE_BASE_URL`, `TRACEFORGE_LLM_KEY` and | |
| `TRACEFORGE_MODEL` secrets. Without them the app still starts and the | |
| dashboard still works; the Submit tab says the pipeline is not configured. | |
| The rewrite is gated structurally, for grounding, and for identity, then a | |
| rewrite identical to the seed is rejected as a no-op. | |
| A seed that already reads as ideal comes back as **no change** rather than | |
| an output or a failure — the pipeline had nothing to edit. | |
| ## Time sensitivity | |
| A trace whose correct answer is a fact that changes over time is a bad | |
| training target when the agent had no way to look that fact up. Train on it | |
| and the model learns to *assert* the fact from its weights — "the current | |
| president is X" — which is wrong the moment X changes. | |
| The same question is a good target when the agent had a retrieval tool. The | |
| ideal path calls the tool and answers from its result, so what the model | |
| imitates is the procedure (search, read, answer), not the fact. The | |
| procedure does not decay. | |
| | class | meaning | | |
| |---|---| | |
| | `time_sensitive` | temporal fact, no retrieval tool — the answer decays | | |
| | `time_anchored` | temporal fact, but a retrieval tool can fetch it | | |
| | `not_time_sensitive` | no time-dependent fact in the task | | |
| `time_anchored` carries a sub-flag: whether the tool was actually **called**. | |
| A temporal task with a search tool available but never used is anchored in | |
| principle — the ideal rewrite should make the agent call it — but the raw | |
| trace answered from weights, so it needs a rewrite rather than a pass. | |
| The detector is deliberately conservative: a false `time_sensitive` throws | |
| away a good trace, so a pattern must name the **fact** (an officeholder, a | |
| price, a released version, the weather), not just a temporal word. "The | |
| current branch" and "the latest fix round" are local artifacts of the | |
| session, not decaying facts. | |
| ## Configuration | |
| Set these as Space secrets / variables: | |
| | variable | meaning | | |
| |---|---| | |
| | `TRACEFORGE_BASE_URL` | OpenAI-compatible endpoint (`…/v1`) | | |
| | `TRACEFORGE_LLM_KEY` | bearer token | | |
| | `TRACEFORGE_MODEL` | writer/judge model id | | |
| | `TRACEFORGE_TRIAGE_MODEL` | cheap labeler (default: `TRACEFORGE_MODEL`) | | |
| | `TRACEFORGE_JUDGE_MODEL` | judge override (default: `TRACEFORGE_MODEL`) | | |
| | `TRACEFORGE_DATA_DIR` | store root (default `./data/space`) | | |
| | `TRACEFORGE_SPACE_WORKERS` | parallel pipeline workers (default 1) | | |
| Without an endpoint the app still starts and the dashboard still works; the | |
| Submit tab reports that the pipeline is not configured instead of hanging. | |
| ## Storage | |
| The store is content-addressed on disk: | |
| ``` | |
| data/space/index.jsonl append-only journal (last row per hash wins) | |
| data/space/seeds/<hash>.json the seed exactly as submitted | |
| data/space/outputs/<hash>.json the improved trace | |
| ``` | |
| A Hugging Face Space's disk is ephemeral unless persistent storage is | |
| attached. For a durable public corpus, mount persistent storage at | |
| `/data` and set `TRACEFORGE_DATA_DIR=/data/space`, or sync the store to a | |
| dataset repo. | |
| ## Local use | |
| ```bash | |
| pip install -e ".[webui]" | |
| python app.py # http://127.0.0.1:7860 | |
| traceforge space serve --port 7860 | |
| traceforge space hash trace.json # print the seed hash | |
| traceforge space stats # store census | |
| ``` | |
| ## License | |
| Private research corpus. All rights reserved. | |