the-apprentice / traces /README.md
AndrewRqy
Trace-sharing infra: traces/sample is committable, add curation script + README badge claim
fd94fc8
|
Raw
History Blame Contribute Delete
2.37 kB
# LLM-call traces
The Apprentice writes one JSONL record per LLM call into this directory
at runtime (default-on; opt out with `ORACLES_TRACE_DISABLE=1`).
## Directory layout
- `oracles-trace-<session>.jsonl` — raw session traces. **Gitignored.**
Each session writes a fresh file named with a 12-char UUID.
- `sample/` — committed sample traces from real playthroughs. **Shipped
with the repo** as the Sharing-is-Caring badge deliverable.
## What's in a record
Each line is one JSON object describing a single LLM call:
```json
{
"ts": <unix epoch float>,
"session": "<12-char hex session id>",
"mode": "json" | "text",
"model": "oracle-wizard-lora", // legacy alias
"model_requested": "oracle-wizard-lora",
"model_returned": "oracle-wizard-lora", // server echo
"using_lora": true, // computed from model_returned
"temperature": 0.9,
"max_tokens": 900,
"system": "<full system prompt>",
"user": "<full user prompt>",
"response": "<raw LLM response, JSON or plain text>",
"latency_ms": 4231,
"usage": {"prompt_tokens": ..., "completion_tokens": ..., ...}
}
```
## Curating a sample for the badge
After a playthrough:
```bash
# Find your session's trace
ls -lt traces/oracles-trace-*.jsonl | head -1
# Copy it to the sample dir under a friendlier name
cp traces/oracles-trace-<id>.jsonl traces/sample/full-playthrough-fantasy-en.jsonl
# Optionally scrub: if you wrote anything personal as an oracle, run
# the included scrub helper (see scripts/scrub_trace.py).
git add traces/sample/full-playthrough-fantasy-en.jsonl
git commit -m "Add sample trace from fantasy/English playthrough"
git push
```
## Privacy note
Traces include every prompt and response. The system prompt is template
text (no PII). The user prompt is the obstacle setup + the player's
oracle text — so if a player typed their own name or anything sensitive
as an oracle, that lands in the trace. The hero name and village name
also get baked into prompts. If you don't want any of that public, don't
copy that session's trace into `sample/`.
The default hero name "Tobin" and default village "the Hollow" are
already non-personal placeholders; sticking with the defaults makes any
trace safe to share verbatim.