AndrewRqy
Add 3-playthrough sample trace + README (Sharing-is-Caring deliverable)
08be558
|
Raw
History Blame Contribute Delete
1.54 kB

Sample LLM traces — Sharing-is-Caring badge

These JSONL files are unedited captures of every LLM call from real playthroughs of The Apprentice. One JSON object per line, schema documented in ../README.md.

Files

File Records Calls Notes
three-playthroughs-fantasy-en.jsonl 117 All oracle-wizard-lora (the fine-tune) Three back-to-back playthroughs in one Python process. Default Tobin / the Hollow hero+village, English, fantasy theme. ~422k prompt tokens / ~43k output tokens total. 90 JSON-mode calls (resolutions + fork picks) + 27 text-mode calls (interludes + epilogue + theme expansion).

Quick exploration

Inspect with jq:

# Distinct models served
jq -r '.model_returned' three-playthroughs-fantasy-en.jsonl | sort -u

# Just the responses
jq -r '.response' three-playthroughs-fantasy-en.jsonl | head -3

# Calls by mode
jq -r '.mode' three-playthroughs-fantasy-en.jsonl | sort | uniq -c

# Latency distribution (ms)
jq -r '.latency_ms' three-playthroughs-fantasy-en.jsonl | sort -n | awk '
  {a[NR]=$1; sum+=$1} END {
    print "min:",a[1],"p50:",a[int(NR/2)],"p95:",a[int(NR*0.95)],"max:",a[NR],"mean:",sum/NR
  }'

How these were generated

LLM-call tracing is default-on in this app. Set ORACLES_TRACE_DISABLE=1 to opt out. Each session writes to traces/oracles-trace-<12-char-hex>.jsonl. The curate_trace script copies one of those session files into this directory under a friendlier label.