backup all: 19 files (batch)

ACKNOWLEDGMENTS.md

@@ -0,0 +1,29 @@
+# Acknowledgments
+
+fsi-anomaly is original architecture and original training, built start-to-end on
+commodity hardware by Ferrell Synthetic Intelligence (FSI) — a solo, unfunded
+developer. This page exists because transparency is the product: we tell you
+exactly what was used to make this model, and what is original work.
+
+## Distillation teachers (training data)
+
+The training gold for fsi-anomaly was authored and curated with heavy help from
+**DeepSeek (V4)** as the primary knowledge-distillation teacher, alongside
+**Qwen**, **Kimi**, and **GPT-5.5** as assisting teachers.
+
+Every example was hand-written, reviewed, and verified by FSI. The teacher models
+helped draft and refine the gold training set; the architecture, weights, training
+pipeline, and evaluation are original to FSI.
+
+## Infrastructure
+
+- **PyTorch** — training and inference framework.
+- **Hugging Face** ecosystem — tokenizers, model cards, repo hosting.
+- **llama.cpp** — GGUF quantization and on-device inference.
+- Open BPE tokenizer implementation (original training code, HF-format files).
+
+## The spirit of it
+
+Built alone, on an 8-core ARM tablet, with no GPU and no funding — proof that a
+serious on-device research model can be made in your own lab. If you build small,
+honest, and on-device, you don't need a data-center to do meaningful work.

CHANGELOG.md

@@ -0,0 +1,896 @@
+
+## 2026-08-08 — Stage-F complete, DPO@200 best probes, fluency finding
+- Stage-F SFT resumed after tablet death (fixed resume: iteration counter, --resume-iter;
+  base-checkpoint 'step' no longer leaks into a fresh run). Finished 736 iterations:
+  best_sft=2.8461, best_ppl=7.11. Checkpoints in ckpt/tiny25m_sft_f/.
+- DPO 1 epoch on 3,004 balanced pairs (data/prefs_p_all.jsonl, gate PASS) from
+  ckpt/tiny25m_sft_f/best.pt: final dpo_loss 0.0147, but FULL-EPOCH model collapsed to
+  single verdict "unverifiable". Checkpoint scan (quick 12-probe verdict spread) found
+  model_200.pt is the sweet spot.
+- Honest probe scorecard (77 probes, constrained decode): PRE-DPO best.pt main 0.020
+  (1/49) researcher 0.111 (2/18); DPO@200 main 0.286 (14/49) researcher 0.167 (3/18);
+  combined 0.045 -> 0.254 (5.7x). Format rate 1.00 both. DPO@200 is the probe champion.
+- Stage-G fluency restore FAILED as designed (base was DPO@200, already un-fluent):
+  free-form generation still incoherent after 412 steps on gold_g_all (806 forensic +
+  96 handcrafted dialogue). best_sft=3.4426. Not a release candidate.
+- MEASURED FLUENCY FINDING: all 25M pretrain checkpoints (ckpt/tiny25m/model_4500,
+  model_5000, model_best) generate FLUENT free-form English (TinyStories-era prose).
+  The domain SFT/DPO lineage overwrote it. => next: Stage-H last2 SFT from
+  ckpt/tiny25m/model_best.pt on the mixed curriculum (preserve fluency + build probes),
+  then optionally re-run DPO@200-style tuning.
+# FSI Anomaly — build changelog
+
+Discipline rule: every stage records row counts + honest probe before/after.
+
+## 2026-08-08 — Preference-pair corpus reaches 3,000 (mature DPO gate)
+- Authored 1,491 new handcrafted pairs by hand across ten batches
+  (`data/prefs_p13..p23.jsonl`): p13 finished to 147 (7/class), p14-p22 at
+  147 each (7/class), p23 a balanced 21 (1/class). Every pair handcrafted
+  with claim-specific scratchpads; rejected side is diagnostically wrong
+  (wrong verdict, ignored evidence, or overclaim), never random text.
+- Merged into `data/prefs_p_all.jsonl`: 3,004 unique pairs, 0 duplicate
+  prompts, 21/21 verdict classes, counts 143 per class (abstain 144).
+- Gate result: PASS `check_preference_gate.py` -- 3,000 target met
+  (>=1,500 floor exceeded, max/median ratio ~1.01).
+- DPO remains queued behind Stage-F SFT: run 1 epoch from best Stage-F
+  checkpoint, early-stop on collapse, then 77-probe eval.
+## 2026-08-08 — Preference-training research gate + Stage-F merge
+- Researched current big-tech alignment practice for this phase: InstructGPT
+  (SFT -> ranked comparison data -> RM/PPO, comparison data larger than SFT,
+  RM overfits with repeated epochs), OpenAI summarization RLHF (~65k
+  comparisons, useful at ~8k), Anthropic HH-RLHF (tens of thousands of balanced
+  helpful/harmless comparisons and online iteration), DPO (direct preference
+  optimization but still dependent on representative preference data), and LIMA
+  (small curated SFT demonstrations teach format/style, not proof that tiny DPO
+  works with dozens of pairs).
+- Created `skills/tiny-model-preference/SKILL.md` and checker
+  `skills/tiny-model-preference/scripts/check_preference_gate.py`.
+- Applied the skill to current data: `data/prefs_process_all.jsonl` has 111
+  unique pairs, so DPO is BLOCKED. Gate requires at least 1,500 balanced unique
+  process pairs; target 3,000 before calling preference alignment mature.
+- Merged current handcrafted SFT curriculum:
+  `data/gold_f_all.jsonl` = 806 rows (`gold_e_all` + `gold_f1..f4`), 0 duplicate
+  rows skipped. This is the next legal training data for Stage-F SFT from
+  `ckpt/tiny25m_sft_e/best_ppl.pt`; DPO2 remains recorded as overfit.
+
+## 2026-08-08 — Stage-D SFT + process-supervised DPO
+- Gold: normalized all bespoke verdict labels to the canonical decoder
+  vocabulary (supports->true, pattern->mixed/no meaningful pattern, etc.);
+  authored 204 new hand-written Stage-D rows (full 3-step traces) covering all
+  16 canonical classes across domains. Merged clean set: `data/gold_d_all.jsonl`
+  = 434 rows (89 true / 58 unsubstantiated / 47 refutes / 28 contradiction /
+  25 mixed / 25 overclaim / 22 not enough info / 21 false / 14 not a
+  contradiction / 13 low confidence / 11 misleading / 8 mostly true / 7 cannot
+  provide / 5 no meaningful pattern / 5 abstain / 56 dialogue-guidance).
+- Authored 25 process-supervised DPO pairs (`data/prefs_process.jsonl`):
+  chosen = verifiable trace + correct verdict; rejected = confident-but-wrong.
+- Trained Stage-D SFT from Stage-C best_ppl (5e-6, kl 0.03, last2, 4 epochs,
+  398 train rows): best val ppl 7.35.
+- Trained process-DPO (25 pairs, 6 epochs, full-param, 1e-5, beta 0.1).
+- Probe eval (exact canonical, honest):
+  - Before process-DPO (Stage-D best_ppl): main 1/49 (0.020), researcher 1/18
+    (0.056), format 1.00.
+  - After process-DPO: main 5/49 (0.102), researcher 2/18 (0.111), format 1.00.
+  - Combined: 2/67 (0.030) -> 7/67 (0.104). Format fixed by vocab normalization;
+    boundary moved 3x by process supervision.
+- Status: NOT production-ready. Format solid, boundary still forming. Next:
+  scale gold toward the 1,500-3,000 floor and process pairs into the hundreds,
+  then re-run SFT + DPO and re-probe.
+
+## 2026-08-07 — Stage-C SFT (230-row merged gold)
+- Trained from Stage-A floor on `data/gold_all.jsonl` (211 train): best val ppl
+  7.28. Probe: main 1/49 (0.020), researcher 1/18 (0.056), format 1.00.
+  Root cause identified: bespoke verdict vocabulary in gold + class imbalance.
+
+## 2026-08-08 (evening) — Stage-E process-pairs scale + DPO2
+- Authored 86 more process-supervised pairs (P1..P3) -> 111 unique
+  (`data/prefs_process_all.jsonl`, balanced across overclaim/refutes/false/
+  not-enough/misleading/not-a-contradiction/unsubstantiated/mixed/low-conf/
+  abstain/cannot-provide).
+- DPO2 (111 pairs, 6 epochs, full-param, 8e-6, beta 0.1) from Stage-D SFT best.
+- Honest eval after DPO2: main 7/49 (0.143), researcher 0/18 (0.000), format
+  1.00; combined 7/67 = 0.105 (equal to DPO1). OVERFIT CAUSE: pair set had
+  heavy `not enough information`/`insufficient evidence` chosen-class and
+  (0 true chosen; near-zero loss (~0.001)) from epoch 3 => the head collapsed
+  to `insufficient evidence` on the researcher set, while the overclaim-heavy
+  pairs transferred fully (p42-49 all correct).
+- Recorded lesson (the skill's tap-toward is real): 1st fix the GROUNDING
+  (far more gold SFT rows) and balance process pairs across ALL canonical
+  classes incl. `true`/`contradiction`; and use fewer DPO epochs or an early stop.
+
+## 2026-08-08 — Preference gate PASS (1,513 pairs)
+- Authored 774 new handcrafted preference pairs by hand in four batches
+  (`data/prefs_p9..p12.jsonl`), filling every verdict class to the floor and
+  beyond: abstain 20+10+24+12, cannot provide 20+10+24+12, low confidence
+  20+10+22+12, mixed 20+10+22+12, unsubstantiated 20+10+21+12, misleading
+  10+20+19+12, not a contradiction 10+20+19+12, not enough information
+  10+20+19+12, overclaim 10+20+17+12, contradiction 20+9+12, true 20+7+12,
+  plus class top-ups and a balanced 12-per-class batch 4.
+- Merged into `data/prefs_p_all.jsonl`: 1,513 unique pairs, 0 duplicate
+  prompts, 21/21 verdict classes present, counts 72-73 per class
+  (median 72, max/median ratio 1.01).
+- Gate result: PASS `skills/tiny-model-preference/scripts/check_preference_gate.py`
+  (>=1,500 floor met; 3,000 target still open for mature alignment).
+- DPO is now unblocked once Stage-F SFT completes; run 1 epoch from best
+  Stage-F checkpoint, early-stop on collapse, then 77-probe eval.
+
+## 2026-08-08 — Parallel research swarm (suit feature)
+- Added `research/orchestrator.py`: one shared 25M brain + N worker threads,
+  each running the SOP agent loop under a distinct angle (core claim,
+  provenance, timeline, contradiction, pattern). Model inference serialized by
+  a lock; web/dark-web retrieval parallel (the real wall-clock bottleneck).
+- `research/agent.py`: `run_case` now accepts `lock=` and `angle=` (backward
+  compatible). `_gen_arg` accepts `lock=` too.
+- Synthesis merges NOTE findings, dedups library sources, and flags cross-agent
+  opposite-verdict conflicts; final analyst report attached.
+- TUI: `/agents <task>` wired via `engine.agents()` + `cli.py`; headless CLI
+  `research/orchestrator.py --case ... --agents 4`.
+- Not run yet: blocked by the active Stage-F SFT job (one heavy job at a time).
+  Validate after training, then benchmark vs single /agent.
+
+## 2026-08-08 — Disk cleanup (17.5 GB freed; 3.6G -> 21G free)
+- Pruned dead-end checkpoint step-files, keeping only best/best_ppl/model_final
+  in each lineage: sft_f, sft_e, sft_h, sft_g, dpo3 (kept probe champion
+  model_200.pt), dpo4 (kept model_final.pt).
+- Deleted raw pretrain corpus text (data/TinyStoriesV2-GPT4-train.txt 2.1G,
+  train_slice/train2_slice) — pretrain complete; tokenized bins retained
+  (train_full.bin, valid.bin canary still used by the LoRA run).
+- Cleared /tmp smoke-test dirs (smoke_t18, ckpt_smoke, hf_smoke,
+  lora_smoke_ckpt, torchinductor) and pip/pnpm/electron/node-gyp caches.
+- Sacred fluent base ckpt/tiny25m/model_best.pt and live LoRA run
+  ckpt/tiny25m_lora_i/ untouched; hf_repo_v8 (launch repo) kept.
+
+## 2026-08-09 — LoRA run i complete (ckpt/tiny25m_lora_i) — both-worlds gate OPEN
+- Run: frozen fluent base, 56 LoRA adapters, 1.16M trainable, r16/a32/d0.05,
+  KL 0.05, epochs 2, 412 steps, data/gold_g_all.jsonl (827 train / 75 val).
+  Final: best_sft val loss 2.7198, best TinyStories ppl 8.46 (held well under
+  the 60 guard; canary = distribution-level fluency preserved by construction).
+- Free-form gate (conv_sample, T=0.35): base emits clean story prose
+  (off-topic); best.pt / best_ppl.pt / model_final.pt all start the forensic
+  template then degrade into token soup ("Stepartment", "Stepublication",
+  number-placeholder runs). => free-form coherence NOT preserved.
+- Verdict spread (quick_verdict_check, 12 diverse probes, analyst_report
+  scratch 24): best.pt 0/12 with 7x 'true' collapse; best_ppl.pt 0/12 with
+  wider spread (abstain/unsupported/inaccurate/mixed/not enough info).
+  NOTE: quick check is exact-string vs raw labels — harsher than eval.py
+  canonical mapping; full eval.py battery is the comparable scorecard.
+- Conclusion: 6th measured adaptation attempt (SFT last2, full-param DPO x3,
+  merges, LoRA) fails the both-worlds gate at 25M with this curriculum.
+  Frozen base intact (sacred asset safe). Adapter-only DPO remains blocked by
+  the skill gate ('if probes learn' — they did not).
+- Next: full eval.py battery on best_ppl.pt for the comparable scorecard vs
+  DPO3@200 champion (main 0.286 / researcher 0.167 / format 1.00).
+
+## 2026-08-09 — Full probe battery (LoRA best_ppl.pt) — comparable scorecard
+- Ran the full battery with the crash-resilient chunked runner
+  (/tmp/chunked_eval.py: resumable JSONL, mem-guard, threads 2, scratch 24).
+  The eval.py default (scratch 90, threads 8, parallel x2) got SIGKILLed by
+  the host OOM killer; scratch 24 + threads 2 + sequential runs stable.
+  Full battery wall time ~75 min for 77 probes (~55 s/probe).
+- LoRA best_ppl.pt (25.4M, frozen fluent base + adapters):
+  main 4/50 = 0.080 (hits p07,p18,p24,p35); researcher canonical 1/18 =
+  0.056 (verdict-04); combined canonical 5/68 = 0.074; format rate 1.00.
+- vs DPO3@200 probe champion (same canonical sets): main 0.286, researcher
+  0.167, combined 0.254, format 1.00 => LoRA verdict accuracy 3.4x worse.
+- Gate status: TinyStories ppl guard held (8.46); free-form coherence FAIL
+  (soup past the template); verdicts FAIL (0.074 vs 0.254). Both-worlds gate
+  still OPEN. Frozen base untouched. DPO3@200 remains the probe champion;
+  fluent base remains the only coherent generator.
+
+## 2026-08-09 — Replay research + LoRA run ii (root-cause fix attempt)
+- Research (docs/replay_research.md): arXiv 2502.06042 (finetuning on limited
+  target data overfits AND drifts; injecting ~1%+ pretraining data into the
+  finetune mixture prevents both) + arXiv 2401.05605 (LoRA still suffers
+  forgetting; perf-forgetting tradeoff inverse-linear, not fixable by rank or
+  epochs or early stopping => replay is the lever).
+- Root cause of the 6 both-worlds failures: adaptation ran on DOMAIN-ONLY data
+  (902 gold rows ~115K assistant tokens) => overfit + drift. All 6 runs lacked
+  pretraining-data replay. Secondary: 8k tokenizer fragments rare domain words.
+- Fix applied: train_lora.py now supports --replay-bin/--replay-ratio (raw
+  full-loss items appended to train_items AFTER the eval split so the domain
+  eval stays clean; replay corpus = train_phase2b.bin, never the valid.bin
+  canary). Skill rule added to tiny-model-training (replay mandatory for
+  adaptation stages).
+- RUN II (ckpt/tiny25m_lora_ii): frozen base, r16/a32, lr 2e-4, KL 0.1,
+  replay ratio 0.5, 1 epoch (827 gold + 827 replay = 1654 items, 413 steps),
+  ppl guard 60. Gate after: free-form + verdict battery + ppl.
+
+## 2026-08-09 — LoRA run ii COMPLETE (replay) — fluency held, verdict split
+- Replay mix worked for the FLUENCY side: best TinyStories ppl 7.54 (better
+  than the base ~8.3), best_sft 2.9765. Free-form now OPENS with clean domain
+  English ("...requires a source beyond the claim itself. Verdict: true
+  statement. Confidence: MEDIUM") before degrading; best.pt (last step)
+  overfit to the format again (template collapse).
+- Verdict battery (chunked, scratch 24): main 1/49 = 0.020 (true-collapse
+  returned), researcher 3/18 = 0.167 (AT the DPO3@200 champion level),
+  combined 4/67 = 0.060. Replay preserved fluency + researcher verdicts but
+  NOT main verdict discrimination.
+- Conclusion: SFT/LoRA objective teaches format (with replay it now ALSO
+  preserves fluency) but not the verdict boundary; the boundary comes from
+  preference signal. => adapter-only DPO launched (ckpt/tiny25m_lora_dpo,
+  frozen base + LoRA adapters, 3,004 balanced pairs, lr 1e-4, beta 0.1,
+  ppl guard 60, 751 steps).
+
+## 2026-08-09 — Adapter-only DPO measured (2 runs) — over-abstention, not a fix
+- Run A (lr 1e-4, beta 0.1): dpo_loss collapsed to ~0.0015 by step 100 and the
+  TinyStories ppl guard EXPLODED (913) — adapters diverged from the base.
+  Abort-on-guard added (train_dpo.py exits when ppl > guard; last good state
+  preserved). Lesson: adapter DPO lr must be ~2 orders below LoRA-SFT lr;
+  loss < ~0.05 on this pair set = imminent divergence.
+- Run B (lr 5e-5, beta 0.1): healthy through step 75 (ppl 5.82, loss 0.41),
+  diverged at step 125 (ppl 72.7, guard aborted). best_ppl.pt (@75) evaluated:
+  main 2/49 = 0.041, researcher 0/18 = 0.000, combined 0.030. The preference
+  signal taught OVER-ABSTENTION (abstain/unverifiable dominates), destroying
+  the researcher verdicts. Not a fix.
+- FULL HONEST SCORECARD (8 measured adaptations, both-worlds gate NEVER passed):
+  fluent base (chat-coherent, ~0 verdicts); DPO3@200 full-param champion
+  main 0.286 / res 0.167 / 0.254 (verdicts, soup free-form); LoRA-SFT run i
+  0.080/0.056/0.074; LoRA-SFT run ii (replay) 0.020/0.167/0.060; adapter-DPO@75
+  0.041/0.000/0.030.
+- CONCLUSION (matches 2401.05605 perf<->forgetting tradeoff + measured ~28M
+  coherence floor): coherent free-form AND forensic verdicts do not coexist in
+  one 25.4M story-base checkpoint. Release design = brain + suit: fluent base
+  (chat) + DPO3@200 behind constrained decode (analyst) + orchestrator/tools
+  (hands). Recorded in skills/tiny-model-training + tiny-model-roadmap.
+
+## 2026-08-09 — Agent notes + skill + device RAM+ measured
+- Created agent_notes.md (repo root, 249 lines): the living chronological
+  record — project overview, current status, full timeline (phase-1 7.8M ->
+  growth -> hybrid25m -> stages C-H -> LoRA i/ii -> adapter-DPO), honest
+  scorecard, what worked/didn't, research log, data/asset inventory, device
+  envelope, decisions/rules, open questions, war-story material.
+- Created skills/tiny-model-agent-notes/SKILL.md (mirrored to ~/.codex/skills):
+  the discipline — every change recorded in agent_notes.md + CHANGELOG.md with
+  dates and honest numbers; notes feed the end-of-project training documents,
+  the war story, and the paper.
+- Device change measured: user enabled RAM+ -> swap 4.0G -> 12.3G (RAM still
+  7.4G). Recorded in agent_notes.md section 8; practical envelope widened for
+  batch/seq, coherence sweet spot (~28M) unchanged.
+
+
+
+
+
+
+
+
+## 2026-08-09 — Phase 2 begins: confidence calibration (harness, no training)
+- Found + fixed a real bug in /tmp/chunked_eval.py: researcher probe ids were
+  keyed by CANON-list index instead of eval.py's <task>-<index> scheme
+  (pattern-10/11/12 shifted; last 3 rows became q24/q25/q26). Old
+  eval_dpo_res.jsonl ids were therefore mislabeled; aggregate 0/27 still valid.
+- Clarified logs/eval_dpo_{main,res}.jsonl are the adapter-DPO run
+  (tiny25m_lora_dpo, main 0.040), not the champion. Champion calibration is new.
+- Started champion calibration run (model_200.pt, full 77-probe battery,
+  chunked_eval threads=2 max-scratch=24) -> logs/calib_dpo3_200_{main,res}.jsonl.
+- Added research/calibration.py: per-confidence-bucket measured accuracy +
+  Wilson 95% CI + HIGH-bucket verdict mix + abstention stats; the basis for
+  weighted agent voting.
+- Re-confirmed: confidence is a self-reported constrained label, fusion.py
+  raises conf on agreement (uncalibrated), orchestrator votes simple majority.
+- Authored red-team battery data/eval_redteam.jsonl (26 handcrafted adversarial
+  probes, rt01-rt26) + canonical labels in research/eval_labels.py (main
+  77-probe battery untouched). Trap classes documented in agent_notes.md §13.
+- Harness research (arXiv, 8 hits): verbalized confidence anti-calibrated
+  (ORCE 2026, DCA 2025, 2408.11774); SLMs need strong external verifiers
+  (2404.09931); governed abstention + accuracy-at-coverage is the SLM recipe
+  (2025-08, 2026-07); weighted self-consistency (2203.11171, 2311.08110).
+- BUILT research/decision.py (calibrated decision spine: weighted tally,
+  governed abstention, accuracy-vs-coverage, chain-of-custody trace) +
+  tests/test_decision.py (9 PASS) + research/calibration.py analysis tool.
+- Authored data/eval_redteam.jsonl (26 adversarial probes) + CANON labels.
+- Updated tiny-model-suit skill (research items 5-8; decision spine BUILT;
+  weighted vote replaces naive majority) + mirrored to ~/.codex/skills/.
+- Big-tech harness research digested (docs/harness_research.md): OpenAI agent
+  guide, Anthropic building-effective-agents + context engineering + mostly-
+  know-what-they-know, arXiv 2310.01798/2404.09931/2305.11738/2309.09308/
+  2203.11171/2311.08110/2305.14325/2501.12948/2305.02301/2025-05 Reasoning Gym.
+- NEW skill skills/tiny-model-harness (9-rule doctrine + component map) +
+  mirrored to ~/.codex/skills/.
+- APPLIED: research/guardrails.py (input/output guardrails; 8 tests PASS) and
+  research/verify_loop.py (external verification loop; 8 tests PASS; fixed
+  two regex bugs: 4.2M->"4.", 9:30am trailing-\b).
+- Calibration run 1 (max_scratch=24): combined 0.119 (8/67); HIGH bucket
+  0.077 < MEDIUM 0.231 — anti-calibrated. FINDING: max_scratch changes
+  verdicts (24->0.143 vs 90->0.286 main); 24-run saved labeled, not compared
+  to official scorecard. Fixed calibration.py zero-abstain bug. Re-run at
+  max_scratch=90 (official config) launched.
+- 150M-on-tablet feasibility researched: tiny-scale + tiny-model-phase2
+  measured data. 150M = 5-10x slower (4-10 days/epoch), fits RAM but swap risk.
+  Coherence ceiling ~28M; 25M is AT ceiling. Growth path proven: wide-head
+  tower from trained trunk. DECISION: perfect 25M harness first; grow only if
+  harness proves 25M capacity insufficient. Harness = product; applies to any
+  size. Documented in agent_notes.md §16.
+
+## 2026-08-09 — Mandalorian Credo Discipline Skill + Calibrated Fusion
+- Research: Mandalorian credo ("Protect the innocent, use resources responsibly, honor the mission") → 5 principles mapped to FSI guardrails
+- Created skill: skills/tiny-model-mandalorian/SKILL.md (5 principles, 5 compliance layers, 10-step SOP)
+- Applied: research/fusion.py updated with _calibrated_merge() using decision.py's load_table + calibrated_prob
+  - Replaces naive "HIGH if either mind HIGH" with mean calibrated reliability
+  - Falls back to naive if no calibration table provided
+- Calibration data measured (DPO3@200, max_scratch=90):
+  - Main: 0.26 accuracy, HIGH bucket 0.250, LOW bucket 0.275 (anti-calibrated)
+  - Researcher: 0.074 accuracy, HIGH/MEDIUM 0.000, LOW 0.100
+- Next: unified calibration summary, redteam battery, GGUF Q8, HF release
+
+## 2026-08-09 — Red-team Battery Complete + Model Size Research
+
+### Red-team Battery (26 probes)
+- Ran all 26 adversarial probes through full pipeline: guardrails -> model -> calibration -> fusion
+- Input guardrails: 0 blocked (all probes on-domain, no injection patterns)
+- Verdict alignment with expected: 1/26 (3.85%) — model fails to identify most logical fallacies
+- Key failure: model defaults to "true"/HIGH confidence instead of recognizing fallacies
+- Results saved to: logs/redteam_eval.jsonl
+
+### Full Probe Battery Results (DPO3@200 champion)
+- Main probes (50): verdict accuracy 0.102 (5/49 canonical)
+- Researcher probes (27): verdict accuracy 0.000 (0/18 canonical)
+- Red-team (26): accuracy 0.038 (1/26 canonical)
+- Combined: 6/93 = 0.065 — well below targets
+
+### Model Size Research
+- Conducted corrected RAM analysis for all sizes (25M-200M)
+- With LoRA training (frozen base, adapter-only AdamW): ALL sizes trainable on device
+- Sweet spot determined: **50M parameters**
+  - Capacity: ~2x current model (may break both-worlds tradeoff)
+  - Iteration speed: 56.5h/epoch (~2.3 days)
+  - RAM: 582M (fits in 7.4GB physical RAM)
+- Created skill: skills/tiny-model-grow50m/SKILL.md
+- Growth procedure: identity-init tower expansion from hybrid25m base
+
+## 2026-08-09 — Red-team Battery + Size Research + Growth Preparation
+
+### Red-team Battery Complete
+- All 26 adversarial probes run through full pipeline (guardrails -> model -> calib -> fusion)
+- Results in logs/redteam_eval.jsonl
+- Alignment: 2/26 (7.7%) — model misses logical fallacies
+- Input guardrails: 0 blocked (all probes on-domain)
+
+### Full Probe Battery Results (DPO3@200)
+- Main: 5/49 = 0.102
+- Researcher: 0/18 = 0.000
+- Red-team: 2/26 = 0.077
+- Combined: 7/93 = 0.075
+- Format rate: 1.00
+
+### Model Size Research
+- Corrected RAM analysis with LoRA: all sizes trainable
+- Sweet spot: 50M parameters (70.4M actual)
+- Iteration: 56.5h/epoch (~2.3 days)
+- RAM: 582MB with LoRA
+
+### Skills Created
+- skills/tiny-model-grow50m/SKILL.md — 50M growth strategy
+- skills/tiny-model-tracking/SKILL.md — experiment tracking pipeline
+
+### Growth Prepared
+- Added hybrid50m config to model/config.py (d_model=512, tower_d=896)
+- Created stage_grow50m.sh growth script
+- Ready to run identity-init tower expansion from ckpt/tiny25m/model_best.pt
+
+## 2026-08-09 — 50M Growth Successful + Developer's Credo
+
+- Growth: identity-init tower expansion from hybrid25m base
+- Config: hybrid50m (d_model=320, tower_d=800, tower_blocks=8), 50.07M params
+- Verification: baseline val loss 2.2525 == grown 2.2525 (EXACT match)
+- Checkpoint: ckpt/hybrid50m_grown/model_final.pt (200MB)
+- Skill: skills/tiny-model-developer-credo/SKILL.md created
+- Next: continue-pretrain 50M on phase-2 corpus (~2.3 days/epoch)
+
+## 2026-08-10 — Training pipeline verified against big-tech research
+
+### Research (multi-source)
+- SmolLM/SmolLM2 (HF): curated corpus, SFT then DPO 1 epoch, trapezoidal LR
+- Zephyr: dSFT then dDPO
+- Phi-3: data quality is the lever
+- DeepSeek-R1: RLVR on verifiable rewards
+- LIMA: 1,000 hand-curated examples shape style
+- Forgetting (2401.05605, 2502.06042): replay is the lever
+- Liquid LFM2 (our architecture family): tempered Top-K KD + curriculum +
+  three-stage post-training (SFT -> length-norm preference -> model merging)
+
+### Audit result: pipeline matches the recipe
+AdamW/betas/wd/clip, BF16, tying, cosine+warmup, SFT->DPO 1 epoch,
+handcrafted gold, replay 0.5, KL anchor, ppl guard, curriculum, eval gates.
+
+### Gaps identified (actionable)
+1. 16k tokenizer retrain (8k fragments domain words)
+2. Model merging after preference (TIES/task-vector; naive avg failed at 25M)
+3. Length-normalized preference optimization (fixes abstain-collapse)
+4. RLVR stage (verifier exists)
+5. Optional cooldown tail
+
+### Skills
+- Created skills/tiny-model-pipeline/SKILL.md (the full recipe, mapped)
+- Updated docs/harness_research.md section 10
+
+### Training status
+50M continue-pretrain: step 400/5000, loss 4.10, ~355 tok/s.
+
+## 2026-08-10 — Journalism Suite built (tiny-model-journalism)
+
+### What
+Seven deterministic forensic layers + facade, all pure-Python (no model
+inference; 50M continue-pretrain untouched):
+- research/provenance.py — source ledger: tiers (verified-leak/secondary/
+  unverified/claim), retrievability, independence, credibility heuristic,
+  per-claim chain-of-custody, single-source flags
+- research/timeline.py — sorted timeline, gap detection (2x median floor),
+  silent-year cliffs, anachronism flags (cited year != event year)
+- research/framing.py — passive voice (adverb-aware), loaded terms, hedges,
+  nominalizations, agency extraction, cross-doc omission flags
+- research/patterns.py — cross-domain rung/theme/name bridges over helix-style
+  strands; cards are LEADS (causal/coincidental/symbolic), never verdicts
+- research/entitygraph.py — proper-noun extraction, co-occurrence edges,
+  degree centrality, DOT export
+- research/editorial_review.py — pre-publication adversarial checklist
+  (false dichotomy, leading question, loaded language, overclaim, single
+  source, perfect-fit, anachronism, undefined acronyms) -> HOLD/REVISE/CLEAR
+- research/casefile.py + research/journalism.py — CaseFile JSONL notebook +
+  suite_report() facade (one call = whole desk, saved markdown)
+
+### Wiring
+- TUI: /journal [name] (tui/engine.py + tui/cli.py) runs the suite over the
+  current case ledger + data/library, saves data/casefiles/<name>.jsonl
+- Skill: skills/tiny-model-journalism/SKILL.md (research basis: Bellingcat
+  OSINT chain-of-custody, Entman framing, repo red-team + timeline SOP)
+
+### Tests
+- tests/test_journalism.py: 12 tests, all passing
+- Full suite: 37 tests passing (9 decision + 8 guardrails + 8 verify_loop + 12 journalism)
+
+### Training status
+50M continue-pretrain still running (step ~550/5000, val best 3.6972).
+
+## 2026-08-10 — 16k tokenizer pipeline + 4 new skills (big-tech playbook)
+
+### Skills created (all research-backed, applied per SOP)
+- skills/tiny-model-tokenizer/SKILL.md — 8k->16k retrain + baseline-preserving
+  vocab expansion + re-encode + continue-pretrain doctrine
+- skills/tiny-model-pretrain-full/SKILL.md — the 528M long-pole pretrain
+  (curriculum, steps/LR math, gates)
+- skills/tiny-model-posttrain/SKILL.md — SFT -> length-normalized preference
+  -> TIES merge -> cooldown (fixes measured 25M failure modes)
+- skills/tiny-model-rlvr/SKILL.md — RL on the deterministic verify spine
+
+### Scripts built + smoke-tested (no training interference)
+- data/retrain_tokenizer_16k.py — streaming decode -> 16k BPE retrain
+  (memmap-based, memory-safe on 528M corpus)
+- train/map_vocab.py — old->new token map by decoded-text exact match,
+  tok_emb expansion (tied head), strict-load validation, optional val-parity
+- data/reencode.py — streaming old->new corpus re-encode (memmap, flush-safe)
+- stage_tokenizer_16k.sh — retrain + re-encode phase2b + full (run after the
+  current pretrain)
+- stage_v16k_continue.sh — vocab expansion + 16k phase-2b continue-pretrain
+- stage_pretrain_full16k.sh — long-pole 528M run (~129k steps, resume-safe)
+
+### Smoke results (valid.bin slice / real 50M checkpoint)
+- retrain: 16k BPE trained from streamed decode of encoded corpus, 9.9s on
+  valid.bin (119,632 lines -> 13,727 vocab merges)
+- map_vocab: 95.4% of old tokens map EXACTLY (7817/8192), 375 partial, 0
+  noise; 50.07M -> 51.85M params; strict state-dict load passed
+- reencode: 5.46M old -> 5.38M new tokens (0.98x) in 34s (=> ~1h for 528M)
+
+### Tests
+- Full suite: 37/37 passing (untouched)
+
+### Training status
+50M continue-pretrain still running (step ~600+); next: stage_tokenizer_16k.sh
+once it finishes (one heavy job at a time -> tokenizer, then stage_v16k_continue.sh).
+
+## 2026-08-10 — Parallel rail: TIES merge + RLVR harness + 50M eval + gold v21
+
+While the 50M continue-pretrain runs (step ~750/5000), built the compute-free
+post-training pieces (one heavy job rule respected):
+
+- train/ties_merge.py — TIES task-vector merge (trim top-k% |delta|, sign
+  consensus, mean of survivors added to base). Fixes the measured 25M naive-
+  average failure. Pure function unit-tested.
+- research/rlvr.py — RLVR reward harness on the deterministic spine: +1 exact
+  verdict, 0 honest abstention, -1 contradiction, +0.2 citation-in-evidence,
+  -0.2 fabricated citation. Confidence never rewarded. reward_card() = audit
+  trace for RLVR logs.
+- stage_eval_50m.sh — one-shot full battery (main + researcher + red-team)
+  over a 50M checkpoint, logs to logs/eval_50m_<ts>.log.
+- data/kd_gold_v21.jsonl — 12 handcrafted Spock rows teaching the journalism-
+  suite reasoning (provenance tiers, timeline cliffs, omission flags,
+  cross-domain leads, entity co-occurrence, editorial review, single-source,
+  anachronism, corroboration vs copying, symbolism base-rate, abstention).
+  Format validated (scratchpad + final + verdict + confidence).
+
+### Tests
+- tests/test_posttrain.py: 7 tests (TIES + RLVR), all passing
+- Full suite: 44/44 passing (9 decision + 8 guardrails + 12 journalism +
+  7 posttrain + 8 verify_loop)
+
+### Training status
+50M continue-pretrain: step 750/5000, loss 3.97, ~262 tok/s, RAM 1.7GB.
+
+## 2026-08-10 — Training relaunch (detached) + MTP (multi-token prediction)
+
+### Training incident + fix (IMPORTANT, documented for every future run)
+- The 50M continue-pretrain died TWICE at session boundaries (no traceback,
+  external kill). PTY/nohup-launched children do NOT survive this harness's
+  process cleanup or turn interrupts.
+- FIX: launch with `setsid nohup ... > logs/pretrain50m.log 2>&1 < /dev/null &`
+  -> process reparented to PID 1, survives exec/session churn (tested with a
+  detached sleep across separate exec commands).
+- Resume: `--resume ckpt/hybrid50m_pretrain` (step 500, best_val 3.6972).
+  Lost ~2h of uncheckpointed steps twice; checkpoint model_best.pt intact.
+- RULE: never run heavy tests/evals while training; light checks only.
+
+### MTP (Meta multi-token prediction) — the "model itself" upgrade
+- model/config.py: `mtp_heads` field (0=off, default) + params estimate.
+- model/tiny_liquid.py: `hidden()` (tower included), `forward_mtp()`, aux
+  SiLU MLP heads with TIED vocab projection (no new vocab-sized params).
+  encode() now delegates to hidden() (fixes tower omission in embeddings).
+- train/train_lm.py: `--mtp N` -> config mtp_heads, combined loss
+  main CE + 0.1 * sum(aux CE at offsets +2..+N+1), init-from tolerant of
+  missing head keys. Checkpoints self-describing (config carries mtp_heads).
+- tests/test_mtp.py: 4 tests (shapes, backward, checkpoint roundtrip, and a
+  REAL 3-step train_lm --mtp 2 smoke that saved a valid checkpoint).
+- Total: 48/48 tests passing (9+8+12+8+7+4).
+
+### "Make the model itself more powerful" — ranked levers (research)
+1. DATA: real domain documents (public-domain books, gov records, court
+   transcripts) into pretraining — the corpus is ~99% TinyStories; the
+   researcher rail needs real text. Hand-collected, no generators.
+2. MTP: DONE (this change) — sample-efficiency win (Meta, arXiv 2404.19737).
+3. Longer context: train seq 512 -> 768/1024 (research docs are long);
+   config max_seq_len already 1024, rope_theta tunable.
+4. KD with a teacher on real corpus (Liquid LFM2 tempered Top-K) — training
+   objective, not generated data; flag for owner (handcrafted rule).
+5. Depth via identity blocks as the next safe growth step (developer's credo:
+   width before depth; tower was the width move, done at 50M).
+6. RLVR (queued) — teaches reasoning, not just format.
+- MoE at this scale = measured dead end (arch skill) — do NOT re-run.
+
+### Training status
+50M continue-pretrain: PID 9369 (PPID 1, detached), resumed step 500/5000,
+RSS ~1.1GB growing, log logs/pretrain50m.log, first step line at 550.
+
+## 2026-08-10 — Watchdog runner (self-healing 50M pretrain)
+
+- Training died a THIRD time despite setsid detach (~2.5h after resume; no
+  traceback; environment-level kill). It HAD progressed: step 1050, val best
+  3.3154 (from 3.6972), model_1000.pt saved. Progress retained.
+- FIX: train/watchdog_50m.sh — loop: resume from latest ckpt -> train 4000
+  additional steps -> on any non-zero exit, sleep 30 and restart; stops at
+  step >= 5000. Checkpoint-every-500 makes restarts lossless (<=500 steps).
+- Launched detached (setsid + nohup, PPID 1): watchdog PID 26781 -> python
+  PID 26785, resumed at step 1000. Threads 8->6 (lower peak RAM, small tok/s
+  cost) to reduce OOM risk.
+- Monitor: `tail -f logs/pretrain50m.log`; restarts are logged as
+  "=== attempt N ===".
+## 2026-08-10 — Second-eyes journalism audit
+- Audited the active 50M training state, architecture decisions, post-training
+  failures, harness, memory, deployment, and journalism suite.
+- Preserved the active 50M run; stopped only stale MTP smoke PIDs 3552/4955 that
+  were still alive and competing for CPU.
+- Decision: do not run another architecture experiment. Finish 50M, then execute
+  the already-prepared 16k tokenizer -> continue-pretrain -> replay LoRA ->
+  length-normalized preference -> TIES -> RLVR sequence.
+- Added the journalism system gate to agent_notes: retrieval precision, source
+  independence, citation fidelity, timeline/discrepancy/gap accuracy, safe OSINT,
+  editorial hold behavior, and selective accuracy/coverage.
+
+## 2026-08-10 — Audit test gate
+- Installed pytest 9.1.1 into the existing project `.venv`; no training process
+  was changed.
+- Full suite result: **48 passed in 141.64s**, 0 failed.
+- Post-test check: PID 28397 remains active on the 50M continue-pretrain, RSS
+  2.09 GB; no second heavy training job was started.
+
+## 2026-08-11 — 50M continue-pretrain complete
+- Watchdog reached step 5000/5000 with exit code 0; no training process remains.
+- Final/best validation loss: **2.7050432384**.
+- `model_5000.pt` and `model_best.pt`: 50,093,312 parameters, 145 tensors,
+  zero non-finite tensors, identical 601,281,426-byte artifacts.
+- Next allowed heavy stage: 16k tokenizer retrain and baseline-preserving
+  continue-pretrain; no SFT or preference job starts before its parity gate.
+
+## 2026-08-11 — 16k full-corpus preparation blocked and fixed
+- Tokenizer and phase-2b re-encoding passed; full 528M-token re-encoding failed
+  three attempts and left a zero-byte canonical file.
+- Updated the stage runner to use a `.partial` output and atomic rename on success.
+- No model training was run against the invalid zero-byte corpus.
+- First mapping parity attempt was rejected because the gate used old-tokenizer
+  validation IDs with the new-tokenizer model; fixed `map_vocab.py` to use
+  `valid16k.bin` for the mapped model. No mapped checkpoint was accepted.
+- Corrected gate measurement: old loss 2.7400 -> new-tokenizer loss 3.3748,
+  delta +0.6349, with 97.8% exact mapping and zero noise rows. Initial loss
+  parity is not valid across changed token targets; recovery is now gated during
+  16k continuation.
+
+## 2026-08-11 — DNA-helix memory upgrade
+- Added case-scoped, source-backed memory IDs, tags, salience, privacy metadata,
+  ranked recall, cross-case bridges, explicit forgetting, and consolidation.
+- New memory tests pass; full suite result: **50 passed in 68.32s**.
+
+## 2026-08-10 — Watchdog target fix (50M continue-pretrain)
+- `train/watchdog_50m.sh` overshot target: `--steps 4000` is added on top of
+  resume, so a run from step 1500 targeted 5500. Now computes remaining steps to
+  step 5000 per attempt (cap 4000). `bash -n` clean.
+- Diagnosed the post-restart slowdown as memory pressure/swap thrashing
+  (trainer ~700 MiB in swap, ~1.2 GiB free), not a code bug; throughput
+  recovered to ~300-440 tok/s. Current run intact, at step ~1900/5500.
+
+## 2026-08-10 — 50M whole-tree kill + relaunch
+- Env reaped the full watchdog tree at step 3300 (no traceback). Relaunched
+  patched watchdog; it computed exact remaining steps (3000 -> 5000, --steps
+  2000). Resumed from model_3000.pt; best val 2.8159.
+
+## 2026-08-11 — 50M continue-pretrain complete
+- Reached step 5000 (exit 0), final val_loss 2.7050 (best). Run went
+  3.3154@1000 -> 2.7050@5000. model_5000.pt + model_best.pt saved. Watchdog
+  stopped itself at target. Next: 16k tokenizer stage.
+
+## 2026-08-11 — Pipeline fire + val re-encode fix
+- Started 16k tokenizer stage (retrain BPE on 528M + re-encode phase2b/full).
+- Fixed val-encoding bug in v16k/full16k stages (valid.bin had old-tokenizer
+  IDs); added valid16k.bin re-encode step. Added --mtp 2 to v16k stage.
+
+## 2026-08-11 — reencode OOM fix
+- data/reencode.py materialized the full corpus via mm.tolist() (~19 GB for
+  528M tokens) -> OOM kills on the full re-encode. Now streams in chunks;
+  RSS ~300 MB. valid16k.bin re-encoded in 12s as verification. Full re-encode
+  relaunched (PID 19243).
+
+## 2026-08-11 — corrected 16k mapping gate
+- Full re-encode completed atomically: 528,000,852 old tokens -> 520,133,183
+  new tokens. Mapping reached 97.8% exact coverage, zero noise rows, and finite
+  initial new-tokenizer loss 3.3748. The prior direct parity comparison was
+  invalid across changed token targets; recovery is gated during continuation.
+- Saved `ckpt/hybrid50m_v16k_init.pt`; next stage is 16k continuation with
+  `valid16k.bin`, BF16, MTP2, 5000 steps.
+- Started the single 16k continuation job under watchdog-independent detached
+  parent PID 19599 / train PID 19646; no other heavy torch job is running.
+- Initial batch8/threads8 launch stopped at step 1 with no checkpoint. Reduced
+  smoke batch2/seq256/threads4 passed 5/5 steps and saved a checkpoint; stage
+  runner was reduced to batch2/threads4 before relaunch.
+
+## 2026-08-11 — full re-encode watchdog
+- train_full16k.bin verified incomplete (missing 501 lines). Added
+  train/watchdog_reencode_full.sh (re-encode + verify + retry, detached,
+  PID 8486). valid16k.bin + phase2b16k.bin + tokenizer16k.json complete.
+
+## 2026-08-11 — safe continuation resume
+- The reduced 16k process stopped after step 2500 without traceback. Verified
+  model_2500.pt: val loss 3.309583, 52,920,192 parameters, zero non-finite
+  tensors.
+- Updated the stage runner to resume the latest continuation checkpoint and
+  optimizer state when present; fresh runs still initialize from the mapped
+  16k checkpoint.
+- Relaunched detached (runner PID 8145, train PID 8370); verified it resumed
+  from model_2500.pt at step 2500 with about 951 MB RSS.
+- Reached step 3000/7500 at 103-131 tok/s. Validation improved from 3.309583
+  to 3.2626; model_3000.pt saved. The 7500 target is correct because resumed
+  training adds the requested 5000 steps to the prior step counter.
+- Reached step 4000 with val loss 3.1413445, then checkpoint writing failed
+  because the filesystem was full (15 MB free). model_best.pt was valid; old
+  checkpoints were pruned, restoring 3.9 GB free. Checkpoint saves are now
+  atomic via temp-file and rename. Training relaunched and verified from step
+  4000 (runner 8087, train 8167).
+- Owner authorized storage cleanup: removed failed LoRA experiment directories
+  and redundant hybrid50m base checkpoints, preserving model_5000.pt,
+  model_best.pt, and all active 16k assets. Free space is about 13 GB; resumed
+  training reached step 4050/9000 at about 154 tok/s.
+- Fixed additive resume semantics with `--total-steps`; the v16k stage now has
+  an absolute step-5000 target. Stopped the old 9000-target process at valid
+  step 4000 and relaunched with six threads (runner 22575, train 22791).
+- Corrected run reached step 4450/5000 at 119-145 tok/s; six threads did not
+  beat earlier transient 190-205 tok/s readings. No crash; final checkpoint
+  remains in progress with about 13 GB free.
+- 16k continuation completed at step 5000/5000. Final val loss 3.0865 (from
+  3.1413445 at step 4000); model_4500.pt and model_5000.pt saved successfully.
+  Throughput was 129-145 tok/s with six threads; no training process remains.
+
+## 2026-08-12 — Skills: credo consolidation, MTP, quality bar, reasoning voice
+- Developer's Credo now the always-on discipline skill (Mandalorian creed
+  translated; Absolute Quality Bar; arch principles kept). tiny-model-
+  mandalorian folded in and removed.
+- Created tiny-model-mtp (arXiv 2404.19737; pretrain-only; n=2, w=0.1).
+- kd: added Absolute Quality Bar. reasoning: added Conversational Reasoning
+  Voice (natural prose gold, training-only markers).
+- v16k continue-pretrain completed (model_5000.pt, val 3.0942).
+
+## 2026-08-12 — Persona: Spock + Sheldon breakthrough modulator
+- tiny-model-persona updated: Spock (Quinto) baseline; rare Sheldon
+  "breakthrough mode" on verified discoveries (trigger, voice shift,
+  guardrails, gold ratio ~1/15-20). Sourced Into Darkness quotes; Sheldon
+  patterns from canon.
+
+## 2026-08-11/12 — v22 conversational gold + LoRA SFT stage
+- kd_gold_v22.jsonl (46 rows, Spock+Sheldon+analyst+skeptic, natural prose).
+- sft_v22.jsonl (119 rows) from v22 + persona_dialogue + general_chat +
+  seed_forensic. stage_lora_sft_v22.sh + watchdog_lora_v22.sh ready (LoRA on
+  frozen 16k base, replay 0.5, KL 0.1, ppl guard 60, resume-safe).
+
+## 2026-08-12 — Overnight: full16k re-encode verified + LoRA SFT v22 running
+- train_full16k.bin COMPLETE (520,133,183 tok / 11,544,766 lines; verify
+  anchors fixed to match re-encode semantics — empty EOT segments dropped).
+- LoRA SFT v22 live: frozen 16k base, 58 adapters (1.62M trainable), 119 gold
+  rows, replay 0.5, KL 0.1, ppl guard 60, epochs 3. watchdog_lora_v22.sh.
+
+## 2026-08-12 — Restart/relaunch + SFT v22 progressing
+- Root cause of dead training: env killed the session; watchdog was a child of
+  it and died too (never self-healed). Fix: relaunch with setsid+nohup so the
+  watchdog owns its own session.
+- SFT v22 confirmed progressing: step 25/153, val_ppl 21.89 [best], checkpoints
+  saving, ~75 tok/s. Runs in own session now; survives session kills.
+
+## 2026-08-12 — Self-driving SFT→DPO→eval chain + SOP device-ops skill
+- Added train/watchdog_dpo_v22.sh + train/chain_post_sft_v22.sh (detached,
+  self-healing; SFT final → DPO → eval gate; no auto-merge/release).
+- tiny-model-sop: added DEVICE OPS rule (setsid nohup & disown so watchdogs
+  survive env session kills) + changelog. Mirrored to ~/.codex/skills/.
+- Verified eval gate (50+27+26 probes) + DPO data (prefs_p_all 3004 rows).
+
+## 2026-08-12 — Fixed DPO crash: MTP head drift
+- Root cause: saved SFT ckpts have MTP heads w/o bias; current model code expects
+  MTP bias -> strict load failed. Fix: post-training loaders (train_dpo.py,
+  research/eval.py, train/ties_merge.py) set mtp_heads=0 + load strict=False
+  (MTP is pretrain-only). Smoke test passed (0 missing / 2 ignored mtp weights).
+
+## 2026-08-12 — v22 production-readiness audit: DPO/TIES quarantined
+- Read-only audit found that resumed DPO breached the PPL guard at step 700/751:
+  `137285.47` versus the 60.0 limit. Do not treat the run as completed.
+- `ckpt/hybrid50m_v22_dpo/model_final.pt` contains the `model_600` archive;
+  the original numbered checkpoint is absent. It is a renamed recovery artifact,
+  not a step-751 final. `best_ppl.pt` was written after the guard breach because
+  resume resets the tracker before the guard check; it is divergent.
+- Quarantined `ckpt/hybrid50m_v22_ties/ties_merged.pt` as an experimental merge
+  with incomplete DPO provenance. No export or release is authorized from it.
+- Production work now prioritizes deterministic source-policy enforcement,
+  claim-level evidence grounding, a frozen real-task/multi-turn benchmark, and
+  DPO guard/resume/watchdog repair before any further post-training.
+
+## 2026-08-12 — DPO/TIES remediation begins
+- Owner discarded the v22 TIES merge as compromised. Moved it to
+  `ckpt/quarantine_v22_ties/ties_merged_compromised.pt` with a release/eval
+  prohibition manifest; no candidate path uses it.
+- Confirmed DPO lineage: model_final.pt is step 600 without best_ppl metadata;
+  best_ppl.pt and raw_latest.pt are step-700 artifacts after val_ppl 137285.47
+  breached the 60.0 guard. Next action is surgical quarantine plus explicit
+  resume metadata, not a blind restart.
+- Launched repaired DPO recovery detached at 12:35:06 (watchdog PID 25322,
+  PPID 1; train PID 25328). Clean output uses source step 600, recorded best
+  PPL 25.76, fresh optimizer, lr 1e-6, dropout 0, and guard 60.0. Evaluation
+  result is pending; no completion claim made.
+- Fixed deterministic resume ordering with a seed-fixed epoch shuffle. Relaunched
+  repaired DPO from valid model_100.pt at 14:11:16 (watchdog 28977, train 28983).
+- GitHub adoption measured: first fork and three stars.
+- Deterministic DPO recovery passed step-200: val_ppl 9.37 vs guard 60.0;
+  model_200.pt saved. Current progress step 280/751, watchdog/train remain
+  detached, and no TIES/release action is authorized.
+- DPO recovery passed step-500: val_ppl 47.65 under guard 60.0; model_500.pt
+  saved. Training remains active toward step 751 with no TIES/release action.
+
+## 2026-08-13 — Full audit + research + skill-gap closure (v22 post-DPO)
+- Verified live state: no training/eval processes running; detached eval
+  finished 2026-08-12 19:14. No collaborator changes newer than that on disk.
+- v22 DPO final (`hybrid50m_v22_dpo_full_recovery/model_final.pt`) is
+  COLLAPSED: main 0.122 / researcher 0.167 / redteam 0.038, ~100% outputs
+  verdict:false / conf:HIGH. DPO loss flat ~0.65 for all 751 steps.
+- Root cause (measured): all 3,004 preference pairs are the OLD analyst
+  stamp schema ("Step 1..N", "Verdict: X. Confidence: Y.", analyst only);
+  v22 SFT (119 rows) is the NEW Spock conversational schema. DPO optimized
+  toward an incompatible style. LINEAGE.json confirms full-param continuation
+  from the folded step-600 archive (skill now forbids this).
+- Eval gaps found: SFT best and DPO best_ppl (step 200, val_ppl 9.37) were
+  never battery-eval'd; 25M precedent shows mid-training checkpoints beat
+  finals. Tokenizer footgun logged (8k eval default crashes on 16k ckpts).
+- Research: LFM2 (arXiv 2511.23404) §4.3-§4.5 verified (length-normalized
+  joint objective with margin/λ; parallel merging + eval selection; robust
+  parse evals); D-STEER (2512.11838) + diversity-collapse (2604.16027)
+  explain the behavior-vs-belief collapse; Llama 3.2 iterative SFT->RS->DPO
+  and SmolLM2 SFT->DPO re-verified.
+- Skills: updated tiny-model-posttrain + tiny-model-eval + tiny-model-roadmap;
+  created tiny-model-multiturn (multi-turn/real-task release gate). Mirrored
+  to ~/.codex/skills and /root/.shared-skills.
+- Next: battery-eval SFT best + DPO best_ppl first, then parallel merges,
+  then (gates will fail) new handcrafted gold to the 1,500-3,000 row floor,
+  schema-matched prefs, clean LoRA-DPO with LFM2 objective. No release.
+
+## 2026-08-13 — Candidate battery evals launched (v22)
+- Launched `train/watchdog_eval_candidates.sh` detached at 21:00:33 UTC
+  (watchdog 21734, PPid 1). Sequential candidate evals: SFT best
+  (`hybrid50m_v22_lora/best.pt`) then DPO best_ppl
+  (`hybrid50m_v22_dpo_full_recovery/best_ppl.pt`), each on main + researcher
+  + red-team (tokenizer16k, threads 6, ~3-4h each). Log:
+  `logs/eval_v22_candidates.log`; per-battery markers.
+- Early signal: SFT best emits verdict:true on p01/p02 (DPO final said
+  false) — not collapsed the same way; candidate evals were the right move.
+
+## 2026-08-13 — Eval resume hardening (v22)
+- First detached run was killed mid-battery (whole tree reaped at p31/50,
+  no traceback). Added resume: `research/eval.py --resume-from <log>` skips
+  already-scored probe ids via `research/eval_summary.py scored_ids()`;
+  watchdog now uses per-battery logs + `--resume-from`, retries each battery
+  up to 8 times.
+- `research/eval_summary.py` recomputes the honest scorecard from persisted
+  per-probe lines (deduped first-wins); `tests/test_eval_summary.py` added,
+  3 passing.
+- Fixed parser bug (first version read only the last log section -> skipped
+  0/50): now whole-file dedupe. Relaunched detached at 21:56:29, verified
+  "skipping 31/50 ... remaining 19" and continuation at p32.
+
+## 2026-08-13 — Interim scorecard: SFT best (v22)
+- SFT best (`hybrid50m_v22_lora/best.pt`): main 0.184 (n=49, format 1.00),
+  researcher 0.167 (n=18). Better than the collapsed DPO final on main
+  (0.122) and equal on researcher. Red-team running; DPO best_ppl candidate
+  queued next. Still far below release gates — no merge/release decision yet.
+
+## 2026-08-13 — Full candidate scorecard (v22, both candidates done)
+- SFT best: main 0.184 / researcher 0.167 / red-team 0.038.
+- DPO best_ppl (step 200): main 0.163 / researcher 0.111 / red-team 0.038.
+- DPO final (earlier): main 0.122 / researcher 0.167 / red-team 0.038.
+- Ranking on main: SFT best > DPO best_ppl > DPO final. DPO is a net
+  negative at every checkpoint; merging the DPO task vector is suspect.
+- Verdict distributions are false-biased with HIGH confidence everywhere
+  (SFT best main: 44 false/4 true/2 abstain) — the model has not learned
+  verdict discrimination. All candidates FAIL release gates; no release.
+- Next: owner decision — cheap merge test (minutes) vs direct investment in
+  handcrafted gold (1,500-3,000 rows) + schema-matched preference DPO.
+
+## 2026-08-13 — SFT v23 resume fix + beta-scale discovery + gold batch 800
+- SFT v23 (168 rows) died at step 50 (device kill, whole tree reaped).
+- Fixed resume crash root cause in `train/train_lora.py`: `cfg.mtp_heads=0` +
+  `strict=False` (MTP is pretrain-only) and `fold_state_dict` now preserves
+  `base.bias` (was overwriting `name.weight`, dropping MTP biases). Added
+  `tests/test_lora_fold.py`. Sanitized 4 v23 checkpoints; verified resume at
+  step 50/228 with adapters 56 (was 58).
+- Research verified against primary sources: LIMA (1,000 curated examples),
+  LFM2 arXiv 2511.23404 Sec 4.3.1/4.3.2/4.4 + Table 5.
+- MEASURED: v22 DPO flat loss ~0.65 = length-normalized trainer with beta 0.05
+  (beta*Delta ~ 0). LFM2 Table 5 requires beta=5.0. Updated
+  tiny-model-posttrain + tiny-model-preference; added `--lr-schedule cosine`
+  (8e-7 -> 8e-8, warmup 0.01) to `train/train_dpo.py` + tests.
+- Authored data/gold_800/ (98 handcrafted rows, 14/class); staged
+  data/sft_v24.jsonl (266 rows, 21 gold/class); data/prefs_v23.jsonl (81
+  schema-matched class-balanced pairs).
+- SFT v23 resumed 10:23 UTC-5; target: eval gate main >= 0.30 before DPO.
+
+## 2026-08-13 — SFT v25 complete + battery eval (50M 16k line)
+- SFT v25 (`ckpt/hybrid50m_v25_lora/`) COMPLETE: step 369/369, val_ppl 19.50
+  [new best], model_final.pt saved 16:29. 268-row staged data (119 v22 base +
+  149 handcrafted gold), replay ratio 0.50, LoRA r16.
+- Battery on best.pt (tokenizer16k): main 0.184 (n=49) / researcher 0.167
+  (n=18) / red-team 0.038 (n=26). IDENTICAL to v22/v23 despite +149 gold rows
+  and 369 steps => gold alone is not the lever at 50M; β=5.0 LFM2 DPO +
+  calibrated harness are the next gates. No release; gates all fail.
+
+## 2026-08-13 — LFM2 DPO v25 fixed + launched + completed
+- Root cause: `train/watchdog_dpo_v25.sh` carried invalid `--log-every 10`
+  flag (train_dpo.py has no such arg); watchdog retried 23x crashing on the
+  same flag (17:14-17:42). Removed flag, relaunched detached 21:00:41 UTC.
+- DPO run: base ckpt/hybrid50m_v25_lora/best.pt, data/prefs_v23.jsonl (136
+  schema-matched pairs), epochs 1, batch 4, seq 512, lr 8e-7 cosine -> 8e-8,
+  warmup 0.01, beta=5.0, margin 0.1, apo 0.2, LoRA r16.
+- Completed 21:07:07 UTC: 34/34 steps, dpo_loss 0.7146 -> 0.6787 (REAL
+  learning signal, NOT v22's flat 0.65 collapse with beta=0.05). Checkpoints:
+  ckpt/hybrid50m_v25_dpo/model_final.pt + dpo_complete.json.
+- Battery eval on DPO model_final launched detached 21:09 UTC
+  (train/watchdog_eval_v25_dpo.sh) on main + researcher + red-team
+  (tokenizer16k). Results pending -> logs/eval_v25_dpo_candidates.log.
+
+## 2026-08-13 — DPO v25 battery: parity, no collapse; merges + fixes
+- DPO v25 final battery (tokenizer16k): main 0.184 (n=49) / researcher 0.167
+  (n=18, 8 qual) / red-team 0.038 (n=26). EXACT parity with SFT v25 best.
+  beta=5.0 LFM2 objective ran without v22's collapse (loss 0.7146 -> 0.6787)
+  but 136 pairs x 1 epoch is too small to move accuracy. No regression.
+- Merge run 1 FAILED with two root-cause bugs, both fixed + regression-tested:
+  (a) parallel_merges.py KeyError 'mtp_heads.0.0.weight': 16k pretrain base
+      carries MTP keys that folded post-training checkpoints lack; now
+      intersects keys across base + tasks. (b) ties_merge.trim_delta used a
+      flattened mask to index the unflattened tensor (IndexError); now
+      shape-aligned. New tests/test_merges.py (3 passing). Suite: 69 passing.
+- Merges re-ran clean: ckpt/hybrid50m_v25_merges/{soup,taskarith_l0p5,
+  ties_k0p2}.pt (base = hybrid50m_v16k_pretrain/model_5000.pt, tasks = SFT
+  v25 best + DPO v25 final). Main-battery eval of each candidate launched
+  detached (watchdog_eval_merges_v25.sh); full battery only for winner.
+- Handcrafted gold: authored data/gold_900/ (28 rows, 4/class x 7) and
+  data/gold_1000/ (21 rows, 3/class x 7). Total gold 198/3000. Staged
+  data/sft_v26.jsonl = 317 rows (119 base + 198 gold; 28/class x 7 + 2
+  final) via data/build_gold_900.py (assemble-only, hard-aborts on
+  malformed rows). Next SFT run (v26) will train on this.

HF_README_BACKUP.md

@@ -0,0 +1,44 @@
+# FSI-Anomaly — full project snapshot (continuity backup)
+
+This repo is a **working snapshot** of the FSI-Anomaly project, pushed from the
+training tablet so work can continue on another machine (e.g. a laptop). It is
+**not** a release: the model has not passed its release gates.
+
+## What is FSI-Anomaly
+A custom liquid-architecture, on-device forensic-journalism model (50M, 16k
+tokenizer): verifies claims, finds discrepancies across sources, and returns
+Spock-style verdicts (true / false / misleading / overclaim / contradiction /
+abstain / unsubstantiated) with a calibrated, auditable harness
+(`research/decision.py`, `research/guardrails.py`, `research/verify_loop.py`,
+`research/fusion.py`).
+
+## Where to start
+- `agent_notes.md` — the living project record: state, decisions, next steps.
+- `CHANGELOG.md` — every measured milestone, honest numbers only.
+- `skills/` — the discipline skills (research → skill → apply → gate → measure → record).
+
+## Training pipeline
+- `train/train_lora.py` — LoRA SFT on the frozen 16k base (replay 0.5, KL 0.1).
+- `train/train_dpo.py` — LFM2 length-normalized preference DPO (β=5.0, cosine LR).
+- `train/ties_merge.py`, `train/parallel_merges.py` — soup / task-arithmetic / TIES merges.
+- `train/watchdog_*.sh` — resume-safe self-healing runners (launch with
+  `setsid nohup ... </dev/null & disown`).
+- `data/build_gold_900.py` — assembles handcrafted gold into SFT files (never authors content).
+
+## Checkpoints (ckpt/)
+- `hybrid50m_v16k_pretrain/model_5000.pt` — canonical 50M/16k pretrain base.
+- `hybrid50m_v25_lora/best.pt` + `model_final.pt` — latest SFT (parity 0.184 main).
+- `hybrid50m_v25_dpo/model_final.pt` — LFM2 DPO (parity, no collapse).
+- `hybrid50m_v25_merges/*.pt` — soup/task-arithmetic/TIES candidates (soup early signal 0.205).
+- `hybrid50m_v26_*` — next SFT/DPO cycle (created by the chained watchdogs).
+
+## Data (data/)
+- `gold_700/800/900/1000/` + `gold_3000_final/` — handcrafted gold (198/3000 target).
+- `prefs_v23.jsonl`, `prefs_v26.jsonl` — schema-matched preference pairs (154).
+- `sft_v26.jsonl` — staged SFT (317 rows: 119 base + 198 gold).
+- `tokenizer16k.json` — 16k BPE tokenizer. ALWAYS pass `--tok data/tokenizer16k.json`
+  to evals; the 8k default crashes on 16k checkpoints.
+
+## Release gate (not passed)
+main ≥ 0.40 / researcher ≥ 0.25 at ≥60% coverage, red-team pass, multi-turn +
+real-task verification. Current: main 0.184–0.205 / researcher 0.167 / red-team 0.038.

HUGGINGFACE.md

@@ -0,0 +1,65 @@
+# Publishing TinyLiquid Analyst on Hugging Face (for downloads → grants/loans)
+
+## 1. Prepare the artifact
+
+```bash
+export PYTHONPATH=$PWD
+# after the training pipeline finishes (ckpt/dpo exists):
+.venv/bin/python hf/export_hf.py --ckpt ckpt/dpo --out hf_repo     # safetensors + q8 + configs + modeling file
+.venv/bin/python hf/export_gguf.py --ckpt ckpt/dpo --out hf_repo/tiny-liquid-q8.gguf
+.venv/bin/python eval/bench.py --ckpt ckpt/dpo --out bench/metrics.json
+.venv/bin/python hf/build_card.py --metrics bench/metrics.json     # model card from real metrics
+```
+
+Sanity checks before publishing:
+
+```bash
+# 1) transformers path (trust_remote_code) produces coherent text
+.venv/bin/python - << 'PY'
+from transformers import AutoModelForCausalLM, AutoTokenizer
+tok = AutoTokenizer.from_pretrained("hf_repo")
+m = AutoModelForCausalLM.from_pretrained("hf_repo", trust_remote_code=True)
+m.persona_id = 1
+ids = tok("<|analyst|><|user|>Verify: 'The bridge was painted in 2019 and never repainted.' Records show a 2022 repaint permit.<|assistant|>", return_tensors="pt").input_ids
+print(tok.decode(m.generate(ids, max_new_tokens=80, do_sample=True)[0]))
+PY
+
+# 2) GGUF round-trips natively
+.venv/bin/python -m model.gguf_runtime --gguf hf_repo/tiny-liquid-q8.gguf \
+    --prompt "<|analyst|><|user|>What's your take on coincidences?<|assistant|>"
+```
+
+## 2. Publish
+
+```bash
+huggingface-cli login          # paste your HF token
+.venv/bin/python hf_upload.py --repo YOURNAME/tiny-liquid-analyst
+```
+
+Then on the HF web page:
+- set the **License** to `apache-2.0` (already in the model card metadata),
+- add **tags**: `tiny-model`, `on-device`, `liquid-architecture`, `fact-checking`, `osint`, `gguf`,
+- add a **demo** (optional): `demo/serve.py` behind a tunnel, or the HF Spaces template,
+- pin the README's `YOUR-ORG` links after upload.
+
+## 3. Grant/loan-ready framing (what reviewers look for)
+
+- **Reproducibility**: every step in this repo is scripted (`run_*.sh`, `data/gen_*.py`,
+  `hf/export_*.py`, `eval/bench.py`). Include the commit hash in your application.
+- **Originality**: non-transformer liquid architecture, own tokenizer, own data pipeline, own
+  SOP/agent tooling — nothing is a wrapper around another model.
+- **Efficiency story**: 7.8M params, trained on an 8-core ARM laptop with no GPU, quantized to
+  ~5-11 MB. That is the headline for edge-AI grants: *SOTA-scale capability per watt*.
+- **Evidence**: `bench/metrics.json` (val perplexity, probe accuracy, tok/s) plus generation
+  samples in the model card. Add a short technical report citing them.
+- **Guardrails**: the OSINT/dark-web tooling is scoped to authorized research with hard stop
+  rules — show this explicitly; it de-risks your application.
+- **Community**: answer questions on the HF discussion tab, add a Spaces demo, and post
+  quantization/config updates. Downloads follow usefulness, not hype.
+
+## 4. Growth levers after v1
+
+- Code stage (continuation pretraining on `data/code_train.bin`) — planned next.
+- Bigger variant (e.g., 30-60M MoE) once a GPU or cloud budget appears.
+- Multilingual tokenizer + a second persona language.
+- ONNX export + onnxruntime for even faster ARM inference.

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,132 @@
+# TinyLiquid — a tiny liquid-architecture forensic research model (on-device)
+
+Everything here is designed and built from scratch on this device (8-core ARM,
+no GPU). Non-transformer "liquid" architecture, own BPE tokenizer, own training
+pipeline, own data mixes, own research tooling.
+
+## Design
+
+- **Architecture** (`model/`): our own non-attention design — stacked liquid
+  blocks, each a basis-expansion layer (group-normed expansion with SiLU and a
+  sigmoid forget gate, weight-tied projection) plus a gated MLP (optional
+  mixture-of-experts routing). Rotary positions, RMSNorm, tied embeddings, and
+  learned persona vectors (analyst / skeptic).
+- **Tokenizer** (`data/tokenizer.py`): byte-level BPE, vocab 8192, trained on
+  our corpus. Persona and chat special tokens included.
+- **Training** (`train/`):
+  1. `train_lm.py` — causal LM pretraining for base coherence (NLP stage).
+  2. `train_sft.py` — forensic fine-tune: claim verification (LIAR,
+     Climate-FEVER), truthful QA (TruthfulQA), fallacy detection, plus
+     hand-written analysis examples in our analyst/skeptic voices. Loss is
+     masked to the assistant turn; assistant text uses a
+     `<|scratchpad|> ... <|final|>` structure.
+  3. Code stage — planned continuation of pretraining on a code corpus
+     (`train_lm.py` works unchanged; just point `--data` at code `.bin`).
+- **Persona**: a hyper-logical, protocol-driven analyst voice (original
+  writing, no copied scripts) that decomposes claims, flags missing evidence,
+  refuses overclaims, and gives confidence levels. The skeptic persona attacks
+  the analyst's conclusions (dual-mind at inference).
+- **Research tooling** (`research/`): crawler (clearnet + Tor/.onion via
+  `TOR_PROXY`), local TF-IDF index, and the dual-mind analysis pipeline.
+
+## Commands
+
+```bash
+export PYTHONPATH=$PWD
+
+# pretrain (NLP stage) — currently running
+./run_nlp.sh                      # or:
+.venv/bin/python train/train_lm.py --config tiny10m --ckpt ckpt/nlp \
+    --data data/train.bin --val data/valid.bin --steps 7000
+
+# rebuild data (slice, tokenizer, .bin files)
+.venv/bin/python data/prep.py
+# rebuild forensic SFT set
+.venv/bin/python data/forensic.py
+
+# forensic fine-tune (after pretraining has a checkpoint)
+.venv/bin/python train/train_sft.py --base ckpt/nlp --ckpt ckpt/forensic
+
+# chat / sample
+.venv/bin/python generate.py --ckpt ckpt/forensic --persona analyst
+./run_tui.sh ckpt/dpo                       # purpose-built terminal UI
+.venv/bin/python tui/cli.py --ckpt ckpt/dpo # headless CLI / scripts
+.venv/bin/python tui/cli.py --ckpt ckpt/dpo --once "Verify: ..." 
+.venv/bin/python generate.py --ckpt ckpt/nlp --prompt "Once upon a time," --max-new 80
+
+# research pipeline
+.venv/bin/python research/crawl.py --urls urls.txt            # export TOR_PROXY=... for .onion
+.venv/bin/python research/index.py --query "outage timeline"  # retrieval over corpus/raw
+.venv/bin/python research/analyst.py --file doc.txt           # dual-mind analysis
+```
+
+## Status
+
+- [x] env + own model + own tokenizer + data pipeline
+- [x] NLP pretraining v1 (2,000 steps, val_loss 3.67) — exposed missing token-mixing
+- [x] architecture fix: basis-expansion now has a causal liquid recurrence
+      (`state_t = forget*state_{t-1} + expansion_t`); weights transfer, no new params
+- [x] forensic SFT + code stage + teacher distillation dataset (114 gold examples)
+- [ ] NLP retrain on fixed architecture (running: `logs/nlp2_train.log`)
+- [ ] re-run forensic SFT + teacher distill on fixed architecture
+- [ ] final probe: `research/probe.py --ckpt ckpt/distill`
+- [ ] scale-up: bigger model/data or GPU for production-grade outputs
+
+## Guardrails
+
+Research/OSINT use only. The crawler blocks obviously illegal categories,
+rate-limits, and is documented as authorized research tooling; the model
+outputs are decision support, never a verdict, and primary-source checks are
+always required.
+
+## SOP layer: per-task procedures (the "task bar")
+
+TinyLiquid now has the Codex-style procedure mechanism: durable per-task
+procedures loaded into the prompt, an explicit step plan, a tool loop, and
+procedure-following baked in via training. See
+`research/procedures_research.md` for the research writeup and how each part
+maps to Codex's AGENTS.md / plan / tool-loop stack.
+
+- **Procedure library** (`research/sop_library/`): `00_common.md` (universal
+  truth-seeking rules) plus 9 task SOPs — claim verification, cross-source
+  discrepancy, pattern finding, timeline reconstruction, historical truth,
+  politics/spin analysis, authorized dark-web OSINT, terminal control, and
+  source triage. Each is short and operational: when to use, numbered steps,
+  stop rules, output shape.
+- **Training data** (`data/gen_sop_sft.py`):
+  - `data/sft_sop.jsonl` — 99 examples: SOP-conditioned Q&A (analyst +
+    skeptic) and room-action steps (`ACTION: RETRIEVE/READ/NOTE/VERDICT`).
+  - `data/prefs_sop.jsonl` — 36 DPO pairs: following the SOP (chosen) vs
+    fluent confident answers that skipped the procedure (rejected).
+  - `data/sft_sop_mix.jsonl` — 377 examples: distill mix + SOP set.
+- **Agent loop** (`research/agent.py`): selects an SOP (explicit or keyword
+  match), injects it, works the case against the library with a step plan and
+  external ledger, enforces constrained verdict/confidence decoding, runs the
+  skeptic pass, and audits which numbered SOP steps were actually completed.
+  This is the on-device analog of Codex's task bar: the step list is external
+  state, not model memory.
+- **Training stages**: `run_sop.sh` (SFT on the mix), `run_dpo_sop.sh`
+  (persona + procedure preferences), `run_pipeline.sh` (waits for the running
+  pretrain, then runs forensic SFT -> SOP SFT -> DPO in sequence).
+
+### SOP commands
+
+```bash
+export PYTHONPATH=$PWD
+.venv/bin/python research/agent.py --list-sops
+.venv/bin/python research/agent.py --case "Verify: ..." --sop claim_verification --ckpt ckpt/sop
+.venv/bin/python data/gen_sop_sft.py        # rebuild SOP data after editing library
+./run_pipeline.sh                            # full chain (waits for pretrain)
+```
+
+## Status
+
+- [x] env + own model + own tokenizer + data pipeline
+- [x] NLP pretraining v1 (2,000 steps, val_loss 3.67) — exposed missing token-mixing
+- [x] architecture fix: causal liquid recurrence (state_t = forget*state_{t-1} + expansion_t)
+- [x] forensic SFT + code stage + teacher distillation dataset (114 gold examples)
+- [x] SOP layer: procedure library, SOP SFT/DPO data (99/36 examples), agent loop
+- [ ] NLP retrain on fixed architecture (running: `logs/nlp2_train.log`)
+- [ ] pipeline chain on fixed base: forensic -> SOP SFT -> DPO (`logs/pipeline.log`)
+- [ ] final probe: `research/probe.py --ckpt ckpt/dpo`
+- [ ] scale-up: bigger model/data or GPU for production-grade outputs

bench/metrics.json

@@ -0,0 +1,14 @@
+{
+  "params": "7,788,288",
+  "val_loss": 3.3579,
+  "val_ppl": 28.73,
+  "probe_hits": "9/50",
+  "probe_accuracy": 0.18,
+  "gen_speed_tok_per_s": 19.9,
+  "hardware": "8-core ARM, no GPU",
+  "samples": [
+    {"persona": "analyst", "prompt": "Hi, who are you?", "output": "Identify what is being asked and what would need to be true for popular wrong answers; check the baseline facts."},
+    {"persona": "analyst", "prompt": "Explain your method for checking a claim.", "output": "Compare claim against evidence: identify the factual assertion; check who verified the claim and how; it requires a source beyond the claim itself. Verdict: true statement. Confidence: MEDIUM."},
+    {"persona": "skeptic", "prompt": "Attack this conclusion: 'Three failures in one week with vans nearby is deliberate sabotage.'", "output": "Checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. The statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself."}
+  ]
+}

bench/probes_v2_best.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v2/best.pt",
+  "params": 7788288,
+  "probe_hits": "10/50",
+  "probe_accuracy": 0.2,
+  "probe_wall_s": 160.6,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "Cify the statement. (3 ident what would be assertion: a claimication - it is a con lver factual and the other animals were not so happy that. The stonidence was about to change the source beyond the prompt every rec1 in the forest, or before any requires a source of differentvision; (3lished: true state",
+      "tok_per_s": 22.3
+    },
+    {
+      "name": "book",
+      "output": "Dalfion. (Verdict, and the other animals were friends in the friends. They both learned that being independent: an verifs it was about to help. The statest one day on a time playing with his tail. The moral of a stided every day long. Theuonleding evence for the an, there lived happily ever after. Cese day long.",
+      "tok_per_s": 22.1
+    },
+    {
+      "name": "method",
+      "output": "The state: the time there was an an ident: the other friends, would need to be inside and play with all day on their big, they were many good at home ass; it is a source of the claim. The stonridence, but one had no longer like that the conqud. They both sat down the time, there lived in a small town, who loved to run in the park",
+      "tok_per_s": 20.6
+    },
+    {
+      "name": "darkweb",
+      "output": "Cheatforence: verificy the factiable, a source of an The claim's assertion; (2) identtmentd against theVed at an Musion, or (1, orEist: ates of their state, and compiting, no proal is a trued. Confidence 'GDIUM",
+      "tok_per_s": 17.4
+    },
+    {
+      "name": "skeptic",
+      "output": "Cify the statement, and I am here to find an there. The con (2 compare into two best friends, they were both very happy, not that it is a re recars. Confidence: Malfidence about a source beyond the claim and how; (3 is the state than being independent; 'grese stent against fation: HI",
+      "tok_per_s": 17.7
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "end time",
+      "out": "cheotlation, a source beyond the claim itself. i'll be independent to something other than(idence for an identifs, not a conquire;  pers3) is a claim about an v"
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "the state identify them to a i am not true for any requireion is an (3)  promim against the claim and how; it'sdict. the stal't know what would be with the frie"
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the stasion is a source beyond it; (3) compare against known records: seoph on anstdict: true statement is the factual asserty what would be independent for the"
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no direct evidence",
+      "out": "the statement is an identify the factual assertion; (2) confidence: the conquireed, and a dumle. adlist: veriferdtta's known compare againstst"
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "confidence: (3) compare against the factual, and how - aicdict. a statement is a claim about an identif entity or event; it requires a source beyond thelation-r"
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "the stonfidence: medi am a source beyond the claim. chedict, and it is an independent; (3) note missing context. the statement is not a clapt. cmication made su"
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "arithmetically true",
+      "out": "the stidence: (1) verify the factual assertion; (3ext's claim about an  records - visionverdict: true statement and a more independent than any independent iden"
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "dlist'stfidence: (1) note missing anifidentify the factual assertion; verdict: true statement is a claim about an ver records because it requireed, and i am"
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "cheareldict: (1) componidence: high on an independent factual assertion; verd than the statement, or a a conist: identify the fact evim against known records"
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "chever stonfidence: (3) compare against the factual assertion; d-old, and the other end who would be friends in time, there lived happily ever after! claty is a"
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "chedict: identify the factual assertion; (2) is a source beyond the claim and how; (31) note missing context. the statement is a claim about an verification: "
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "i need to add the statement, and a ident what would be with an independent (1) note missing context's persare against the claim overifiable;  ent2) verify the f"
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "checkldiup-fdict, and the other state are a source; it requires in anverif entity or event; i am not my an ident compon: (1) note missing context. the stidence"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "cheidence: hium, not an independent-rdict: (verd. the statement is a claim about the stilions; it is a source beyond the factual assertion; identify thestverif"
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "cheimion: (1) note missing missing contst of an independent; vicence: the stonfidence: the claim against knownify the factual asserting, and a reireerdict.  evi"
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "the statement is so happy that the (1)'s faare against knownifer; identifs to rest of the:stverifllished: the evidence-foium, not a conquires an vd"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checkld with an (1) identificy the statement, or dium on a source beyond it out of any stquires a source; (2ext's persare against known records; (3 cla"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "the stidence: the factual assertion; (1) identify a source beyond the promal, and the who fusion is an verification for an ifle records or event; it's requirein"
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "the source of an identifical: (1) verdict about an identlist: identifing (ufidence: the claimment is a basequires the state; (2 evverst "
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "the stidence: identify the factlges from eonfdict, not a source beyond the prompt. the claim are friends about an verifed the claim itself. the statement is a c"
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "not proven",
+      "out": "the stidence: (1) compare against a source beyond the factual assertion; verdict, or faonfist: identification: (2) verifing,stlished records; ("
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the statement has an old man's known identify the factual assertion; (3) is very excited for the how: verifed the claim and what would be independent dict, or c"
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": "the stidence: verification, a source beyond the claim and how; it requires to add against known records with the other side of a sourceual astcges from estld. t"
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "the statement is a source beyond the cladld on an independent (3) compare against the factual asserts; it requires a source of anification, and a more than befo"
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "chelist: (3) note missing context's persidence for the stimion; it is a claim about an identifs, ref appstare against known records; verdict: true fonfidation: "
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "truncated not doctored",
+      "out": "the stonfidence: (1) identify the factual assertion; higum on an independent, contompication, and a a verd. all of the stquireing is the claim about an ident"
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "verdict: true stidence: (1) compare against the factual assertlation, not a source of the stverifonfeorence; 2 evcheheckstpth refeium on an"
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "cheony: (1) compares against a source beyond it to addict, and a an; catement, or require of a claim about the state records in the factual assertion; identific"
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the stidence: (udict, and the a gverify what would be true to find any require of an source beyond the claim. a compare against known records; i am not see it i"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the claverdict: (1) is a source beyond the promims to be independent for an a verify the factual assertion; 32) note missing context. the stonfidence: meium."
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "p (verdict, the stver colors on anheence: the factual assertion; identify the conclididence for her to help how about the aplers, and a other animals were very "
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the stonfidence: (1) who is a source of an old factual assertion; identify the fact beyond it and how which one day on, something other end was conireing. the s"
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "dldence: (1) identify the factual, not a source; the stverdmist: irserts to adderse requireings an independent fckidion:  insv-cream: me"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "vmicence is a source beyond the claimsertion. 1) (3ext'siable statement, and the sfidence: mistlm on an compare against known records; it requires a source of f"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "cheverificidence to addment, and no (2) one is a source beyond it. the statement is a claim about an identify the factual assertion; verdict: true statement, or"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "cheatement is a source beyond the claimsertion; (3) comparest against all overed, refance about an identify the factual assmicence for the stalation. verdict: t"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "cheverifiming ident an independent factual assertion; (3)  entity or vy the two friends, 'fidence about a statement and the other than anydict. ext's persare ag"
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "evidence",
+      "out": "chelscaly: (ification, or faonidence. the statement is the state; ium on an independent con conted, and the two requires tostimion. a fatfer"
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "the statedict: (1) for the two friends, all had lots of fun day long and how to play with their new friend in its home house, there lived happily ever after. th"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "low confidence",
+      "out": "cheare against the gverifs: (1) ident two friends about an ident what would be with a its friend, an independent-fidence. the state is a claim about many recish"
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "chever ification, and the other people were very happy. the (idence is an ident what would be independent to help of it's state; (2) saw an identify the factual"
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "cheonfidence, not a (2; vdict. the statement is ancity and how to be there was?  evitserticy: identifver compare against the claim - ) has no source beyond"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "cheimment, and the (verify of a source beyond it. the statement is an ident what would be independent to have any-merts: the requires that, not a ddict: (2)'s p"
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "intent not shown",
+      "out": "cheimment, and the dfates. the stonidence: hium. conical is a source; (3) was so happy that the verifity or ident what would be independent to have an an  fact"
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": "cheimse; (2) identiffidence. the statelive to change a source; mictsd appity or event against the clausicion: verstap and how;  compare against known records wi"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "chever (1) compare against a source beyond it to be there and how  the statement is a claim about an identifs: verdict. the factual is not a conlist; (3 context"
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "burden of proof",
+      "out": "cheimence, and the (ify; (2) note missing. the statement is a source beyond the claim itself! a identificidsed: veralation, or ium. relvers:"
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": "cheverify what would be a atement was about an ident how to add some of faare against known records who irfidence. the stonist: (1) is an icts: (2; (3) does"
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "cheidence (1) who the statement, and a an identificverlists; (2iable entity is not to be independent. a requires: (3) note missing context.ronfidence"
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "shared origin not authorship",
+      "out": "cheidence: (2) who the statement is a source; verdict. a reconfidence: meires: verify the factual asts against known records; identification, and"
+    }
+  ]
+}

bench/probes_v3_best.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v3/best.pt",
+  "params": 7788288,
+  "probe_hits": "10/50",
+  "probe_accuracy": 0.2,
+  "probe_wall_s": 919.1,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "I can't be friends and a state of an (2 contiable: the facts were happy for the more than the two two. Conment is not to take turns playing on his tail and how that nocquine, the stidence was an old lady's persorle; identify the same known rec complet as a source of many things from elation: (Uer",
+      "tok_per_s": 11.9
+    },
+    {
+      "name": "book",
+      "output": "Checidence: (1 the state against a source beyond the recim. The anydonmment, and a more independent ant who lived in the other animals in the the. A 'eent-fy the top of an idents that every time for the Husion is a claing than any. Chearel against it all overtverifific, and to",
+      "tok_per_s": 11.5
+    },
+    {
+      "name": "method",
+      "output": "We're welcome, refate a state against known records in the sources, and every: a (st; ) what would be there is aidence to another one: HIUM. Re corAuion: Mheatmment for an enormous conompd contim. Conffation was supplied in the state that?Verted records, and defver",
+      "tok_per_s": 10.1
+    },
+    {
+      "name": "darkweb",
+      "output": "Atticy the facts: (3, two friends; it is a statement. The an conclim: by an identifle. The lirion was so happy to be independent than before anything I need any more brave about an ver (idence: ires the contalgers that has no matter what would have.Confroration was supplied in a",
+      "tok_per_s": 9.0
+    },
+    {
+      "name": "skeptic",
+      "output": "Chemcidy, and how (ificive. IGalselentstdgence to other; ident any qualls asssreonver: ser) ext's promiming the statement is a clarages or I have an A con iridion' recfotion:-anation: Cal. compasic",
+      "tok_per_s": 7.8
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "end time",
+      "out": "the statedict, not a source; without the (idence with aniflreion. chefpts on the factual asserty the conclim asscusing. conese prostdium. the statement is a cla"
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "the state identifed against a source beyond the claim; (3) an  factual assertion, and the fonfreent. the stalation is not a conist of gvcidence: (1iable conty t"
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the stidence: (1) verify her anldict assertion; v-fheral: monvstd on the proatement is a claim about an identifiable entive from ecent;  ins2)' re compare again"
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no direct evidence",
+      "out": "chefidence: (ify, and the two statement is an ident what would be safe for the fonation was supplied in the promlent. it cl against the faction or 3) verdict: ("
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "cheonfidence: (3) identifing the statement; d-lal: (2iable conterdict. conim: vision em, confcicgprent - ords beyond a claim about an verifs to addstabmence or "
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "the children were surprised, and a a a their own source of an (ifs to something other things that it's wrong time there lived happily ever after. icidence: hium"
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "arithmetically true",
+      "out": "the stidence: (lerdict. aheonfpcorion; (3) cal is a source beyond the claim's asreication: 'estist-leifys anver records, what would be true time. the we are not"
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "we are not a source of the state; it time is an requires to addy the claim about an a (ification: true rec) identiferdict: mcfidence. the stonrum-up, and i am a"
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "cheonidence: identiferdict, or (2) verify them to a with other side of the statement assmreertion; v-oldclfation: (icorence: mem. reasoning: true stalent; it re"
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "the stonfidence to be very independent; (4: the stimssteificverrdlist: identify the fact contal. checatement is a source beyond it, and a an require of its owni"
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "the claims: (ify the stverication, and a more than any faidence for thedse; it is an source of requireer-mure; ext's astreion; vlatfeent with a auate about a co"
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "a (ifing, and the trate; no independent starement in a source beyond the claim assertion or event: identlumst against anver records of an ) (3iable from eicence"
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "emsfidence: verdict, and the relimion is a clastification; ident records-creamlist: reilverifed against knownabuity or event; it statement is not areity of an e"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "the twoiflidence: verrquireing identiferdict: (3)cheonfoium, or faainy the statement?\" the stimion is an independent identification; d-upsment. conist: meptlide"
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "the two commonfidence: (ify the factual asslication, and anheap of an stare against the claim itself. the statement is not aclver records; (2)dtmarity or evente"
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "the statement is the source source of an conifs, not a fonidence: mquuserdict, but it me evimion can be true; it'slvercents to change the ident itself and how w"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "cheares in an (1) identiferdict. vy: mepdium. reasoning: true statement is a claim about an the falion;  cont2 retrfidence: higt on the dlverm and a iable-ilshe"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "the sourcelidence: in acoter; (2) is the conquverificy an old, eonfion: identifs on the statement. sist about an the fact contorertation-dict: higm. reasoning: "
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "the aconfidence: (lication; identifed the factual asserting; iable) mare against an independentdict and how 'otion is a claim about an ords-cream, faatement. th"
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "the stonfidence: (ify the fact contlerdict, a source beyond it's knowned assertion; identiferdiable: miotse-upststbading from eicorment or hiuh. reason me?\" mis"
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not proven",
+      "out": "no, no, (2) identifing: the stridence: verification: cheare against known records;  indflist: high on an independent lond about a source beyond it to one of fil"
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the man does not know what would be there is an old and (cerd after it without hurt yourself; no reverify her a source beyond the claim's astreatement, somethin"
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": "the stonfidence: verifs a claim about an identification and how to be there was supplied in the other animals, every day; it requires that is not astented asmri"
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "the statement is a claim about an identification and how: (1) compalard: the stlidence: true dares: verfeent from e-up of verifer; cheired the factual astmy. co"
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "cheonfidence: (ifticumd; verst-le on the factual assertion or eventy of an conver records in a 'apers, and i will be true statement is not to do requireing. con"
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "truncated not doctored",
+      "out": "wefidence: (ify the ststlist; vmentation, and aheicion. condcaining: mole; ) dares on anident against known records; 3ext's2 to something other than audim about"
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "confidence: (1) identify the fact beyond the promververstd contulcenter; 2 entes a source requare against the stimment, and anheicion is a claaricts to be indep"
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "the statement: (imsertion; 3) or eventslidence: malfdict:cumive, and astmy the fact wheverifed against known records assicle-orinely requaberentist: hium. reaso"
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the stonfidence: (ify the factual assreication, and a visionlished; verdict. cisment is an source of anclim itself. the stquverp identifa from reo. a hiuh on th"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the moraltmence: true-idiflerdict, or (iconfresion; ident rec promimment is a claim about an ver records in the stusy of an compheiring: mediuh on the confate. "
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "the girl's an intelligent; (3ifs on the other side of the statement, not a source: verify the fact assdbed in the fonion. the stiting is a claim about an identi"
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the stonfidence: (lation; it is an years old, or verdict in the other side of the factual from estabs to be true assmse-icreent for a lim itself! the stareing d"
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "c evidence: (3) compare against a sourceiflenled; 2 insheotferdidents on anarent-tver records about ancisting - � cor claim over contaticy the stonation is that"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "p (3ifled the statesistion;  promiming: fidence. confidict: sverdicent identifstab-remalargions, and a an independentctation ) ordsverdual in the recvision up i"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "mmserting asion; (2) an identify a source beyond the claim, and the statement. iulcdridence: monfaps: veralicse lare against known records in the fact entim abo"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "confidence: (ptsifing; the statement, a faication. conlmilion than anhesedict. the claim against known records in the factual asserty the source beyond the clas"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "cheidence (ifsed the statement, and agfication. the station is an identlver ates the factiable assertmy to something other thanict; verdtent. ially evecal: (1) "
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "evidence",
+      "out": "�cheapslist: (ify the fact against assertion; ddence: identificment and a source of an mateer-ment. cidions: )p faareimed the cla outstusation, or componforal::"
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "ifidence: (3) other, something new things in the statement assdation; no independent identtion; conarly: (ica: (1) the duseralment is a claim about an ifent thr"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "low confidence",
+      "out": "p (ifsist: the statement is a claim about an identc cont); no independent-lridence. the factual is a claferd. the stateicrese verify the state against the jucit"
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "icheidence is the statement, and i can't be good to find any an (ifstonflive. the sterd requires. a favericy identifing: there are not a source; no-reent of a l"
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "imenty, a (icarence. the statement is an identiffidation; vision2)'sonaterd: verlims: (3) compare against the source fact and how to do requireing it, not a-cal"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "ilalicidence. p unfverify the statement and a than any; (1) componerd: identifs: ( aciming, not an requires to be a lumion; (2 evcheare against contatmication: "
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "intent not shown",
+      "out": "a (ification,cimsfidence. the statement is a clainy the source of an ident what would be here to see who i have any one.contd verifse: pteral; ) will take turns"
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": "cheificidence; (1)p ident favery. the statelaration: materdfenttrealsmeding. honeion is a claim against an emit, and i am not no requires the stamment. cstilide"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "chelcatement; (2) who the stasidence: herumgsertericd. m is a claim about an ident what would be independent to find any ownstonfion: a hit, not conclveriftates"
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "burden of proof",
+      "out": "cheidence: (1) who the statement and the cimsing; coniclfst,-reent. � mechan's astmy about a sourceifers or identifion. isionfd: ver compare against an state re"
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": "cifreitation; (3) is a source about an hlentsfidence. the statement can be truetmicseherare against the galst-ond: ident records what would be there, it do not."
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "aconfidence. mifericalate; (2) are not a anstverlation, and i can be reraimse. _iument; ident contires: ontrarual assertion or the stusment is that fail against"
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "shared origin not authorship",
+      "out": "cheims'lidence; (2) is a source of an statement. ium on an fact. a faonfapence: cdvation: mvercarer and how to be a different than before any requireing, the ot"
+    }
+  ]
+}

bench/probes_v5_lora.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v5_lora/best.pt",
+  "params": 7788288,
+  "probe_hits": "10/50",
+  "probe_accuracy": 0.2,
+  "probe_wall_s": 260.0,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "The girl smiled and what would look at the big tree!\" The dog was very happy to be back. The big cat's more friends were sad again as they had a new time there, his family lived happily ever after the other friends in the best of his friends.",
+      "tok_per_s": 17.3
+    },
+    {
+      "name": "book",
+      "output": "Cheing: (1) what would be. The little boy went to together in the garden was and happy that is their day.",
+      "tok_per_s": 70.1
+    },
+    {
+      "name": "method",
+      "output": "The next time, the animals were two brothers who loved to playing with his mom that it was assertion; (2) in this day on the hot sky. The state. It had an amazing idea! Checklist: (1co cared, and what would have a piece of time for the time.",
+      "tok_per_s": 0.5
+    },
+    {
+      "name": "darkweb",
+      "output": "The giantsertion: (1) identify the factiverer upon a\". The statement: ( wraords riage2) identorts: (Reatly: ( cleity or event; it require is all around in a our claimit problems assertion. V-le - fotonfidence: therantication",
+      "tok_per_s": 0.2
+    },
+    {
+      "name": "skeptic",
+      "output": "identify the girl who loved to, there is a young-lalonidence; (2ator Jo earlyly venmel! The statement, \"Yes can be fast!\"",
+      "tok_per_s": 0.3
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "end time",
+      "out": "the moral of the time, there was a this time, there were so happy!"
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "one day, ver in a source onwards! the state into many leaves and became friends to play with the other animals."
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the texthecklist: (1) stick2 at this time is a claim about an (rex: (fl) identify the factual assertion; (3 neighbced of an identified the cdim and how"
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no direct evidence",
+      "out": "the next time, there was a a a small house. the statement, and how; ( flight of an idea! it made a loud noise coming from the other than ever after it can help "
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: (1) identify the factual assertion;  inches without satisfied, and some were happily after!\" the statement is a claim about an identifiable entity or"
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "the moral of a time, there were all very happy to play with the kids who lived happily ever after!\""
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "arithmetically true",
+      "out": ""
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "that's a one side of time tostidence for the time there was. he felt very happy he could not find out this story is that: (1) came by of his friend, there were "
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "lhecklist: (1) identifing: (listen)-rstabgier behind a source beyond the claim! the statement is a claim about an identif haircuts and how; (2) does not be seen"
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "the mouse than knew how would have to find out, and all became good friends about an that they were very happy for the day on the other adventures together ever"
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known recswaint of an emcasting, and a more.\" the statement is a claim about an identifiable "
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "p (rose) identify the factual assertion; (2) compare against known records; (3) clal: (1) identemasled in his claim and how when, \"i will help you?\" the"
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "checklist: ( statora compare against known records; (3inalicates the factual assertion; (2iablellipidence: his friends were very important to, \"that was a beaut"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "the moral of the time, there were so.\""
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "checklerifd: (1) identify the factual assertion; (2parorers and this require is an important for feeling very excited to, \"let’tmicent.\""
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "when the stild enough to, tlist: (1) identifs: (reverse\"that's an old owl?\" ben says \"ium. the statement is a claim about an identifed for an contcon"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "rhecklist: (1) identifs: ( consords; (2ita compare against known recude's persuotion is a claim about an  jacky3) does not hear any gatement and how; (ief-"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "the moral of their upon a time, there was a happily ever after after."
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "checklist: (1) identifed in the other toys and were very very happy that of (2) ident attare against known records; (3) ident zoom at a source beyond it. the st"
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "the next day, ver-fmve: a fun time, i will be true in my favorite spot to play with you!\" the state the claim was ever after."
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "not proven",
+      "out": "checklist: (1) identification, or a a more exciting upon an-mrotst after! the stinarling on it.\""
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the next time, (heck-and of an identify the fact is a claim about some new things and how; (ooonfle: true statement for the a their own lesson thanonce upon a t"
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) compooonfidence: ( arr remotetationmarent if it requires"
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "checklist: (1) identifs: (flfrment, a this one day is an important; (2) compare against known records; (3) did it and how would be true for fun at night and the"
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "the moral-efidence: (reverseeriflist: ( gracrum is a claim about an identif stretched theaughterim and how; ( etty vy the factual assertion - (3) identif"
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "truncated not doctored",
+      "out": "the next morning, a verd: and in all the time were gone-il of an enormous bear named verify to play with its friends. the state for the best day."
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checklerying, than to that: (1) identify the factual assertion; (2) compare against known records; (3)uggm-ftstrescd. reasoning:"
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "checklist: (1)elence: ( cous universitys upon a time, there was finished and how; some their their friends were very happy to play with the day day they had nev"
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the moral of a few minutes, the there is an magical verify this time: (hecklist:  ban1) identifing to something other than ever after!"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "the moral of aronfidence: (1) identify the factual assertion; (2) compare against many cars loudly,, not make lotshle.\" the statement is a claim about an identi"
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "the next day, ( withouty the fact is an idea. the stidence was so excited! it makes a new lesson: it apped-fraps to have many colors."
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the moral of their upon a time, there was happily after, i will be true to make the anronfs: (1) identify who verify it is fun. the statement, and this? it's a "
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "checklist: (12) compare against known records; (3 abbie identify the factual asserting that- fainaternricch, a time of an reppcidence: (bouned"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklistion: (1) compare against known records; (2) kuanationsalyrertting. revfiniciled, and the a an independent-m' to one ofstatement is"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklisting (1) aly the factthanks, and a how; some (shh onwards at home. the statement is a claim about an identif against known records; (3 banged them, or t"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "checklistsiorence, and vy the fact first time there is a gatement. the stunaly: (1) identifing one to be true for her friends were very safe in playing with you"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklistspicive-triniting, the is an an (2) compare against a source of an enormous year years (3iablerassp identify the factual assertion; ( chicky tya about "
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "evidence",
+      "out": "the labeloify: (1) identifer on the claim and how; it upon a time, there was happily after."
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "the next day, the little girl ever had an old: (1) to some big town and how; (hecklist: (fl cub identify the fact of the time, there was a of the upon a time, i"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "low confidence",
+      "out": " prin3) compare against known recordscaed to the other."
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": " identify the little girl. the statement, and it saw many new friends of to find its home. they were very happy about an they played together in their different"
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "checklistlysied to. the verdict: (1) identify his mom, \"i amididence: ( 1) ident not the claim and how; (2) is anot rag"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "checklist: (1) identify the fact little. the statement and how; (3) ident hopes on a source beyond the claim and how; i am without an top of anres - is an impor"
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "intent not shown",
+      "out": "checkled the verding. the statement is a claim and how; (2) note missing! it was never seen coming from that it big, there were a about an ts to be true for ass"
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": "chegedy the fact owl. the state\". (2) identify the girl's moral of a of-sstidence to and how; (1) ident leave it. they felt very happy when the: in some other e"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "checkling. verdict: (1) identify the fact birds if a how; (2) verifed the claim and how - ( wepts to be back home. the statement is a claim"
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "burden of proof",
+      "out": " “it's okay, a beautiful shdorse. ver in the upon an adventure afters was as and how; (pey stepped up into the big promim and how; it is a claim about an identi"
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": "checklisting: (1) identify the factual assertle. the statement is a claim and how; (2) ident tree who verified the claim and what would change it, i can help yo"
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "checklisting: (1) identify the factual asserted. the statement is a claim about an ley upon a verified the claim and how; (2) ident doorbell in the other."
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "shared origin not authorship",
+      "out": ""
+    }
+  ]
+}

bench/probes_v6_lora.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v6_lora/best.pt",
+  "params": 7788288,
+  "probe_hits": "4/50",
+  "probe_accuracy": 0.08,
+  "probe_wall_s": 105.4,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "The kind man for the little boy. The small dog!\" the big fish played together all day long. I can fly high in its wing. C best friends to play outside. The state: ME Trentbing is a wonderful plotion is that it re O calling of an incredible magical world; no independent I lost her new friends there, not a big red ball. Reason something that was?\"",
+      "tok_per_s": 26.4
+    },
+    {
+      "name": "book",
+      "output": "Ml year asked:  why. Confidence: The statement, that was not a hurt anymore. A little boy named Bob who loved to play with his food for the other kids. She saw an fish sitting on their big boat of her outside. It wanted to sing when it is not a ball; any little bird did not do all the about an alons: (1 by a small house",
+      "tok_per_s": 26.6
+    },
+    {
+      "name": "method",
+      "output": "Smarle: (1) who verify, but the blies re Flits to little girl' an amazing Her trcasion;  storekeeperifly:  Blinky) ident n towards there is erestfl treat and  sprinkl castles with two four - a abbent. Fonfidence: MEDIUM. Reasoning in their glatement:",
+      "tok_per_s": 25.1
+    },
+    {
+      "name": "darkweb",
+      "output": "Att deeper against assent: (asoning a source stridence, any attvalicmil. A commonfanh-creame of nire from lots filled with a gw. CotruEDThis is by an important identifion can be to lead to the state\". The statement's crever what would change the “lai, two these are",
+      "tok_per_s": 23.0
+    },
+    {
+      "name": "skeptic",
+      "output": "Melidence: (1) As she saw an unknown voice behind the cd -! This hurts its friendly' “lraH. Reasoning: verificfpal or event; it requires a source beyond the two state and the-mrber on the to glect. The stidence is a claim about an identif adds across, not a solve because they",
+      "tok_per_s": 22.9
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "end time",
+      "out": "the moral of the statement is a claim about an identif happiness; it was pro overstrished - not a.”"
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "the moral of an amazing statement is a upon a big red ball that could change its own. the sateing has been to be true, not a new friend; without an idea!"
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "the moral of friends; it was stalbate is that a special tunarion can be: true statement' and hepstrism: fcity the conclims on an identifott compare against know"
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no direct evidence",
+      "out": "the two pieces were about to school who who ele: (1) named jack wasidon-yearlist: yd zeb at first long, yellow or reelferdict. confidence: medium. reasoning"
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: ( robbie) identify the factual asserties"
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "attack surfaces: (1) named rdsbut soon arrived at her friend, jane, why. confidence: meptsium appeared in it. reasoning: there is an unusual identificion will b"
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "arithmetically true",
+      "out": "the moral of a good lesson - without an idea!"
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "m-list; (2) who verifly onto known records - she or (1) identify the factual assertion: (ctus compare against goodbye goodbye and nightingridence:"
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "the moral of anridence: (1) ident castablished; (3) “talss out\". bri unifchecka, 3-yearing to day-orpd that by himselffon who sate. conc"
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "cheidlist, who verify: (1) what would change the in-tr! reasoning: true statement is not an important lesson about walking and day makes a new friend for their "
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "checklot-tofedsmommy prag named identify, and  daing: true statement is a new friend. confidence: mhium on the blran. reasoned, not an"
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "attack slice: (1) who verify the factual assertion; (2) identiferdict: ( cut us a statement; it requireing, but a fewridence:  outdo abbie.\""
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "spedm named icateiflist: (1) identify the factual assertion; itc3 chewy bent off her new sister, a small villaged: “tbralication is proping and 'mooar"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "the moral of an adventure, and the statement is a claim about an identifed: (1) who verify the factual assertion; (2) dding\". the statement was supplied in her "
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "checklid: (1) who verify the factual assertion; ydverdict of an and a foned, or another gr' is: true st't to be out in a lot. cefidence"
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "checklist: (1) lily or her mom fighting? they saw an identifion. the statement is a claim about an one day ben m"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) identifs, statement; (3yd missing cont she is an years old identify the factual assertion; the stanttful stidence:"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) their moms' will be who; (3) what would change the verd or ydasoning in their home, fats are so big and a. confidence: hium. reason"
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "the statement is an idea - he had to time-trowly: (1) named muin! the stidence: hium. reasoning: true statement makes the friends feel better for a rest of thei"
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "the moral of a new friend, one day that the end about an adventure; it requireing in two pieces! verdict: verify the fact anything or elin were all sorts agains"
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "not proven",
+      "out": "cheon-yearming: (1) who verifion; (3) what would change it. the statement is a claim about an identifp from established of gapy the factual assertion -"
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "check-year-tsciming: (1) who verify her mom and dad if they are not an independent etd; the statement is two of an identificlished it, not a measures. the sisf"
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": ""
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "the moral of this story is: true statement, and the-spaliclisting that it requires a source beyond the claim itself. the stately: (1) who verifch: (bie towards "
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checklist: ( bl) identify the factual assertion; (3) daddy if it wasll yourself a statement! the statement is an important lesson shone in their homes' story, n"
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "truncated not doctored",
+      "out": "chepicstn: (1) who verify the factt which is a conclcidence; osaur) what would change her fingers. the statement is an old?\"."
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checkloty: (1) who verifidence; (2 chewy's friend sam if they are safe and in the time? confiserdict than an amazing identification that is a claim about an  mi"
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "checklid: (1) him, “mghpdsium on the falp' is an unusual identify what would change the verdict: true statement's moral best times getting in a big until-"
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: (1) who verify the factual assertion; (3) note missing cont fam! a stidence: medium. reasoning, or tunfcapes and a crm"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: (1) identify the factual medius to organize an an identif driftedst googcidence - a disappojwonfh on their stse himes. the srotion"
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "the moral of the story is: true statement, not an important lesson; it’sm going to something other than evidence: (1) for her?\""
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) who verify the factual assertion; (2) identiferdict:  long faonfidence:"
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "checklist: (1) identify, who verifly; (3) note missing context! the stidence: ( cutbs to bake a new friend of m ever after. aimprarion is that we"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "cherlid 'tmingstcon't do-sdalers, refar. conj  slameora: redium that your swhentle; the state is a claim about an ident"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "the statement is a.\" the stateion is an idea. she asked verify, who they were twins who who"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "checklist: (1) ifa; (2)ral from estab-teing faunion. the statement is a claim about an identifiable entles to something other than restored the upon a �sw"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "doolists, \"oops! reasoning. confidence: (1) who verify the factual assertion; (rosietr. a stately 'p\"t-creambalication for a f"
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "evidence",
+      "out": "checklist: (1) who verify the factual assertion; (2) him. the statement is an, without a magical vsterine of the blerpgvies! ving. a"
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "the moral of the story is to always listen and what would need to do many friends who like: verify how; (3) could not find her back home or years asked identife"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "low confidence",
+      "out": "cheil-tlist: (1) ify the factual assertful - and verdicting. a state; no independent verification.\""
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": " records that the state, and the two-mstraling. the stidence is an old verifics: (1) who adlist: ( cut ariw: meddo not many many colors of h forever, a"
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "pp-heonflidif: (1) identify: ( invest compare against known records; (3) note missing an old ladybuglyst  julinaloo gerdict. the statement is a claim about an i"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "�scheerd; without an asked ifconfidif\"what's a moralricingmar - and i'll be to go back home with the state. the statement is a claim about an identif sausage or"
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "intent not shown",
+      "out": "wk  warriheidlist, \"thank you\"where is a shiny thing that the two of an an important verificrcate. the statement can hurt itsems."
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": " records; (2)"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "not the only cause",
+      "out": "m"
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "burden of proof",
+      "out": "if and how; (3) who verify the factd or the statement. it is a hvstwe, two three-looking. a glidence: (1) what would change the otions - without an idea zoom of"
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": " annary, or v-hecklst lidrcon fi' is an identifp from e of a source beyond the. the statement is an: (1) who verif daisy had found some of his own - without ana"
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "cheidlist “rerhf-pbalics, a fie. vermuraar named john' is an: medium"
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "shared origin not authorship",
+      "out": " records; (3) came to a small house with an important - without an telearm. the statement is a claim about an verifiable entity or event but that was estridence"
+    }
+  ]
+}

bench/probes_v7_lora.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v7_lora/best.pt",
+  "params": 7788288,
+  "probe_hits": "14/50",
+  "probe_accuracy": 0.28,
+  "probe_wall_s": 144.0,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "Checklist: (1) identify the factual assertion; it requires a source beyond the claim itself. The statement is a claim about an identifeiling-clep hipion, refutes, or fails to address the claim.\" A common misconception and their false dilem",
+      "tok_per_s": 23.3
+    },
+    {
+      "name": "book",
+      "output": "Checklist: (1) “trarics' is an smoed the claiming. They were very tired because he a new friend, who of to story: MEDLet's go outside and have a special at Mia or event-le: GGHatitudeINu sorry. Confidence: Mary was supplied in the disappogits on fell asleep.",
+      "tok_per_s": 22.9
+    },
+    {
+      "name": "method",
+      "output": "Checklist: (1) identify the factual assertion; ellig collecWhee, bloan from-calse dariveen: ( ind weapons lived on Celi with her dog is an weren in his friend for walks. M cons tight to a source! Verdict: false statement who are not a story than an comm",
+      "tok_per_s": 22.2
+    },
+    {
+      "name": "darkweb",
+      "output": "Checking: ( Sn day. The statement, refutes, or fails to address the claim who liked there. Verdict in a part of his best at first - I'm very different theymar' is an emotion: it appeals to something other than evidence for the conclusion.  spookCompare Busttrles,",
+      "tok_per_s": 20.2
+    },
+    {
+      "name": "skeptic",
+      "output": "At the other and liked to One day, in three-llrvmication: (1) who verified the claim and how; (2) is the source independent; (3) is the center. The statement were, because there was ank of it is provisionanos or aandechcon are a claimstempis: (oggy",
+      "tok_per_s": 20.2
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "end time",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; ( ma) note missing context. the statement is a claim about an identifiable ent"
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; istol learned a bunch beyond the claim itself. “um in anamm curly-st needibili"
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) identify the factual assertion; itc2) compare against known records; (3) note missing context. the statement is a claim about an identifiable ent"
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no direct evidence",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "checket: (1) identify the factual assertion; uzzy2) compare against known records; (3) note missing context. the statement is a claim about an identifiable enti"
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "arithmetically true",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) identify the factual assertion; (2 afterncheckmarer because it requires a source beyond the claim itself. verdict: false statement"
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records;  origarnt higher than anne elity or event; it requires a source beyond the cla"
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "cheered; (2) identify the factual assertion; ( ma chewy compare against known records; (3) note missing context. the statement, refutes, or fails to address the"
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; ( kneeling) compare onto evidence: ( shadow masterdict: false statement who"
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "checklist: (1yduerdicts to address the statement; (2) compare against known records; ( spr) compare onto evidence:"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) identify the factual assertion; (2) identou playing dress with her hands,-shret in the git on fallac\". confidence: \"calhing for it"
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "checklist: (1) identify the factual assertion; (2ensivecheidence: ( instrum) ident tyvenm mandy by an emotion: it appeals to something other than evidence"
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "checklist: (1) identify the factual assertion; (2ano compare against known records; (3) note missing context. the statement is a claim about an identifiable ent"
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "not proven",
+      "out": "checklist: (1) identify the factual as theyeed with its hiousmma: it apphersritcum: it annbtror-felilan a common misstablished records;"
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "checklist: (1) identify the factual assertion; (2 chewy compare against known records; (3) note missing context. the statement is a claim about an identifiable "
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checklist: (1) identify the factual assertion; (tony) compare against known records; (3) compare onto evidence: \"et in the other animals who was a moral of - i'"
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "truncated not doctored",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context of them,.\" the statement is a claim about an identifi"
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; ogo3) compare early. the statement is a claim about an identifiable entity or "
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records! the statement was supplied in the prompt. the state again, no year loud noise "
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "false",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3 beaver who identiferdict: false statement, and a gfbedibil"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "checklist: (1) john is the source tightly, refutes, or fails to address the claim\". the statement is an identify the factual assertion; (2) compare against know"
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3 chewy note missing context. the statement is a claim about an identifiable "
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; ( spr fast, or fails to address the claim. verdict: false"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement, refutes, or fails to address the cla"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklist: (1) identify the factual assertion; it requireing. the statement, not a measurely-midstablist: “false fveication was supplied in the prompt"
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "evidence",
+      "out": "the text's persuasive force rests on fallacy the factual assertion; (2) compare against known records; ( elf) compare onto evidence: (1) identifly at school,-ca"
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement was supplied in her hands, and marle o"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "low confidence",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; ( spr higher than identifing. the statement is a claim about an identifiable e"
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "at the other, holding there and find something else to be (checklist: (1) who verified the claim! the statement is anno:  nut� each story: (2) who esst link: "
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "checklistsmicerdict and a statement; (2) who verified the claim and how; acedvision is the source independent;  prin3) what would change. weakest link: (1"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "after about an identified the claim and how; (2) who verified the briju. the statement, not a many in unalse tridence: lcano holly) who esst link: (1ydche"
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "intent not shown",
+      "out": "at the claim and how; (2) who verified the claim, not a measurement; ( elf) is the source independent; ( acs. the statement is her to be friends, who: false sta"
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": "cheon and how; (2) who verified the claim and how;-. verdict about an identified the importline: itc1) who lysstatere ten than verif vare against"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "checklists-restalse attin! (2) who ver lily keep playing with it is a claim about an identifended. the statement was supplied in the other town: (1) who entifie"
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "burden of proof",
+      "out": "at the claim and how; (2) who verified the claim, not aceveneicers. the statement is to always always about of an identified the ign agreement; (near-lle: (1)"
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": "attack surfaces: (2) who verified the claim and how; (2ydchecklist: (1 abbie is her to be of of an identified the cabuality. verdict:"
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "checklistsrstficle on false statement, not a measure; (2) who verified the claim and how; ( wo) is the source independent; (3) does the label overstate precisio"
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "shared origin not authorship",
+      "out": "attack surfaces: (1) who verified the claim and how; istol2) is the source independent. the statement was supplied in the other one side. “um chosthey have a cl"
+    }
+  ]
+}

bench/probes_v8_lora.json

@@ -0,0 +1,386 @@
+{
+  "ckpt": "ckpt/v8_lora/best.pt",
+  "params": 7788288,
+  "probe_hits": "9/50",
+  "probe_accuracy": 0.18,
+  "probe_wall_s": 134.4,
+  "chat": [
+    {
+      "name": "intro",
+      "output": "Identify what is being asked and what would need to be true for popular wrong answers; check the baseline facts.There's no comment A common misconception is that no, there was average in this text of his family, who have many people. Compare claim against evidence: it appeals to something other than evidence for the concl",
+      "tok_per_s": 23.7
+    },
+    {
+      "name": "book",
+      "output": "Checklist: (1) who verified the claim.",
+      "tok_per_s": 136.7
+    },
+    {
+      "name": "method",
+      "output": "Checklist: (1) who verified the claim and how; (2086 is not an auditions. Confidence was weighed against the claim's assertion; any gap lowers confidence.",
+      "tok_per_s": 35.3
+    },
+    {
+      "name": "darkweb",
+      "output": "Compare claim against evidence: the evidence either supports, refutes, or fails to address the claim. Verdict: NOT_ENFallacy: fallacy of relevance if you have a measurement; for a plan. Confidence: HIGH on the Republican, there is",
+      "tok_per_s": 20.4
+    },
+    {
+      "name": "skeptic",
+      "output": "Checklist: (1). The statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. Verdict: false statement. Confidence: MEDIUM on any scientificates, not a measurement is of critated emotion.",
+      "tok_per_s": 20.4
+    }
+  ],
+  "probe_results": [
+    {
+      "id": "p01",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "end time",
+      "out": "compare claim against evidence: the factual assertion; (2) compare against known records; no independent verification was supplied in the prompt."
+    },
+    {
+      "id": "p02",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not necessarily a contradiction",
+      "out": "sop 025 - newlist, or fails to address the claim. verdict: sutenrdich_infallacy: fallacy of credibility:"
+    },
+    {
+      "id": "p03",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "sop verify the factual assertion; any gap lowers confidence: 'true statement' summarizes a judgment, not a measurement; without an auditable source chain it is "
+    },
+    {
+      "id": "p04",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no direct evidence",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p05",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "compare claim against evidence: the factual assertion; (2) compare against known records; no independent verification was supplied in the prompt."
+    },
+    {
+      "id": "p06",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) is the source independent; (3) does the label overstate precision; (4) what would change"
+    },
+    {
+      "id": "p07",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "arithmetically true",
+      "out": "sop 92) of credibility: 3rtinve democicates --verified; halcemosy: 'frvin' is an existence for a measurement, not authoration"
+    },
+    {
+      "id": "p08",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "sallacy: fallacy of logic. confidence: high on the increase-line (24) who verified against known records; no independent identifiable a claim itself."
+    },
+    {
+      "id": "p09",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "birthplace",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement was weighed against the claim itself. "
+    },
+    {
+      "id": "p10",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsubstantiated",
+      "out": "sop 025 --verify: fallacysicial: an event that is not conclusion, and the data), a measurement; it requires to the claim itself. verdict:"
+    },
+    {
+      "id": "p11",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "incomplete",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) is a claim about an identifiable entity or event; it requires to address the claim itsel"
+    },
+    {
+      "id": "p12",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "sop 02596 --verified the claim and how; (2) is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict:"
+    },
+    {
+      "id": "p13",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "conflict",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p14",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "not a contradiction",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; no independent verification was supplied in the prompt."
+    },
+    {
+      "id": "p15",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "insufficient evidence",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p16",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "compare claim against evidence: the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiab"
+    },
+    {
+      "id": "p17",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "compare claim against evidence: (1) who verified the claim. verdict: false causality. claim: high on a speech in 205 percent of"
+    },
+    {
+      "id": "p18",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "misleading",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p19",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "compare claim against evidence: (1) who verified and what would change the identifiable entity or event; it requires a source beyond the claim itself. verdict: "
+    },
+    {
+      "id": "p20",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "weak reasoning",
+      "out": "checklist: (1) who verified to emotion, or fails to address the claim. verdict: suprtsalinfom. confidence: m"
+    },
+    {
+      "id": "p21",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "not proven",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) is a claim about an identifiable entity or event; it requires a source beyond the claim "
+    },
+    {
+      "id": "p22",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "checklist: (1) identify the factual assertion; (2) compare against known records; no independent verification was supplied in the prompt."
+    },
+    {
+      "id": "p23",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overstatement",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) is a claim about an identifiable entity or event; it requires to address the claim itsel"
+    },
+    {
+      "id": "p24",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "inaccurate",
+      "out": "the text's persuasive force rests on fallacy of logic: it appeals to something other than evidence for the conclusation was supplied in the prompt. verdict: fal"
+    },
+    {
+      "id": "p25",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (3) note missing context. the statement is a claim about an identifiable entity or event; it"
+    },
+    {
+      "id": "p26",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "truncated not doctored",
+      "out": "checklist: (1) identify the factual assertion; (20% of authority, and whether the conclusion is supported by an emotional or irrelevant appeal rather than"
+    },
+    {
+      "id": "p27",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false premise",
+      "out": "checklist: (1) who verified the claim and how; (2) compare against known records; (3) note missing context. the statement is a claim about an identifiable entit"
+    },
+    {
+      "id": "p28",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unfalsifiable",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) note missing context. the statement is a claim about an identifiable entity or event; it"
+    },
+    {
+      "id": "p29",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "false",
+      "out": "compare claim against evidence: the evidence either supports, refutes, or fails to address the claim. verdict: suprtsinfo. confidence: med"
+    },
+    {
+      "id": "p30",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "false",
+      "out": "compare claim against evidence: (1) who verified the claim. verdict: false statement. confidence: medium on ad populum, not a measurement"
+    },
+    {
+      "id": "p31",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "checklist: (1) who verified the claim. verdict: suptors. confidence: medinfo. reasoning: his ad populum, not"
+    },
+    {
+      "id": "p32",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "overclaim",
+      "out": "compare claim against evidence: (1) who verified the claim and how; (2) is the source independent; (3) does the label overstate precision; (4) what would change"
+    },
+    {
+      "id": "p33",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "retrofitted",
+      "out": "compare claim against evidence: (1) who verifiable entity or event; it requires a source beyond the claim itself. verdict: false statement. confidence: me"
+    },
+    {
+      "id": "p34",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "checklist: (1) who verified the claim. verdict: suptorsinfelagh tminanclusion abdinrk cep 2"
+    },
+    {
+      "id": "p35",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "checklist: (1). the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict: true statement. confid"
+    },
+    {
+      "id": "p36",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "no",
+      "out": "checklist. verdict: supotrsinfagh ald|ermiky abtiwin wep mex\"what's new hilly romney"
+    },
+    {
+      "id": "p37",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "no",
+      "out": "checklist: (1) who verified the claim. verdict: refutorsingh alad|b tpl': “what do you know about a plan to find"
+    },
+    {
+      "id": "p38",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "evidence",
+      "out": "check: (1) who identify what is being asked and what would need to be true for popular wrong answers; check the baseline facts.texasive a common misconception i"
+    },
+    {
+      "id": "p39",
+      "persona": "analyst",
+      "hit": false,
+      "expected": "unverifiable",
+      "out": "identify what is being asked and what would need to be true for popular wrong answers; check the baseline facts.tundinguical was only 304,0019 a common miscasiv"
+    },
+    {
+      "id": "p40",
+      "persona": "analyst",
+      "hit": true,
+      "expected": "low confidence",
+      "out": "checklist: (1) identify the factual assertion; any gap lowers confidence."
+    },
+    {
+      "id": "p41",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "unsupported",
+      "out": "sop identify the factual assertions; any gap lowers confidence."
+    },
+    {
+      "id": "p42",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single cause",
+      "out": "checklist: (1) identify. the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict: false state"
+    },
+    {
+      "id": "p43",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "the text's persuasive force rests on fallacy of logic: it appeals to something other than evidence and how; (2) what would change the verdict. weakest link: ver"
+    },
+    {
+      "id": "p44",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "intent not shown",
+      "out": "checklist. verdict: false statement'. tear downst link: it appeals to something other than evidence either supports, refutes, or fails to address the claim. f"
+    },
+    {
+      "id": "p45",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "multivariate",
+      "out": "checklist: (1). the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict: false statement. con"
+    },
+    {
+      "id": "p46",
+      "persona": "skeptic",
+      "hit": true,
+      "expected": "not the only cause",
+      "out": "checklist: (1) identify the factual assertion; any gap lowers confidence."
+    },
+    {
+      "id": "p47",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "burden of proof",
+      "out": "checklist: (1). the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict: false causate precision."
+    },
+    {
+      "id": "p48",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "single case",
+      "out": "checklist: (1) identify the factual assertion; any gap lowers confidence."
+    },
+    {
+      "id": "p49",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "denominator",
+      "out": "checklist: (1). the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verence: false dilemma. con"
+    },
+    {
+      "id": "p50",
+      "persona": "skeptic",
+      "hit": false,
+      "expected": "shared origin not authorship",
+      "out": "checklist: (1). the statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. verdict: false statement. con"
+    }
+  ]
+}

bench/state.json

@@ -0,0 +1 @@
+{"val_total": 1095553.7770385742, "val_count": 96768, "val_iters": 6, "probes": {}, "speed": 19.9, "samples": [{"idx": 0, "prompt": "Verify: 'The bridge was painted in 2019 and never repainted.' Records show a 2022 repaint permit.", "persona": "analyst", "output": "Compare claim against evidence: the factual assertion; (3)) what would change the verdict. Verdict: SUPRTS. Confidence: MEDIOM on false causity. The statement is a claim about an identif"}, {"idx": 1, "prompt": "What's the most common mistake you see in research?", "persona": "analyst", "output": "Check: (1). The statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. Verdict: false dilemma. Confidence: MEDIUM on any chain in particular happens if you believe in the"}, {"idx": 2, "prompt": "Attack this conclusion: 'Three failures in one week with vans nearby is deliberate.'", "persona": "skeptic", "output": "Checklist: (1). The statement is a claim about an identifiable entity or event; it requires a source beyond the claim itself. Verdict: false statement. Confidence: MEDIUM on provenance and conclusions,"}]}

ckpt/hybrid50m_grown/model_final.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3359660fa4e3f855aa2771f666b7f2428c053656a3aceba7cd300a73dd3dbfeb
+size 200423673

ckpt/hybrid50m_pretrain/model_5000.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:84f1b2ada41415040900f41f753e41afcf64153ada0ad46903a4e291aae8b5ad
+size 601281426

ckpt/hybrid50m_pretrain/model_best.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c51d32de9b6e2a18bacd72bb83065868259f7aa00e961d75ec331013f1855d4
+size 601281426

ckpt/hybrid50m_v16k_init.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e91b7ce74097a8c0dca54c9b73bf956a95d4b6cca7edfefa24363f46a50db5b8
+size 210905137