Add Codex judge evidence

AGENT_TRACE.md

@@ -44,12 +44,12 @@ Codex helped:
 Current decisions:
 
 - Deployed model: `openbmb/MiniCPM5-1B`.
-- Deployed adapter: `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v4`.
+- Deployed adapter: `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8`.
 - Deployed backend: Transformers on ZeroGPU.
 - Local/eval model path: llama-cpp-python remains available for GGUF models.
 - Fine-tuning: completed through a Modal-trained MiniCPM5-1B LoRA adapter.
 - Primary badges: Off-Brand, Sharing is Caring, Field Notes.
-- Defensible badges: Off the Grid and Tiny Titan, documented carefully.
+- Defensible badges: Tiny Titan and Well-Tuned, documented carefully.
 - Sponsor target: OpenBMB, because MiniCPM is central to the app.
 
 ## 2026-06-07
@@ -57,7 +57,7 @@ Current decisions:
 Codex helped:
 
 - run and compare MiniCPM5-1B LoRA evals against earlier 8B adapter evidence
-- promote `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v4` as the final deployed adapter
+- promote `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v4` as the first strong 1B deployed adapter candidate
 - commit 320-case and 394-case hard guarded eval reports
 - update README, setup, eval, training, and honest-submission evidence
 - refine the custom Gradio Server UI for readability, elderly-friendly wording, copy-to-trusted-person behavior, and final model disclosure
@@ -71,3 +71,33 @@ Final model evidence:
 - `0` unsafe action violations
 - `0` invalid predictions
 - `0` model errors
+
+## 2026-06-09 / 2026-06-10
+
+Codex helped:
+
+- add fresh public-pattern calibration data for wrong-number crypto/trading, marketplace money movement, task/job scams, MFA-code theft, toll/tax/benefit notices, and safe family/logistics contrasts
+- train and evaluate the MiniCPM5-1B LoRA v8 path on Modal
+- diagnose preemption during a long Modal eval and preserve the final successful run as public evidence
+- tighten the deterministic safety guard for wrong-number investment grooming without over-promoting ordinary family/school logistics
+- add regression tests for guard behavior
+- promote `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8` as the final deployed adapter
+- update the Space README, model card, dataset card, collection notes, and final submission evidence so v8 is consistently framed as final
+- add `CODEX_JUDGE_EVIDENCE.md` to map Codex-attributed commits to files, final metrics, and public artifacts
+
+Final v8 model evidence:
+
+- 632-case hard guarded eval: `579/632` risk accuracy (`91.61%`)
+- `0` dangerous-as-safe
+- `0` dangerous-as-needs-check
+- `0` safe-as-dangerous-or-suspicious
+- `0` unsafe action violations
+- `0` invalid predictions
+- `0` model errors
+
+Public final artifacts:
+
+- Space: https://huggingface.co/spaces/build-small-hackathon/jawbreaker
+- Model: https://huggingface.co/build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8
+- Dataset/eval bundle: https://huggingface.co/datasets/build-small-hackathon/jawbreaker-scam-defense-data
+- Collection: https://huggingface.co/collections/build-small-hackathon/jawbreaker-6a263632dcd0b6d41ca914ff

CODEX_BUILD_LOG.md

@@ -67,9 +67,9 @@ Measured results:
 
 ## MiniCPM5-1B LoRA v4
 
-Codex helped turn the Tiny Titan experiment into the final model path.
+Codex helped turn the Tiny Titan experiment into the first strong MiniCPM5-1B production candidate.
 
-The final deployed path is:
+The v4 path was:
 
 - Base model: `openbmb/MiniCPM5-1B`
 - Adapter: `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v4`
@@ -90,3 +90,43 @@ The completed hard guarded eval evidence:
 - `0` model errors
 
 Codex also helped add the committed eval reports under `eval/reports/`, update the Space and GitHub code defaults, and keep the Hugging Face Space history separate from GitHub history through cherry-picked Space sync commits.
+
+## MiniCPM5-1B LoRA v8
+
+Codex helped extend the 1B path after fresh scam-pattern evals exposed two failure modes:
+
+- wrong-number crypto / gold / trading grooming could be softened below `dangerous`
+- ordinary family, school, pharmacy, and logistics messages could be over-called
+
+The v8 path added:
+
+- `training/generate_v7_data.py` and `training/generate_v8_data.py` for fresh public-pattern and failure-driven calibration
+- `training/data/train_v8.jsonl`, `dev_v8.jsonl`, and `test_v8.jsonl`
+- `eval/hard_v8_eval.jsonl`
+- safety-guard calibration for wrong-number investment grooming
+- regression tests in `tests/test_app_guard.py`
+- the final report `eval/reports/jawbreaker-minicpm5-1b-lora-v8-hard632-safetyguard-v4.json`
+- `MODEL_CARD_MINICPM5_LORA_V8.md`
+- `CODEX_JUDGE_EVIDENCE.md`
+
+The final deployed path is:
+
+- Base model: `openbmb/MiniCPM5-1B`
+- Adapter: `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8`
+- Runtime: Transformers on ZeroGPU
+- Training and eval: Modal A100
+
+The completed hard guarded eval evidence:
+
+- 632 cases
+- `91.61%` risk accuracy
+- `88.77%` scam type accuracy
+- `90.69%` mean tactic recall
+- `0` dangerous-as-safe
+- `0` dangerous-as-needs-check
+- `0` safe-as-dangerous-or-suspicious
+- `0` unsafe action violations
+- `0` invalid predictions
+- `0` model errors
+
+Codex also helped update the Space README, model card, dataset card, collection notes, and public documentation so v8 is presented as the final judged model and v4 is retained as comparison evidence.

CODEX_JUDGE_EVIDENCE.md

@@ -0,0 +1,87 @@
+# OpenAI Codex Judge Evidence
+
+This file is a compact evidence map for the OpenAI/Codex track. It is meant to make the Codex contribution auditable from the public repository without relying on private chat logs.
+
+## What Codex Contributed
+
+Codex materially contributed across the product, model, eval, and submission surface:
+
+- App runtime and UI: `app.py`, `style.css`, `jawbreaker/render.py`
+- Scam analysis contract: `jawbreaker/schema.py`, `jawbreaker/analyzers.py`, `jawbreaker/prompt.py`
+- Safety guard and schema repair: `jawbreaker/analyzers.py`, `tests/test_app_guard.py`, `tests/test_schema.py`
+- Eval harness: `eval/run_eval.py`, `eval/README.md`, `eval/reports/`
+- Modal training/eval workflow: `training/train_lora.py`, `training/modal_train.py`, `training/modal_eval.py`
+- Data generation/calibration: `training/generate_jawbreaker_data.py`, `training/generate_v3_data.py`, `training/generate_v4_data.py`, `training/generate_v5_data.py`, `training/generate_v6_data.py`, `training/generate_v7_data.py`, `training/generate_v8_data.py`
+- Public evidence packaging: `README.md`, `MODEL_CARD_MINICPM5_LORA_V8.md`, `CODEX_BUILD_LOG.md`, `AGENT_TRACE.md`, `FIELD_NOTES.md`, `HONEST_SUBMISSION.md`
+
+## Commit Evidence
+
+Recent Codex-attributed commits include:
+
+| Commit | Evidence |
+| --- | --- |
+| `28030af` | Adds the final MiniCPM5-1B LoRA v8 model card. |
+| `f468585` | Promotes v8 as the final safety-calibrated model and commits the 632-case report. |
+| `7a67ead` | Covers indirect wrong-number investment grooming guard gaps. |
+| `7b5bbb2` | Calibrates wrong-number investment guard behavior. |
+| `74617f3` | Tightens safety guard calibration for benign family/school messages. |
+| `1b67b5e` | Adds v8 failure-driven calibration data. |
+| `d5215ed` | Extends Modal eval timeout for larger guarded evals. |
+| `3407348` | Adds fresh public-pattern calibration data. |
+| `05e76bd` | Publishes the custom kitchen-table UI. |
+
+Each listed commit includes:
+
+```text
+Co-authored-by: Codex <codex@openai.com>
+```
+
+## Final Model Decision
+
+Final judged model:
+
+- Base: `openbmb/MiniCPM5-1B`
+- Adapter: `build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8`
+- Live app: https://huggingface.co/spaces/build-small-hackathon/jawbreaker
+- Model card: https://huggingface.co/build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8
+- Dataset/eval bundle: https://huggingface.co/datasets/build-small-hackathon/jawbreaker-scam-defense-data
+
+Final guarded Modal eval:
+
+- Suite: `eval/hard_v8_eval.jsonl`
+- Report: `eval/reports/jawbreaker-minicpm5-1b-lora-v8-hard632-safetyguard-v4.json`
+- Cases: `632`
+- Risk accuracy: `579/632` (`91.61%`)
+- Scam type accuracy: `561/632` (`88.77%`)
+- Mean tactic recall: `90.69%`
+- Dangerous as safe: `0`
+- Dangerous as needs-check: `0`
+- Safe as dangerous or suspicious: `0`
+- Unsafe action violations: `0`
+- Invalid predictions: `0`
+- Model errors: `0`
+
+## Why This Matters For Codex Judging
+
+The Codex contribution was not limited to boilerplate. Codex helped create the core engineering loop:
+
+1. Build a working Gradio/Gradio Server scam-defense app.
+2. Define a strict JSON contract for the small model.
+3. Generate and publish synthetic/sanitized training and eval data.
+4. Train and evaluate MiniCPM LoRA adapters through Modal.
+5. Identify failure modes from eval output.
+6. Add targeted calibration data and deterministic safety guards.
+7. Re-run hard evals and promote the safer model.
+8. Package the final app, model, dataset, collection, model card, and README evidence.
+
+## Local Verification
+
+Useful checks:
+
+```bash
+git log --format=full --grep='Co-authored-by: Codex'
+python3 -m pytest tests/test_app_guard.py tests/test_schema.py tests/test_eval_dataset.py
+python3 -m json.tool eval/reports/jawbreaker-minicpm5-1b-lora-v8-hard632-safetyguard-v4.json
+```
+
+The public repository, Space, model card, dataset, and collection are the intended judge-facing evidence. Private chat transcripts are not required to verify the build.

README.md

@@ -49,7 +49,7 @@ Scam defense for someone you love.
 - **Backyard AI:** a practical scam-defense safety card for non-technical people and their families.
 - **Best MiniCPM Build / Tiny Titan / Well-Tuned:** `openbmb/MiniCPM5-1B` + [Jawbreaker LoRA v8](https://huggingface.co/build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8), evaluated on a 632-case hard suite with **0 dangerous undercalls** and **0 safe overcalls**.
 - **Best Use of Modal:** Modal A100 was used for LoRA training and guarded eval runs; see [`training/modal_train.py`](training/modal_train.py), [`training/modal_eval.py`](training/modal_eval.py), the [`632-case v8 report`](eval/reports/jawbreaker-minicpm5-1b-lora-v8-hard632-safetyguard-v4.json), plus the earlier [`394-case v4 report`](eval/reports/jawbreaker-minicpm5-1b-lora-v4-hard394-guarded.json).
-- **Best Use of Codex:** Codex-attributed commits plus [`AGENT_TRACE.md`](AGENT_TRACE.md) and [`CODEX_BUILD_LOG.md`](CODEX_BUILD_LOG.md), with file-level contribution notes below.
+- **OpenAI / Best Use of Codex:** Codex-attributed commits plus [`CODEX_JUDGE_EVIDENCE.md`](CODEX_JUDGE_EVIDENCE.md), [`AGENT_TRACE.md`](AGENT_TRACE.md), and [`CODEX_BUILD_LOG.md`](CODEX_BUILD_LOG.md), with file-level contribution notes below.
 - **Off Brand / Sharing is Caring / Field Notes:** custom candy-brutalist Gradio UI, public [dataset/eval bundle](https://huggingface.co/datasets/build-small-hackathon/jawbreaker-scam-defense-data), and [`FIELD_NOTES.md`](FIELD_NOTES.md).
 - **Submission package:** [Live Space](https://huggingface.co/spaces/build-small-hackathon/jawbreaker), [model](https://huggingface.co/build-small-hackathon/jawbreaker-minicpm5-1b-lora-v8), [dataset](https://huggingface.co/datasets/build-small-hackathon/jawbreaker-scam-defense-data), and [collection](https://huggingface.co/collections/build-small-hackathon/jawbreaker-6a263632dcd0b6d41ca914ff).
 
@@ -73,7 +73,7 @@ The problem is specific: scam messages now arrive as urgent, personal, plausible
 - Public dataset/eval bundle: https://huggingface.co/datasets/build-small-hackathon/jawbreaker-scam-defense-data
 - Hugging Face collection: https://huggingface.co/collections/build-small-hackathon/jawbreaker-6a263632dcd0b6d41ca914ff
 
-## Built With Codex
+## Built With OpenAI Codex
 
 This project is being built with OpenAI Codex in the Codex desktop app. Codex is being used for planning, implementation, eval design, Gradio UI iteration, testing, deployment, and submission documentation.
 
@@ -81,6 +81,7 @@ Codex evidence:
 
 - Public GitHub repo linked from this Space README.
 - Codex-attributed commits are included for build work.
+- `CODEX_JUDGE_EVIDENCE.md` maps Codex-attributed commits to concrete files, model/eval decisions, and final public artifacts.
 - Codex scaffolded and iterated on `app.py`, the custom Gradio Server UI, `jawbreaker/` analyzer/schema/render modules, `eval/run_eval.py`, `training/train_lora.py`, `training/modal_train.py`, `training/modal_eval.py`, and the public submission docs.
 - `AGENT_TRACE.md` records the development process.
 - `FIELD_NOTES.md` records product and technical decisions.
@@ -152,7 +153,7 @@ Training/eval artifacts:
 | --- | --- | --- |
 | Backyard AI | Targeted | Practical scam-defense app for someone close, with a focused safety workflow. |
 | Best MiniCPM Build | Targeted | `openbmb/MiniCPM5-1B` is the core runtime model, with a published Jawbreaker LoRA adapter. |
-| Best Use of Codex | Targeted | Public GitHub repo includes Codex-attributed commits plus `AGENT_TRACE.md` and `CODEX_BUILD_LOG.md`. |
+| OpenAI / Best Use of Codex | Targeted | Public GitHub repo includes Codex-attributed commits plus `CODEX_JUDGE_EVIDENCE.md`, `AGENT_TRACE.md`, and `CODEX_BUILD_LOG.md`. |
 | Best Use of Modal | Targeted | Modal A100 was used for PEFT/LoRA training and guarded eval runs across the MiniCPM calibration path; see `training/modal_train.py`, `training/modal_eval.py`, and the committed 632/394/320-case eval report files. |
 | Tiny Titan | Targeted | The deployed model is `openbmb/MiniCPM5-1B`, well under the 4B badge threshold. |
 | Off Brand | Targeted | Custom Gradio UI beyond the stock component look. |